DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 8/27/2026 has been entered.
Status of Claims
Claims 1-18, and 23-24 are pending of which claims 1, 11 and 16 are in independent form.
Claims 1-18, and 23-24 are rejected under 35 U.S.C. 103.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-18, 21, and 22 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Regarding 35 USC 101 Abstract idea:
Applicant’s arguments, see “Remarks”, filed on 8/27/2026, with respect to 35 USC 101 (Abstract Idea) have been fully considered and are persuasive. The 35 USC 101 of claims 1-18, 21, and 23-24 has been withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 5, 6, 7, 10-12, 15-17, 23, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Moustafa; Hassnaa et al. (US 20220335340 A1) [Moustafa] in view of Malladi; Sastry KM et al. (US 20240370251 A1) [Malladi] in view of Luo; Xiyang et al. (US 20230362399 A1) [Luo].
Regarding claims 1, 11, and 16, Moustafa discloses, a method for managing data in a distributed system, the method being performed by a first data processing system configured as a stream manager provided in the distributed system and comprising: obtaining, by an entity that is remote to a data user and from the data user, a data stream intended for the data user (the edge gateway may receive a data stream including one or more data packets from a source ¶ [0159], the edge gateway 802 may receive the data stream from the cloud data center ¶ [0160]-[0161], data streams ingested by at least one other super node ¶ [0207], [0210], [0215]-[0220], ingest at least a portion of the target data stream ¶ [0394], [0405], [0418], [0393]), wherein the entity is the stream manager (metadata management by generating and managing metadata corresponding to a data stream ¶ [0197], [0221], digital rights management (DRM) circuitry to manage the digital rights of data streams ¶ [0226], metadata/data enrichment manager 640 … execute metadata creation and/or post-processing routines intended to extract context and meaning from the source stream/files ¶ [0133], resource manager orchestration circuitry can identify one or more preferred nodes as super nodes to monitor data streams between data sources and one or more data stream consumers ¶ [0202], resource manager orchestration circuitry can effectuate identification and mitigation of an ethical divergence in a data stream by reducing utilization of resource(s) ¶ [0271]), the data user is associated with a second data processing system that will use the data stream to provide computer implemented services (client endpoints (in the form of mobile devices, computers, autonomous vehicles, business computing equipment, industrial processing equipment) ¶ [0082], endpoint client hardware device ¶ [0144], endpoint layer ¶ [0153], the endpoint devices 832, 834, 836 ¶ [0158]-[0164]. Also see ¶ [0046], [0393]), and the data stream originates from a data originating device (data streamed from a data source ¶ [0046], receiving data stream from a source ¶ [0159], [0160], source node sourcing the target data stream ¶ [0309]) that sends the data stream indirectly to the second data processing system through the stream manager (gateway receives stream from source and forwards to target device ¶ [0159]. Stream passes through gateway between source and destination ¶ [0160], stream flows from source node to consumer nodes ¶ [0046]);
filtering, by the entity, the data stream for relevant content to the data user to obtain a filtered data stream (In some examples, data in a data stream may allow layering characteristics, which can be utilized to filter data or modify data during a mitigation stage. For example, image data may relate to visual map data that has multiple layers (and therefore, potentially multiple filters) such as a first/highest map layer that shows only boundaries and roads, a next map layer may include buildings, a next map layer may include names of roads and buildings, a next map layer may add satellite imagery as an overlay but only with a pixel granularity that allows for detecting/discerning objects that are greater than 100 feet across, the next several map layers may tighten the focus to allow object detection at smaller granularities but certain classified objects within the visual map data may be blurred out or blacked out for privacy purposes, and a final map layer may reveal the blurred/blacked out areas. Each of these visual map layers may be associated with a filter at a node (e.g., a super node) for filtering out consumption requests or filtering in data monitoring of such data/data streams ¶ [0049]. Also see ¶ [0045], [0057]-[0058], [0067], [0225], [0264], [0338], [0346]);
signing, by the entity, the filtered data stream to obtain a signed filtered data stream ( DRM management of target data stream…tag data in stream with metadata…hash tracking information into the data stream…implement blockchain for target data stream ¶ [0226], inject hashed metadata tags into the stream ¶ [0197], examples disclosed herein include AI/ML algorithms to learn the data content type and learn the nominal traffic on the network and node behavior (e.g., system and data access, data transfer and modification, etc.) and detect significant, periodic, unusual changes to nominal conditions and flag alerts. In some examples, the ADM system may include and/or otherwise implement an example contextual metadata/event-chain correlation manager that generates a graph node representation of the data stream based on multiple classified topics and objects detected and/or incorporates relationships/affinity to one another ¶ [0063]. For example, a content type of the data stream may include a characteristic classifying the content within the data stream as image data, audio data, textual data, telemetry data (e.g., sensor data, etc.), or any other type of data ¶ [0215], metadata tags and hashed tracking information ¶ [0354]-[0356]. Also see ¶ [0050], [0060], [0168], [0169], [0403], [0414]);
analyzing, by the entity, the signed filtered data stream to obtain metadata for the signed filtered data stream that is relevant to at least one use of the data stream by the data user ( For example, flags associated with such characteristics may be in the headers of data packets within the data stream. In some examples, data within the data stream may be tagged with metadata describing such characteristics ¶ [0050]. In some examples, the data consumption tracker circuitry 1316 can implement the data ingestion manager 606 of FIG. 6. In some examples, the data consumption tracker circuitry 1316 tracks data by filtering a target data stream to obtain metadata. In some examples, the data consumption tracker circuitry 1316 analyzes the target AI application node behavior corresponding to obtained metadata. In some examples, the data consumption tracker circuitry 1316 adds to a metadata count and a target AI application behavior count for obtained metadata in the target data stream and the corresponding behavior. In some examples, the data consumption tracker circuitry 1316 generates an alert to be processed by the data usage monitoring circuitry 1300, such as in a designated alert processing node, when the metadata count and a target AI application behavior count for obtained metadata satisfies a threshold value ¶ [0225]-[0226]. In some examples, the ADM system may include and/or otherwise implement an example contextual metadata/event-chain correlation manager that generates a graph node representation of the data stream based on multiple classified topics and objects detected and/or incorporates relationships/affinity to one another. In some such examples, the contextual metadata/event-chain correlation manager may enable more comprehensive comparisons of data and data streams to identify closely correlated content which may represent patterns in detection or targets to call out as non-random ¶ [0063]. Also see ¶ [0168], [0197], [0208], [0306]-[0309], [0346]-[0348], [0395]-[0396]);
packaging, by the entity, the signed filtered data stream with the metadata to obtain an enhanced stream (In the illustrated example, the ADM system 600 includes the metadata/data enrichment manager 640 to schedule and/or execute metadata creation and/or post-processing routines intended to extract context and meaning from the source stream/files to enhance source files to decrease noise and/or clarify/focus subjects of interest. In some examples, the online metadata agent may access existing metadata or enhancement functionality within a node or launch a selected algorithm package to perform real time metadata/enhancement actions on the data stream and create a source-file linked metadata record that may be passed to the data query manager 610 for incorporation and synchronization with other authorized and/or relevant instances of the data query manager 610. In the example of source file enhancement, the original file may be archived and linked with appropriate metadata record while the modified file is returned to the requestor ¶ [0133], [0134], [0137]. Also see ¶ [0120], [0169], [0173], [0197], [0346], [0354], [0394]); and
providing, by the entity, the enhanced stream to the second data processing system (gateway receives stream from source and transmits it to target service ¶ [0159], gateway forwards stream to target destination ¶ [0160], stream transmitted from source node to consumer node ¶ [0046]. Target AI application node consumed target data stream ¶ [0393]-[0405]) to cause the second data processing system to provision the computer implemented services using the enhanced stream (virtual Edge instances include: a first virtual Edge 432, offered to a first tenant (Tenant 1), which offers a first combination of Edge storage, computing, and services; and a second virtual Edge 434, offered to a second tenant (Tenant 2), which offers a second combination of Edge storage, computing, and services. The virtual Edge instances 432, 434 are distributed among the Edge nodes 422, 424, and may include scenarios in which a request and response are fulfilled from the same or different Edge nodes. The configuration of the Edge nodes 422, 424 to operate in a distributed yet coordinated fashion occurs based on Edge provisioning functions 450 ¶ [0084], orchestration of multiple applications…fulfill requests and responses of client endpoints ¶ [0088], provisioning it with resources and applications ¶ [0090]-[0091], metadata/data enrichment manager …, the metadata or enhancement request routine may be configured and/or otherwise generated to take inputs from a user or process/application to articulate the types of metadata/enhancement ¶ [0133], perform real time metadata/enhancement actions on the data stream ¶ [0134], also see ¶ [0197], [0394], consumer node ¶ [0046], target service/appliance [0159]-[0160], target AI application node ¶ [0393], executes workloads ¶ [0202], orchestrated resources ¶ [0271]);
data is added to the metadata (system 600 includes the metadata/data enrichment manager 640 to schedule and/or execute metadata creation and/or post-processing routines intended to extract context and meaning from the source stream/files to enhance source files to decrease noise and/or clarify/focus subjects of interest ¶ [0133]-[0134]; In some examples, the interface circuitry 1302 tags portions of carries out metadata management by generating and managing metadata corresponding to a data stream and/or to an AI application node attempting to consume the data stream ¶ [0197]; The data usage monitoring circuitry 1300 of the illustrated example includes the metadata manager circuitry 1308 to manage metadata related to a data stream ¶ [0221]).
However, Mustafa does not explicitly facilitate performing, by the entity, a pre-processing workflow for the data stream to perform one or more data stream processing processes on the data stream, the pre-processing workflow comprising at least.
Malladi discloses, performing, by the entity, a pre-processing workflow for the data stream… the pre-processing workflow comprising at least (preprocessing continuous streams of raw data ¶ [0195], ML workflow executed on preprocessed data ¶ [0196]-[0198], processing ingested stream data ¶ [0018], data processing pipelines transformed stream data ¶ [0161]-[0164]), to perform one or more data stream processing processes on the data stream (stream processing ¶ [0014], analytics processing… processing the ingested stream data ¶ [0018], [0027], CEP engine and stream processing ¶ [0139], preprocessing, segmentation, enrichment, model execution ¶ [0195]-[0198]).
It would have been obvious to one ordinary skilled in the art at the time of the present invention to combine the teachings of the cited references because Malladi’s system would have allowed Moustafa to facilitate performing, by the entity, a pre-processing workflow for the data stream to perform one or more data stream processing processes on the data stream, the pre-processing workflow comprising at least. The motivation to combine is apparent in the Moustafa’s reference, because it improved computing systems, architectures, and techniques including improved edge analytics are needed to handle the large amounts of data generated by industrial machines.
However, neither Moustafa nor Malladi explicitly facilitates using a trained machine learning model … the analyzing comprising extracting hidden data from the signed filtered data stream using the trained machine learning model, the hidden data being derived from an arrangement of pixels making up the signed filtered data stream and is not explicitly marked in data making up the signed filtered data stream, and the hidden data.
Luo discloses, using a trained machine learning model (As discussed in detail below, the encoding, detection and decoding of the watermark can be performed by machine learning models that are trained to generate, detect and decode watermarks irrespective of any distortions at which the image is captured. To do this, the machine learning models are trained jointly so that the machine learning models are able to detect and decode watermarks generated by machine learning models involved during the training process ¶ [0029]. Also see ¶ [0053]-[0054], [0058]-[0061], [0070]-[0071], [0080], [0082]-[0083]) … the analyzing comprising extracting hidden data from the signed filtered data stream using the trained machine learning model (To decode the watermark detected in the possibly encoded image 302, the image analysis and decoder apparatus includes a watermark decoder 134. In some implementations, the watermark decoder 134 can implement a decoder machine learning model 134a that is configured to process the modified portion of the possibly encoded image 302, and generate, as output, a predicted first data item ¶ [0080]; a portion of the distorted image 450 is provided as input to the decoder machine learning model 134a … The decoder machine learning model 134a processes the identified portion of the distorted image 450 or the modified portion of the distorted image 450 to generate a predicted first data item 460 included in the image ¶ [0090]; The server system 102 decodes watermark to generate a predicted first data item (540). As described with reference to FIG. 4, to decode the watermark detected in the watermarked training image, the decoder machine learning model 134a processes the modified portion of the distorted training watermarked image to generate as output, a predicted first data item ¶ [0104]. Also see ¶ [0070]-[0071]) the hidden data being derived from an arrangement of pixels making up the signed filtered data stream (The server system 102 generates a first digital watermark (630). As described with reference to FIG. 1, the encoder machine learning model 112 implemented within the watermark generator 110 of the server system 102 is configured to receive as input, the first data item 122, to generate a first watermark 124 that encodes the first data item 122 into the first watermark 124. In some implementations, the first watermark 124 can be a matrix-type barcode that represents the first data item 122 as depicted in FIG. 2. The first watermark 124 can have a pre-defined size in terms of a number of rows and columns of pixels. Each pixel in the first watermark 124 can encode multiple bits of data, where the value of the multiple bits is represented by a different color. For example, a pixel that encodes the binary value ‘00’ may be black while a pixel that encodes the binary value ‘11’ may be white. Similarly, a pixel that encodes the binary value ‘01’ may be a lighter shade of black (for e.g., dark grey) while a pixel that encodes the binary value ‘10’ may be an even lighter shade of black (for e.g., light grey). In some implementations, the smallest encoding unit of the first watermark may actually be larger than a single pixel. But, for purposes of the examples described herein, the smallest encoding unit is assumed to be a single pixel. It should be appreciated, however, that the techniques described herein may be extended to implementations where the smallest encoding unit is a set of multiple pixels, e.g., a 2×2 or 3×3 set of pixels ¶ [0122]. the encoded source image 130 is an image that results from the client device 104 rendering the second watermark 126 over the source image 128a. Even though the second watermark 126 is separate from the source image 128a, the encoded source image 130 processed by the image analysis and decoder apparatus 118 may be a merged image showing the second watermark 126 blended over the source image 128a ¶ [0050]; Also see ¶ [0040], [0086], [0112]) and is not explicitly marked in data making up the signed filtered data stream, and the hidden (This specification describes systems, methods, devices and techniques for detecting and decoding visually discernible watermarks in captured reproductions of content (e.g., digital photos of content presented at a client device). While the description that follows describes watermark detection with respect to visually discernible watermarks, but the techniques can also be applied to visually perceptible watermarks. The visually discernible watermarks, referred to as simply “watermarks” for brevity, are semi-transparent, and visually discernible to a human user under normal viewing conditions, such that the watermarks can be embedded in content without degrading the visual quality of the content ¶ [0028]; the pixels classified as the first class (i.e., encoded using the second watermark) even though visually indiscernible to a human eye, is distinguishable to the watermark detector machine learning model 132a ¶ [0059], [0125]).
It would have been obvious to one ordinary skilled in the art at the time of the present invention to combine the teachings of the cited references because Luo’s system would have allowed Moustafa and Malladi to facilitate using a trained machine learning model … the analyzing comprising extracting hidden data from the signed filtered data stream using the trained machine learning model, the hidden data being derived from an arrangement of pixels making up the signed filtered data stream and is not explicitly marked in data making up the signed filtered data stream, and the hidden data. The motivation to combine is apparent in the Moustafa and Malladi’s reference, because there is a need to improve embedding watermarks in digital content as well as recovering watermarks embedded in digital content.
Regarding claims 2, 12 and 17, the combination of Moustafa, Malladi, and Luo discloses, wherein the data stream comprises a media file (Moustafa: a content type of the data stream may include classifying the content within the data stream as image data, audio data, textual data, telemetry data (e.g., sensor data, etc.), or any other type of data, or a combination of two or more types of such data ¶ [0048]. Also see ¶ [0119], [0215]).
Regarding claims 5, and 15, the combination of Moustafa, Malladi, and Luo discloses, wherein analyzing the signed filtered data stream comprises: obtaining, for a frame from the signed filtered data stream, at least one selected from a group consisting of: a caption, a keyword, a location, and a timestamp (Moustafa: example activities, operations, tasks, etc., can include tagging data within the datastream with one or more metadata tags, such as injecting a hash that represents time, date, location, identification of data stream ownership, etc. into the data stream, blockchaining data within the data stream, modifying the data stream to provide only a portion of the data, obscuring data within the data stream (e.g., a sensitive portion of image data in the data stream may be blurred out), or prohibiting the consumption of the data stream by one or more AI application nodes, among other operations ¶ [0169]. The patterns (e.g., solid, dotted, striped, hashed, etc.) of the various major nodes and adjacent nodes illustrated in FIG. 12 depict the various descriptors (e.g., keywords) of the metadata associated with raw data, ingested data, stored data, etc. stored in the distributed datastore 644 of FIG. 6, the datastore 1060 of FIG. 10, etc. ¶ [0182]. For example, the interface circuitry 1302 can access a data stream and hash metadata tags (e.g., time, date, location, identification of a security stakeholder of the data in the data stream, etc.) into the data stream for tracking or other identification purposes ¶ [0197]. In some examples, the metadata manager circuitry 1308 can filter the data using one or more filter parameters (e.g., a type of data, a type of device that produced the data, a timestamp or range of timestamps associated with the data, metadata associated with the data, etc.) to identify a subset of the data ¶ [0337]).
Regarding claim 6, the combination of Moustafa, Malladi, and Luo discloses, wherein analyzing the signed filtered data stream comprises: obtaining, for a video segment from the signed filtered data stream, at least one selected from a group consisting of: a title, a description of content of the video segment; a keyword, and a timestamp of an occurrence in the video segment (Moustafa: The second graph model 1204 includes a second example major node 1212, a third example adjacent node 1214, a fourth example adjacent node 1216, and an example adjacent node grouping 1218. The patterns (e.g., solid, dotted, striped, hashed, etc.) of the various major nodes and adjacent nodes illustrated in FIG. 12 depict the various descriptors (e.g., keywords) of the metadata associated with raw data, ingested data, stored data, etc. stored in the distributed datastore 644 of FIG. 6, the datastore 1060 of FIG. 10, etc. The example adjacent nodes illustrated in FIG. 12 represent the metadata (e.g., the metadata 1324 of FIG. 13) associated with the ingested data, stored data, raw data etc. stored in the distributed datastore 644 of FIG. 6, the datastore 1320 of FIG. 13, etc. Additionally and/or alternatively, the graph models 1202, 1204 of metadata descriptors illustrated in FIG. 12 can represent data points, raw data, ingested data, stored data, etc., stored in other memory or other storage devices in a cloud (e.g., the edge cloud 110, the cloud data center 130, etc.). For example, the nodes in the graph models 1202 and 1204 may represent data points from ingested data (e.g., data within a target data stream) ¶ [0182]-[0183] and [0337]).
Regarding claim 7, the combination of Moustafa, Malladi, and Luo discloses, wherein analyzing the signed filtered data stream comprises: obtaining, for a video segment from the signed filtered data stream, at least one selected from a group consisting of: a title, a description of content of the video segment; a keyword, and a timestamp of an occurrence in the video segment (Moustafa: The data ingestion manager 606 may pre-process the data by metadata tagging with data management settings (e.g., a locality or location of the data, expiration date, a source of the data, a type of the data, etc.) ¶ [0166]. Example activities, operations, tasks, etc., can include tagging data within the datastream with one or more metadata tags, such as injecting a hash that represents time, date, location, identification of data stream ownership, etc. into the data stream, blockchaining data within the data stream, modifying the data stream to provide only a portion of the data, obscuring data within the data stream (e.g., a sensitive portion of image data in the data stream may be blurred out), or prohibiting the consumption of the data stream by one or more AI application nodes, among other operations ¶ [0169]. Also see ¶ [0197], [0226], [0337])
Regarding claim 10, the combination of Moustafa, Malladi, and Luo discloses, wherein packaging the signed filtered data stream with the metadata to obtain the enhanced stream comprises: adding associations between portions of the metadata with portions of the signed filtered data stream (Moustafa: In some examples, the ADM console 602 may implement metadata tagging (e.g., add, remove, and/or modify metadata) ¶ [0107]. For example, the AMR 634 can provide metadata associated with ingested data to the resource manager/orchestration agent 642. In some examples, the resource manager/orchestration agent 642 can identify an AI/ML model that corresponds to the metadata and provide the AI/ML model to the AMR 634 for execution and/or instantiation at the node to execute a workload associated with the ingested data or data to be subsequently ingested ¶ [0166]. he patterns (e.g., solid, dotted, striped, hashed, etc.) of the various major nodes and adjacent nodes illustrated in FIG. 12 depict the various descriptors (e.g., keywords) of the metadata associated with raw data, ingested data, stored data, etc. stored in the distributed datastore 644 of FIG. 6, the datastore 1060 of FIG. 10, etc. The example adjacent nodes illustrated in FIG. 12 represent the metadata (e.g., the metadata 1324 of FIG. 13) associated with the ingested data, stored data, raw data etc. stored in the distributed datastore 644 of FIG. 6, the datastore 1320 of FIG. 13, etc. ¶ [0182], [0184], [0212]).
Regarding claim 21, (Cancelled).
Regarding claim 22, (Cancelled).
Regarding claim 23, the combination of Moustafa, Malladi, and Luo discloses, wherein causing the second data processing system to provision the computer implemented services using the enhanced stream comprises causing the second data processing system to disable or enable hardware components of the second data processing system, based on the enhanced stream (Moustafa: base station compute, acceleration and network resources can provide services in order to scale to workload demands on an as needed basis by activating dormant capacity (subscription, capacity on demand) in order to manage corner cases, emergencies or to provide longevity for deployed resources over a significantly longer implemented lifecycle ¶ [0071]; To achieve results with low latency, the services executed within the Edge cloud 110 balance varying requirements in terms of: (a) Priority (throughput or latency) and Quality of Service (QoS) (e.g., traffic for an autonomous car may have higher priority than a temperature sensor in terms of response time requirement; or, a performance sensitivity/bottleneck may exist at a compute/accelerator, memory, storage, or network resource, depending on the application) ¶ [0074]; in an edge environment that enables nodes to communicate with each other over one or more networks (e.g., one or more wired networks and/or one or more wireless networks, etc.), the communication may be in the form of network traffic ¶ [0046]; Edge computing nodes may partition resources (memory, central processing unit (CPU), graphics processing unit (GPU), interrupt controller, input/output (I/O) controller, memory controller, bus controller, etc.) where respective partitionings may contain a RoT capability and where fan-out and layering according to a DICE model may further be applied to Edge Nodes. Cloud computing nodes often use containers, FaaS engines, servlets, servers, or other computation abstraction that may be partitioned according to a DICE layering and fan-out structure to support a RoT context for each ¶ [0086], a pod controller oversees the partitioning and allocation of containers and resources ¶ [0090]-[0091], In some disclosed examples, the ADM system may assign metadata to the target data stream to cause orchestration of edge resources to monitor and potentially modify data usage/consumption of data within the target data stream by the target AI application node ¶ [0059], [0198]-[0201]).
Regarding claim 24, the combination of Moustafa, Malladi, and Luo discloses, wherein the enhanced stream is provided to the second data processing system to eliminate bottlenecks and resource constraints that the second data processing system would otherwise experience if the data stream is provided directly to the second data processing system without first being preprocessed into the enhanced stream by the entity (Malladi: he system provides a software apparatus in the form of a platform designed to perform machine-learning workflows across datacenters or “cloud” and compute resources available near sensor networks or “edge”, for example, from medium-sized servers (e.g., a dual-core processor and 4 gigabytes of memory) to miniaturized nodes (e.g., a single core processor core with less than 1 gigabyte of memory) ¶ [0194]-[0198]. Foghorn provides: Enriched IoT device and sensor data access for edge apps in both stream and batch modes. Highly efficient and expressive DSL for executing analytical functions. Powerful miniaturized analytics engine that can run on low footprint machines ¶ [0063]; FogHorn provides an efficient and highly scalable edge analytics platform that enables real-time, on-site stream processing of sensor data from industrial machines ¶ [0064]-[0068], [0122]).
Claim(s) 3, 8, 9, 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Moustafa in view of Malladi in view Luo in view of SHEN; Maying et al. (US 20250384660 A1) [Shen].
Regarding claims 3, 13, and 18, the combination of Moustafa, Malladi, and Luo clearly teaches all the limitations of claim 2.
However, neither one of Moustafa, Malladi, or Luo explicitly facilitate wherein filtering the data stream for the relevant content comprises: comparing frames of the media file to obtain similarity scores for each of the frames; adding a first portion of the frames having first similarity scores that meet criteria; and discarding a second portion of the frames having second similarity scores that do not meet the criteria, the first similarity scores and the second similarity scores being ones of the similarity scores for each of the frames.
Shen discloses, wherein filtering the data stream for the relevant content comprises: comparing frames of the media file to obtain similarity scores for each of the frames (the processing circuitry is to evaluate a performance of an objection detection model that is trained according to the dataset relative to being trained/updated according to the plurality of image frames. In some implementations, the processing circuitry is to remove the at least one image frame based at least on a similarity score between the visual embedding of the at least one image frame and the visual embedding of the at least one other image frame ¶ [0008], [0012], [0013], [0056], [0057], [0061]);
adding a first portion of the frames having first similarity scores that meet criteria (At 210, other samples, e.g., new image frames, can be selectively added to the dataset to enrich the dataset. For example, for a given (new) image frame, a semantic embedding can be determined in order to identify a (closest) candidate cluster to which the given image frame may potentially be assigned, and a visual embedding of the given image frame can be compared (e.g., using cosine similarity) to visual embeddings of image frames in the candidate cluster to determine whether the given image frame is sufficiently distinct to be selected for inclusion in the cluster ¶ [0057]); and
discarding a second portion of the frames having second similarity scores that do not meet the criteria, the first similarity scores and the second similarity scores being ones of the similarity scores for each of the frames (At 205, data samples, such as image frames, can be retrieved as clusters, where the image frames are clustered according to semantic features of the data samples. For example, each cluster can have a subset of the image frames that are semantically similar, such as to represent similar objects, scenes, and/or actions. From any given cluster, data samples can be removed that are visually similar to one or more other data samples of the given cluster. For example, a first image frame can be removed from a first cluster responsive to a cosine similarity of the first image frame and a second image frame of the first cluster being greater than a threshold similarity. The similarities of image frames can be iteratively evaluated to allow for removal of image frames from the clusters until a termination condition, such as a target number and/or size of remaining image frames and/or until no further image frames meet the conditions for removal from their respective thresholds ¶ [0056]-[0057], [0061]. Also see ¶ [0012]-[0013]).
It would have been obvious to one ordinary skilled in the art at the time of the present invention to combine the teachings of the cited references because Shen’s system would have allowed Moustafa, Malladi, and Luo to facilitate the data stream for the relevant content comprises: comparing frames of the media file to obtain similarity scores for each of the frames; adding a first portion of the frames having first similarity scores that meet criteria; and discarding a second portion of the frames having second similarity scores that do not meet the criteria, the first similarity scores and the second similarity scores being ones of the similarity scores for each of the frames. The motivation to combine is apparent in the Moustafa, Malladi, and Luo’s reference, because there need for improve efficient datasets to facilitate improvement of AI models trained using such datasets, such as to achieve or exceed target performance criteria with reduced sized datasets.
Regarding claim 8, the combination of Moustafa, Malladi, Luo, and Shen discloses, wherein analyzing the signed filtered data stream comprises: obtaining, for a video segment from the signed filtered data stream, a transcription (Shen: For example, the caption generator 112 can include at least one of a VLM or a MMLM to receive, as input, a data sample from data source 104 (e.g., an image frame) and generate, as output, a description of the data sample. The description can indicate features of any one or more objects represented in the data sample. The system 100 can provide to the caption generator 112 one or more prompts for requesting information to include in the description. The prompts can include, for example and without limitation, requests such as a general scene description, a general description of what is happening in the scene, important objects to consider while driving, or dynamic objects. For example, given an image frame representing a scene of a road and a crosswalk, the system 100 can prompt the caption generator 112 to generate a description of a location of the crosswalk (e.g., relative to a position from which the image frame is captured), any pedestrians in the crosswalk, and any vehicles on the road ¶ [0044]-[0045], [0047]. Such an association may include, for example, the generation of a caption or portion of text that is embedded (e.g., as metadata) with an input source text or image. In one or more implementations, an output of a language model may be used to determine the validity of an input source material for further processing, or inclusion in a dataset ¶ [0071]).
Regarding claim 9, the combination of Moustafa, Malladi, Luo, and Shen discloses, wherein analyzing the signed filtered data stream comprises: obtaining, for a video segment from the signed filtered data stream, a summarization of content depicted in the video segment (Shen: For example, the caption generator 112 can include at least one of a VLM or a MMLM to receive, as input, a data sample from data source 104 (e.g., an image frame) and generate, as output, a description of the data sample. The description can indicate features of any one or more objects represented in the data sample. The system 100 can provide to the caption generator 112 one or more prompts for requesting information to include in the description. The prompts can include, for example and without limitation, requests such as a general scene description, a general description of what is happening in the scene, important objects to consider while driving, or dynamic objects. For example, given an image frame representing a scene of a road and a crosswalk, the system 100 can prompt the caption generator 112 to generate a description of a location of the crosswalk (e.g., relative to a position from which the image frame is captured), any pedestrians in the crosswalk, and any vehicles on the road ¶ [0044]-[0045], [0047]. Such an association may include, for example, the generation of a caption or portion of text that is embedded (e.g., as metadata) with an input source text or image. In one or more implementations, an output of a language model may be used to determine the validity of an input source material for further processing, or inclusion in a dataset ¶ [0071]).
Claim(s) 4, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Moustafa in view of Malladi in view of Luo in view of SHEN in view of REN; Yuzhuo et al. (US 20260009639 A1) [Ren].
Regarding claims 4, and 14, the combination of Moustafa, Malladi, Luo, and Shen teach all the limitations of claim 3, and 13.
However, neither one of Moustafa, Malladi, Luo, or Shen explicitly facilitates wherein comparing the frames comprises: obtaining a first frame of the frames; obtaining a second frame of the frames that is temporally ordered immediately after the first frame of the frames; calculating a pixel-by-pixel difference between the first frame and the second frame to obtain a similarity score for the second frame; in a first instance of the calculating where the similarity score is below a threshold, marking the second frame for inclusion in the first portion; and in a second instance of the calculating where the similarity score is above the threshold, marking the second frame for inclusion in the second portion.
Ren discloses, wherein comparing the frames comprises: obtaining a first frame of the frames; obtaining a second frame of the frames that is temporally ordered immediately after the first frame of the frames; calculating a pixel-by-pixel difference between the first frame and the second frame to obtain a similarity score for the second frame; in a first instance of the calculating where the similarity score is below a threshold, marking the second frame for inclusion in the first portion; and in a second instance of the calculating where the similarity score is above the threshold, marking the second frame for inclusion in the second portion (Taking the test frame mask 230 as an example, the static region detector 210 may compare the test frame 120 (or data derived therefrom) to corresponding simulated data. For example, the target scene being observed may be simulated in a static state as a 3D model of the environment (e.g., an empty vehicle cabin). In some embodiments, a virtual camera may be positioned in the 3D environment model, a simulated frame (e.g., a 2D image) may be generated, and the simulated frame may be compared to the test frame 120 (e.g., converting to greyscale, applying thresholding, subtracting one image from the other to represent) to identify areas where pixel values differ more than a threshold amount (e.g., representing occlusions) and remove those regions from the test frame mask 230. Additionally or alternatively, the 3D environment model may be used to generate a simulated depth map, and the static region detector 210 may convert the test frame 120 into a depth map using any known technique (e.g., using a neural network), compute the difference between the depth values in corresponding pixels of the test depth map generated from the test frame 120 and the simulated depth map, use the resulting difference map to identify areas where the depth varies more than a threshold amount, and remove those regions from the test frame mask 230. In some embodiments, the static region detector 210 may segment the background (e.g., the static regions, such as the frame of the car, car seats, etc.) from the foreground (e.g., the dynamic regions, such as the occupants of the vehicle) in the test frame 120, and remove regions from the test frame mask 230 corresponding to the foreground. As such, the static region detector 210 may identify and remove regions from the test frame mask 230 that do not correspond to the expected static scene (e.g., detected occlusions) ¶ [0038] and [0043]).
It would have been obvious to one ordinary skilled in the art at the time of the present invention to combine the teachings of the cited references because Ren’s system would have allowed Moustafa, Malladi, Luo, and Shen to facilitate wherein comparing the frames comprises: obtaining a first frame of the frames; obtaining a second frame of the frames that is temporally ordered immediately after the first frame of the frames; calculating a pixel-by-pixel difference between the first frame and the second frame to obtain a similarity score for the second frame; in a first instance of the calculating where the similarity score is below a threshold, marking the second frame for inclusion in the first portion; and in a second instance of the calculating where the similarity score is above the threshold, marking the second frame for inclusion in the second portion. The motivation to combine is apparent in the Moustafa, Malladi, Luo, and Shen’s reference, because there need for improved calibration techniques to reduce time, effort, and computational demands.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMAD S ROSTAMI whose telephone number is (571)270-1980. The examiner can normally be reached Mon-Fri From 9 a.m. to 5 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at (571)270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
9/17/2026
/MOHAMMAD S ROSTAMI/Primary Examiner, Art Unit 2154
Prosecution Insights