DETAILED ACTION
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 8, 12, 13, 18-20 are rejected under 35 U.S.C. 102 (a)(2) as being anticipated by Othman et al. (Pub. No. US 2021/0304421).
Regarding claim 1, Othman teaches One or more processors comprising processing circuitry to: compute a plurality of representations (person feature vector) corresponding to a behavior of one or more subjects within an environment based on a first time frame of streaming sensor data (video frame), wherein the streaming sensor data includes behavior data of one or more subjects (persons) within an environment (monitored location) captured from a plurality of optical sensors [Para. 31; Para. 30 “the cameras 12 are positioned within a monitored location 34 which can be a space such as an interior or exterior of a structure, a segment of land, or a combination thereof”; Para. 9 “The cameras each produce a sequence of video frames which include a prior video frame and a current video frame, with each video frame containing detections which depict one or more of the persons”; and Para. 43 “at step 606, the person featurizer 57 analyzes the visual features of the input image 54 associated with each detection 58 and generates the corresponding person feature vector 55”. Since the claim is silent as to what the behavior is/means, any activity/action/feature/motion taught by the prior art could be interpreted to mean behavior. The claim also unclear as to what the term “corresponding” is mean/applied; Therefore, it is very broad]; associate, based at least on trajectory tracking data (motion data) for the one or more subjects, one or more prior behavior states (incumbent track) of a first plurality of prior behavior states with the plurality of representations (video frame) [Para. 9 “The visual processing unit establishes a track identity for each person appearing in the previous video frame by detecting visual features and motion data for the person, and associating the visual features and motion data with an incumbent track”; Para. 38 “Each incumbent track 59 corresponds to a specific detection 58 which has been identified by the visual processing unit 14 in at least one video frame 50 prior to the current video frame 50”; Para. 37 and 43, shows that visual features are part of person feature vector]; assign a second plurality of prior behavior states (new track) to the plurality of representations (detections) based on at least one of the plurality of representations not having an associated prior behavior state (no corresponding incumbent track) from the first plurality of prior behavior states [para. 38 “The tracking module 64 is adapted to either match each detection 58 to an incumbent track 59, or assign the detection 58 to a new track if no corresponding incumbent track 59 is present” and Para. 42 “In a preferred embodiment, the Hungarian algorithm, or Kuhn-Munkres algorithm, is used to determine a maximum sum assignment to create matchings between the detections 58 and tracks 59 that results in a maximization of overall likelihood 44 for the entire matrix”]; and update at least one of the first plurality of prior behavior states (incumbent track) or the second plurality of prior behavior states (new track) based on the plurality of representations to generate updated behavior states (incumbent track) [Para. 46 “Any incumbent track 59 or new track 59N which is matched to one of the detections 58 will be maintained as an incumbent track 59 when the next video frame is processed by the tracking module 64, and the motion data 59P for each incumbent track 59 is updated accordingly”].
Regarding claims 8 and 18, Othman teaches generate the plurality of representations using a machine learning model trained to detect characteristics representing the one or more subjects based at least on the streaming sensor data [para. 5, 11, 36, and 37].
Regarding claims 12 and 19, Othman teaches wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine [Para. 3, 4, and 50].
Regarding claim 13, Othman teaches a system comprising one or more processors (processor 15a) to: generate a plurality of representations (person feature vector) computed based on optical image streaming data (video frame) representing one or more tracked subjects (persons) within a monitored area (monitored location) [Para. 31; Para. 30 “the cameras 12 are positioned within a monitored location 34 which can be a space such as an interior or exterior of a structure, a segment of land, or a combination thereof”; Para. 9 “The cameras each produce a sequence of video frames which include a prior video frame and a current video frame, with each video frame containing detections which depict one or more of the persons”; and Para. 43 “at step 606, the person featurizer 57 analyzes the visual features of the input image 54 associated with each detection 58 and generates the corresponding person feature vector 55”];
associate, based at least on trajectory tracking data (motion data), a first set of individual representations of the plurality of representations with one or more prior behavior states (incumbent track) of a first plurality of prior behavior states computed from the optical image streaming data (video frame) [Para. 9 “The visual processing unit establishes a track identity for each person appearing in the previous video frame by detecting visual features and motion data for the person, and associating the visual features and motion data with an incumbent track”; Para. 38 “Each incumbent track 59 corresponds to a specific detection 58 which has been identified by the visual processing unit 14 in at least one video frame 50 prior to the current video frame 50”; Para. 37 and 43, shows that visual features are part of person feature vector]; assign, using an assignment process, a second plurality of prior behavior states (new track) to a second set of individual representations (detections) of the plurality of representations, based on at least one of the plurality of representations not having an associated prior behavior state (No corresponding incumbent track) from the first plurality of prior behavior states [para. 38 “The tracking module 64 is adapted to either match each detection 58 to an incumbent track 59, or assign the detection 58 to a new track if no corresponding incumbent track 59 is present” and Para. 42 “In a preferred embodiment, the Hungarian algorithm, or Kuhn-Munkres algorithm, is used to determine a maximum sum assignment to create matchings between the detections 58 and tracks 59 that results in a maximization of overall likelihood 44 for the entire matrix”]; and update at least one of the first plurality of prior behavior states (incumbent track) and the second plurality of prior behavior states (new track) based on the plurality of representations to generate updated behavior states (incumbent track) [Para. 46 “Any incumbent track 59 or new track 59N which is matched to one of the detections 58 will be maintained as an incumbent track 59 when the next video frame is processed by the tracking module 64, and the motion data 59P for each incumbent track 59 is updated accordingly”].
Claim 20 is rejected for the same reasons as claim 1. Furthermore, Othman teaches a method to perform the claim limitations [see abstract and brined summary].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Othman et al. (Pub. No. US 2021/0304421) in view of Taylor (Pub. No. US 2012/0249802).
Regarding claim 2, Othman doesn’t explicitly teach the claim limitation.
However, Taylor teaches wherein the processing circuitry is further to: define one or more anchors (track) to associate individual prior behavior (state vector) states from at least one of the first plurality of prior behavior states and the second plurality of prior behavior states with a global [unique] identifier (ID) [52].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman to teach the claim limitations above, feature as taught by Taylor; because the modification enables the system to improves reliable, accurate multi-camera target tracking over a wide area by using self-localizing distributed smart cameras that share compact sighting/track data and maintain persistent globally unique track identifiers as objects move through the network.
Claims 3, 4, 6, 14, 15, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Othman et al. (Pub. No. US 2021/0304421) in view of Zhu et al. (Pub. No. US 2018/0204093).
Regarding claims 3 and 14, Othman doesn’t explicitly teach the claim limitation.
However, Zhu teaches cluster the at least one of the plurality of representations (bounding boxes) not having the associated prior behavior state (existing identity) from the first plurality of prior behavior states to generate one or more clusters based at least on similarity (pairwise similarity metrics) [Para. 31, claim 3 “image has not yet been associated with an existing identity of the entity database”]; and assign, using a matching algorithm (assignment matrix), the second plurality of prior behavior states/identities to the one or more clusters [Para. 75].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman to teach the claim limitations above, feature as taught by Zhu; because the modification enables the system to improve a person reidentification accuracy by clustering similar detections before matching to an identity database and creating a new identity when no existing identity matches, reducing missed/incorrect identity associations.
Regarding claims 4 and 15, Othman doesn’t explicitly teach the claim limitation.
However, Zhu teaches initialize (create) a new anchor (addition identity) and corresponding behavior state (identity database) for at least one cluster (image clusters) of the one or more clusters (image clusters) based on the at least one cluster not being assigned (has not yet been associated) at least one of the second plurality of prior behavior states by the matching algorithm (assignment Matrix) [Para. 70, claim 3 “determining that at least one of the one or more image clusters represents an image of a person whose image has not yet been associated with an existing identity of the entity database”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman to teach the claim limitations above, feature as taught by Zhu; because the modification enables the system to improve a person reidentification accuracy by clustering similar detections before matching to an identity database and creating a new identity when no existing identity matches, reducing missed/incorrect identity associations.
Regarding claims 6 and 17, Othman teaches a Hungarian matching algorithm [Para. 42].
Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Othman et al. (Pub. No. US 2021/0304421) in view of in view of Zhu et al. (Pub. No. US 2018/0204093) and further in view of Campos et al. (Pub. No. US 2003/0212702).
Regarding claims 5 and 16, Othman in view of Zhu doesn’t explicitly teach the claim limitation.
However, Campos teaches wherein the processing circuitry is further to cluster the at least one of the plurality of representations (training data) not having the associated prior behavior state, based on a hierarchical clustering process that performs operations to: based at least on a similarity (Euclidean distance) of representations in the plurality of representations, determine a subject prediction number (maximum number of clusters) based on a first clustering process that clusters (core k-means process) at least one of the plurality of representations not having the associated prior behavior state [Para. 122, 125 and 127]; and based at least on the prediction number (maximum number of clusters), apply to the at least one of the plurality of representations (data point) not having the associated prior behavior state, a second clustering process (k-means process) to cluster the at least one of the plurality of representations not having the associated prior behavior state [Para. 122 and 127].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman in view of Zhu to teach the claim limitations above, feature as taught by Campos; because the modification enables the system to improve the efficiency of clustering large datasets by using a hierarchical k-means partitioning approach that splits data in stages while controlling the maximum number of clusters.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Othman et al. (Pub. No. US 2021/0304421) in view of in view of Yu et al. (Pub. No. US 2022/0245835).
Regarding claim 7, Othman doesn’t explicitly teach the claim limitation.
Yu teaches wherein the behavior data (update tracking data) comprises at least one of appearance data (appearance) or spatiotemporal data (geometry and motion) represented by the plurality of behavior embeddings (geo-motion embedding, appearance embedding) [Para. 36, 37, and 41].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman to teach the claim limitations above, feature as taught by Yu; because the modification enables the system to improve multi-object tracking data association by using learned geo-motion and appearance embeddings that encode an object’s motion/geometry and visual appearance over time to more accurately match new detections to existing tracks.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Othman et al. (Pub. No. US 2021/0304421) in view of in view of Narayanswamy et al. (Pub. No. US 2017/0134619).
Regarding claim 9, Othman doesn’t explicitly teach the claim limitation.
However, Narayanswamy teaches wherein the streaming sensor data comprises synchronized optical image streaming data (synchronized image data) that includes individual image feeds (streaming video streams) from the plurality of optical sensors, wherein the plurality of optical sensors is synchronized to capture the individual image feeds at the same time [Para. 22, 44, 43].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman in view of Zhu to teach the claim limitations above, feature as taught by Narayanswamy; because the modification enables the system to improve multi-sensor camera capture by providing hardware-trigged synchronization and timestamped aggregation so corresponding frames from multiple cameras can be reliably aligned in time for downstream processing.
.
Regarding claim 10, Othman doesn’t explicitly teach the claim limitation.
However, Narayanswamy teaches wherein the processing circuitry is further to: map the plurality of representations to a global image coordinate system based at least on camera calibration parameters (rotation matrix)/(translation vector) associated with the plurality of optical sensors [Para. 11, 24, and 25].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman to teach the claim limitations above, feature as taught by Narayanswamy; because the modification enables the system to improve multi-sensor camera capture by providing hardware-trigged synchronization and timestamped aggregation so corresponding frames from multiple cameras can be reliably aligned in time for downstream processing.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Othman et al. (Pub. No. US 2021/0304421) in view of in view of Shen et al. (Pub. No. US 2022/0358314).
Regarding claim 11, Othman doesn’t explicitly teach the claim limitation.
Shen teaches wherein the processing circuitry is further to cause a display of a computer vision-based view of the one or more subjects for at least part of the environment based at least on the updated behavior states [Para. 37, 38, 49, and 42].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Othman to teach the claim limitations above, feature as taught by Shen; because the modification enables the system to improve multi-sensor camera capture by providing hardware-trigged synchronization and timestamped aggregation so corresponding frames from multiple cameras can be reliably aligned in time for downstream processing.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-7 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-0101 (IN USA OR CANADA) or 571-272-1000.
/SOLOMON G BEZUAYEHU/ Primary Examiner, Art Unit 2666
Prosecution Insights