DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/9/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot in view of the new grounds of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 5, 7-10, 13, 15-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Naphade et al (US20190294869) in view of Elluswamy et al (US20200249685).
Regarding claim 1, Naphade teaches a method, comprising:
obtaining, by one or more processors coupled with non-transitory memory (704 and 706 in fig. 7), a first plurality of images each including corresponding first time stamps, the first plurality of images corresponding to video data of a physical environment (para. [0019], In operation, upon capturing a video, motion of an object in the video may be tracked to generate an object trajectory that includes a series of positions of the object with corresponding time stamps; para. [0020], the motion of an object captured in a video may similarly be tracked to generate an observed object trajectory that includes a series of positions of the current object and corresponding time stamps; para. [0026], One way in which an object trajectory may be represented includes a set or series of positions or coordinates of an object and corresponding times (e.g., timestamps) for the positions);
extracting, by the one or more processors and from among the first plurality of images, a second plurality of images (In fig. 1, a first plurality of images may be input to 100 and a second plurality of images may be input to 116) each having corresponding second time stamps (para. [0020], the motion of an object captured in a video may similarly be tracked to generate an observed object trajectory that includes a series of positions of the current object and corresponding time stamps; para. [0026], One way in which an object trajectory may be represented includes a set or series of positions or coordinates of an object and corresponding times (e.g., timestamps) for the positions) and each having corresponding features indicating an object in the physical environment (para. [0026], Detected objects may be indicated, for example, using a bounding box; para. [0085], As illustrated, various detected objects are visually emphasized, in this case with bounding boxes around the objects); and
training, by the one or more processors and with input including features and the second time stamps corresponding to features (104 and 106 in fig. 1), a machine learning model (108 in fig. 1) to generate an output indicating a pattern of movement of one or more objects corresponding to the features and the second time stamps corresponding to the features (para. [0019], The object trajectory, along with numerous other object trajectories observed over time, may be used as input into a machine learning model (e.g., a long short-term memory (LSTM) network) of a model training system; para. [0020], The observed object trajectory may be input into an inference component that also uses the trained (or deployed) machine learning model and/or normal object behavior templates (e.g., as computed by the machine learning model) as an input to identify anomalies with respect to the current observed object trajectory. More specifically, the inference component may use the LSTM network and the current object trajectory to compute a deviation from normal object behavior; para. [0029]; para. [0034]).
Naphade fails to teach detecting a feature in one or more images of the first plurality of images;
wherein the second plurality of images, each having the detected feature, is a subset of the first subset of images and a number of the second plurality of images is less than a number of the first plurality of images; and
training with inputs including the detected feature and time stamps corresponding to the detected feature.
However Elluswamy teaches a method of detecting a feature in one or more images of the first plurality of images (para. [0015], The sensor data may include an image; para. [0027], to identify the location of relevant features in the sensor data such as lane lines, traffic control signals, objects, etc);
wherein a number of the second plurality of images is less than a number of the first plurality of images (para. [0035], discarding unnecessary frames. It would be obvious for each image to have the detected feature); and
training with inputs including the detected feature and time stamps corresponding to the detected feature (para. [0011], Using data captured by sensors on a vehicle to capture the environment of the vehicle and vehicle operating parameters, a training data set is created. In some embodiments, a time series of captured data is used to generate the training data; para. [0027], [0034]).
Therefore taking the combined teachings of Naphade and Elluswamy as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Elluswamy into the method of Naphade. The motivation to combine Elluswamy and Naphade would be to significantly improve the safety of the autonomous vehicle (para. [0013] of Elluswamy).
Regarding claim 2, the modified invention of Naphade teaches a method further comprising:
extracting, by the one or more processors and via a second machine learning model trained with input including a third plurality of images having one or more second features indicating objects in the physical environment, the second plurality of images (120 in fig. 1 of Naphade; para. [0026] of Naphade, Object detection may be performed in any number of ways, such as, for example, via machine learning (e.g. convolutional neural networks) or other computer vision algorithms). The claim does not explicitly recite that the number of third images is different than the first number of images. Therefore both the first plurality of images and the second plurality of images may be the same.
Regarding claim 5, the modified invention of Naphade teaches a method further comprising:
generating, by the one or more processors via the trained machine learning model, the output indicating the pattern of movement of a second object (para. [0018] of Naphade, Accordingly, embodiments described herein are directed to tracking objects, such as vehicles or pedestrians, and using the object trajectories to learn real-time object movement insights, such as anomalies and various trajectory features; para. [0026] of Naphade, In this regard, object detection may be performed to detect one or more objects within the source data (e.g., within a video represented by image data captured by a camera(s)))); and
linking, by the one or more processors and based on a feature of a predetermined pattern of movement, the pattern of movement with the predetermined type of movement (abstract of Naphade, By comparing the observed object trajectory to the expected object trajectory, a determination may be made that the observed object trajectory is indicative of an anomaly; para. [0047] of Naphade, To do so, the inference component 126 may compare an observed or actual object trajectory with an expected object trajectory (e.g., predicted object trajectory and/or trajectory template(s) as generated by the deployed model 124) to identify whether a deviation exists and/or an extent of a deviation; para. [0048] of Naphade).
Regarding claim 7, the modified invention of Naphade teaches a method further comprising:
generating, by the one or more processors via the trained machine learning model, the output indicating the pattern of movement of a second object and indicating a second pattern of movement of a third object (para. [0018] of Naphade, Accordingly, embodiments described herein are directed to tracking objects, such as vehicles or pedestrians, and using the object trajectories to learn real-time object movement insights, such as anomalies and various trajectory features; para. [0026] of Naphade, In this regard, object detection may be performed to detect one or more objects within the source data (e.g., within a video represented by image data captured by a camera(s)))); and
linking, by the one or more processors and based on a feature of a predetermined pattern of movement, the pattern of movement and the second pattern of movement with the predetermined type of movement (abstract of Naphade, By comparing the observed object trajectory to the expected object trajectory, a determination may be made that the observed object trajectory is indicative of an anomaly; para. [0047] of Naphade, To do so, the inference component 126 may compare an observed or actual object trajectory with an expected object trajectory (e.g., predicted object trajectory and/or trajectory template(s) as generated by the deployed model 124) to identify whether a deviation exists and/or an extent of a deviation; para. [0048] of Naphade).
Regarding claim 8, the modified invention of Naphade teaches a method wherein the physical environment corresponds to a roadway (figs. 3A and 3B of Naphade), and the one or more objects corresponding to one or more of a vehicle, a person, an item of debris, or any combination thereof located at least partially in the roadway (figs. 3A-3B of Naphade; para. [0084] of Naphade).
Regarding claim 9, the claim recites similar subject matter as claim 1 and is rejected for the same reasons as stated above.
Regarding claim 10, the claim recites similar subject matter as claim 2 and is rejected for the same reasons as stated above.
Regarding claim 13, the claim recites similar subject matter as claim 5 and is rejected for the same reasons as stated above.
Regarding claim 15, the claim recites similar subject matter as claim 7 and is rejected for the same reasons as stated above.
Regarding claim 16, the claim recites similar subject matter as claim 8 and is rejected for the same reasons as stated above.
Regarding claim 17, the claim recites similar subject matter as claim 1 and is rejected for the same reasons as stated above.
Regarding claim 18, the claim recites similar subject matter as claim 5 and is rejected for the same reasons as stated above.
Regarding claim 20, the claim recites similar subject matter as claim 7 and is rejected for the same reasons as stated above.
Claim(s) 3, 4, 11, and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Naphade et al (US20190294869) and Elluswamy et al (US20200249685) in view of Kulandai Samy et al (US20250021624).
Regarding claim 3, the modified invention of Naphade fails to teach a method further comprising:
selecting, by the one or more processors and based on a frame rate of the video data, the second plurality of images, the second time stamps separated by a time interval corresponding to the frame rate.
However Kulandai Samy teaches selecting a plurality of images based on a frame rate of the video data, the second time stamps separated by a time interval corresponding to the frame rate (para. [0033], [0047]).
Therefore taking the combined teachings of Naphade and Elluswamy with Kulandai Samy as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Kulandai Samy into the method of Naphade and Elluswamy. The motivation to combine Kulandai Samy, Elluswamy and Naphade would be to prevent an increase in bias of the detection model (para. [0033] of Kulandai Samy).
Regarding claim 4, the modified invention of Naphade fails to teach a method further comprising:
selecting, by the one or more processors and based on a predetermined time period, the second plurality of images, a difference between an earliest time stamp among the second time stamps and a latest time stamp among the second time stamps less than or equal to the predetermined time period.
However Kulandai Samy teaches selecting a second plurality of images based on a predetermined time period, a difference between an earliest time stamp among the second time stamps and a latest time stamp among the second time stamps less than or equal to the predetermined time period (para. [0033], [0047]).
Therefore taking the combined teachings of Naphade and Elluswamy with Kulandai Samy as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Kulandai Samy into the method of Naphade and Elluswamy. The motivation to combine Kulandai Samy, Elluswamy and Naphade would be to prevent an increase in bias of the detection model (para. [0033] of Kulandai Samy).
Regarding claim 11, the claim recites similar subject matter as claim 3 and is rejected for the same reasons as stated above.
Regarding claim 12, the claim recites similar subject matter as claim 4 and is rejected for the same reasons as stated above.
Claim(s) 6, 14 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Naphade et al (US20190294869) and Elluswamy et al (US20200249685) in view of Sharma Banjade et al (US20240214786).
Regarding claim 6, the modified invention of Naphade fails to teach a method further comprising:
generating, by the one or more processors via the trained machine learning model, the output indicating a second pattern of movement of a third object, the second pattern of movement intersecting with the pattern of movement in one or more of the second plurality of images.
However Sharma Banjade teaches generating outputs, using a trained machine learning model with image inputs (para. [0393]), indicating patterns of movement of a plurality of objects, the patterns of movement of the plurality of objects intersecting one another (para. [0290]).
Therefore taking the combined teachings of Naphade and Elluswamy with Sharma Banjade as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Sharma Banjade into the method of Naphade and Elluswamy. The motivation to combine Sharma Banjade, Elluswamy and Naphade would be to trigger an appropriate collision avoidance action (para. [0289] of Sharma Banjade).
Regarding claim 14, the claim recites similar subject matter as claim 6 and is rejected for the same reasons as stated above.
Regarding claim 19, the claim recites similar subject matter as claim 6 and is rejected for the same reasons as stated above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEON VIET Q NGUYEN whose telephone number is (571)270-1185. The examiner can normally be reached Mon-Fri 11AM-7PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEON VIET Q NGUYEN/Primary Examiner, Art Unit 2663
Prosecution Insights