DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Initially, it is noted that the Reply filed 20 February 2026 included certain defects that should be addressed on future filings. These defects are in the claim amendments section which included spurious text “Error! Reference source not found.”. This spurious text is being ignored as an inadvertent error but more careful attention should be paid in the future to avoid a Notice of Non-Compliant Amendment and the resulting prosecution delay.
The 20 February 2026 amendment overcomes the 35 USC 101 rejection by adding the word “non-transitory” to claim 22.
Response to Arguments
Applicant's arguments filed 20 February 2026 have been fully considered but they are not persuasive.
The 112(b) rejection essentially states that image data in and of itself is merely an array of numbers representing, e.g., color values, and does not automatically identify bounding boxes or common objects. More formally, “image data” is typically defined as follows:
Data produced by scanning a surface with an optical or electronic device. Common examples include scanned documents, remotely sensed data (for example, satellite images), and aerial photographs. An image is stored as a raster dataset of binary or integer values that represent the intensity of reflected light, heat, or other range of values on the electromagnetic spectrum. Image Data Definition | GIS Dictionary downloaded 10 April 2026
Applicant argues that “image data” includes sequential images that are received and those images include bounding boxes identifying objects such that the metes and bounds are clear.
In response, it is clear that Applicant is acting as their own lexicographer by explicitly redefining “image data” to “comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images”.
In further response, because the claimed invention passively receives this “image data” then Applicant tacitly admits that detecting objects, recognizing objects such that their identity is known, and conducting object re-identification (object identified in two or more sequential images) are individually and collectively prior art with respect to the instant invention. In other words, because the invention plays and claims no active role in creating the “image data” and the underlying detecting, identifying, and re-identifying of the object in order to form the claimed received bounding boxes then all of this “image data” must be pre-existing and conventional. Examination will proceed given this lexicographal definition and admission of prior art regarding the entirely of the receiving step.
In response to Applicant’ arguments regarding the Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 3, 17, and 20-22 are rejected under 35 U.S.C. 103 as being unpatentable over Jain {Dutt Jain, Suyog, Bo Xiong, and Kristen Grauman. "Fusionseg: Learning to combine motion and appearance for fully automatic segmentation of generic objects in videos." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017} and Mueller {Muller, Matthias, et al. "Trackingnet: A large-scale dataset and benchmark for object tracking in the wild." Proceedings of the European conference on computer vision (ECCV), 2018}.
Claim 1
In regards to claim 1, Jain discloses a computer-implemented method of generating training data for one or more computer models
{See abstract and section 3.30. As to computer-implemented, see sections 3.4 including the implementation details and section 4 Results in which the disclosed models are implemented on a computer having a processor and memory to produce the disclosed test results. See also the citations and explanations below}, the method comprising:
receiving a plurality of image data, wherein the image data comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images
{section 3 including input sequence of video frames [I1, I2, … IN] wherein the sequential images of the video frames comprise objects. Section 3.2, Fig. 3, input video dataset includes bounding boxes labeled for each object. As to “common objects identified in two or more sequential images” note that the optical flow algorithm used by Jain computes motion/flow of a common/same object identified in sequential image frames};
generating a score indicating movement of the identified one or more common objects detected in two or more sequential images
{Section 3.2 Motion Stream including computing a score indicating movement (optical flow) of the object in sequential images which is within the BRI (broadest reasonable interpretation) consistent with the instant invention’s Fig. 1, movement score S > threshold triggers creating of positive training data, [0051]-[0052] of published instant specification.}; and
outputting the two or more sequential images as positive training data when the score exceeds a threshold
{Section 3.2 which eliminates bounding boxes (segmentations) wherein the object and background lack distinct optical flow such that the motion model can learn from desired cues (bounding boxes with motion score/flow exceeding a threshold). This “optical flow test” employs a 2-norm between the average (score) value within the bounding box and b) the average value in a surrounding area exceeds a threshold (30), and, if this threshold is exceeded indicating significant motion of the bounding box segments then they are added to (positive) training set. See also Fig. 3 illustrating this process. Even if the pair of bounding boxes from sequential images that are used to determine the score (flow value exceeding a threshold) are not viewed as being output and used as training data, the sequential and iterative nature of Jain’s process would, at the next iteration, output another of the sequential images and bounding box of the common/persistent object for training provided that significant object motion (above the threshold) continues}.
Mueller is an analogous reference from the same field of generating training data or otherwise reasonably pertinent to the problem faced by the inventor. See abstract and cites below.
Mueller also teaches
receiving a plurality of image data, wherein the image data comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images
{See Abstract, Sections 1, 3.2, 3.3, and 4 discusses processing received videos (series of sequential images) captured by a video camera and including common objects that are annotated and contained within bounding boxes using object tracking};
determining a correlation between the identified one or more common objects detected in two or more sequential images using the bounding boxes for each of the one or more common objects
{Sections 3.2 TrackingNet in which Turkers are used to track/correlate an object and its bounding box to annotate the successive frames with the correlation/tracking. Moreover, an optical flow algorithm is used to guess/predict the next bounding box in successive frames. See also Section 3.3 that analyzes the correlation/variation between bounding boxes of common objects in successive video frames}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Jain’s method of generating training data for one or more computer models that already receives a plurality of image data, wherein the image data comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images and generates a score indicating movement of the identified one or more common objects detected in two or more sequential images such that the method also determines a correlation between the identified one or more common objects detected in two or more sequential images using the bounding boxes for each of the one or more common objects as taught by Mueller with the result being that Jain generates a score indicating movement of the identified one or more common objects detected in two or more sequential images based on the determined correlation because such correlation adds a useful object tracking features to the training data set generation by correlating/tracking common objects in sequential images of the video; because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results particularly in view of Mueller’s use of optical flow for object tracking and in which optical flow is also used to calculate the score (motion amount) such that the two techniques are easily combined with common components producing a synergistic result.
Claim 2
In regards to claim 2, Jain discloses wherein, generating the bounding box for each of the one or more objects comprises
{section 3.1 indicates that the input is a video dataset that is already annotated by bounding boxes for the objects but does not discuss how these bounding boxes are generated}.
Muller also teaches generating the bounding box for each of the one or more objects comprises using one or more trained computer models
{see Sections 2 and 4 that annotates objects with bounding boxes including object classification and detection, deep network-based trackers}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Jain’s method of generating training data for one or more computer models that already receives a plurality of image data, wherein the image data comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images and which also generates the bounding boxes such that generating the bounding box for each of the one or more objects comprises using one or more trained computer models as taught by Mueller because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 3
In regards to claim 3, Jain is not relied upon to disclose but Mueller teaches wherein, the bounding box for each of the one or more objects comprises manual labelling by one or more human users {Section 3.3 includes various manual annotations/labels summarized in the last 10 entries of Table 2}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Jain’s method of generating training data for one or more computer models that already receives a plurality of image data, wherein the image data comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images and which also generates the bounding boxes such that wherein, the bounding box for each of the one or more objects comprises manual labelling by one or more human users as taught by Mueller because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 17
In regards to claim 17, Jain discloses outputting one or more pairs of bounding boxes for one or more common objects {see mapping of claim 1. Even if the pair of bounding boxes from sequential images that are used to determine the score (flow value exceeding a threshold) are not viewed as being output and used as training data, the sequential and iterative nature of Jain’s process would, at the next iteration, output another of the sequential images and bounding box of the common/persistent object for training provided that significant object motion (above the threshold) continues}
Claim 20
In regards to claim 20, Jain is not relied upon to disclose but Mueller teaches wherein the one or more common objects in two or more sequential images are identified by any or any combination of: manually identifying common objects between each of the images; matching objects detected in multiple images and applying a link between detected objects in different images {see the object classification and detection, deep network based trackers in section 4 that detect/match objects detected in temporally-related images of the video. See also the VATIC tool in section 3.2 that uses optical flow to match objects detected in multiple images}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Jain’s method of generating training data for one or more computer models that already receives a plurality of image data, wherein the image data comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images and which also generates the bounding boxes such that wherein the one or more common objects in two or more sequential images are identified by any or any combination of: manually identifying common objects between each of the images; matching objects detected in multiple images and applying a link between detected objects in different images as taught by Mueller because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 21 and 22
Jain discloses (claim 21) a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 1 and (claim 22) a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 1 {see sections 3.4 including the implementation details and section 4 Results in which the disclosed models are implemented on a computer having a processor, memory, algorithms and programs executing on computers to carry out the disclosed method and generate the disclosed output data (test results)
Claims 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Jain and Muller as applied to claim 1 above, and further in view of Zhu (US 20220188695 A1).
Claim 4
In regards to claims 4, Jain discloses determining at least two sequential images
{see above cites for claim 1 in which video data including time-sequential images (frames) are being processed to track the common objects across frames and while the use of timestamps and other time-related metadata is implied by such processing Jain does not explicitly mention such metadata or timestamp data}.
Zhu is an analogous reference from the same field of generating training data for computer models. See abstract, figs. 3 including step 308.
Zhu also teaches determining at least two sequential images based on metadata associated with the image data, (claim 5) wherein, the metadata associated with the image data comprises timestamp data {Fig. 3, step 304 tracks objects using a CNN and determines sequential images based on time stamp metadata, [0045].
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Jain’s determining at least two sequential images such that it is based on metadata associated with the image data, (claim 5) wherein, the metadata associated with the image data comprises timestamp data as taught by Zhu because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Jain and Muller as applied to claim 1 above, and further in view of Chen (US 20190325223 A1)
Claim 6
In regards to claim 6, Jain discloses determining at least two sequential images
Chen is an analogous reference that is reasonably pertinent to the problem faced by the inventor which is tracking bounding boxes/objects between frames. See abstract, Figs. 3C, 6, 7 and cites below.
Chen also teaches determining at least two sequential images based on similarities of determined one or more objects in at least two sequential images {Fig. 6, including calculation of affinity scores, [0016], [0049]-[0056]}
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Jain such that it determines at least two sequential images based on similarities of determined one or more objects in at least two sequential images as taught by Chen because Chen motivates doing so in [0056], because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 7 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Jain and Muller as applied to claim 1 above, and further in view of Verma (US 2023/0014805 A1).
Claim 7
In regards to claim 7, Mueller discloses wherein determining a correlation between the identified one or more common objects comprises
Verma is an analogous reference that is reasonably pertinent to the problem faced by the inventor which is tracking bounding boxes/objects between frames. Verma also teaches determining a correlation between the identified one or more common objects comprises comparing at least two corners of a bounding box of one or more common objects in a first image to at least two corresponding corners of a bounding box for the same one or more common objects in a sequential image {Figs 6A-D identifying ROI (bounding boxes) and operation 606 that compares corners as claimed, [0102]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified the base combination including Meuller’s object tracking such that it determines a correlation between the identified one or more common objects comprises comparing at least two corners of a bounding box of one or more common objects in a first image to at least two corresponding corners of a bounding box for the same one or more common objects in a sequential image as taught by Verma because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 12
In regards to claim 12, Jain is not relied upon to disclose but Venma teaches wherein generating a score is based on the deviation between at least two corners of a bounding box of one or more common objects in a first image and at least two corresponding corners of a bounding box for the same one or more common objects in a sequential image {Figs 6A-D identifying ROI (bounding boxes) and operation 606 that compares corners to generate a linear displacement score 608, [0102]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified the base combination including Meuller’s object tracking and Jain score generation such that wherein generating a score is based on the deviation between at least two corners of a bounding box of one or more common objects in a first image and at least two corresponding corners of a bounding box for the same one or more common objects in a sequential image as taught by Verma because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kulandai (US 20220405939 A1) discloses a computer-implemented method of generating training data for one or more computer models {See abstract, Figs. 1-3, 6 including selecting step 608 that selects training images from the video, and citations below}, the method comprising:
receiving a plurality of image data, wherein the image data comprises two or more sequential images wherein the image data comprises one or more objects determined in each of the plurality of images and wherein the image data comprises a bounding box identifying each of the one or more objects in each of the plurality of images and wherein the image data comprises one or more common objects identified in two or more sequential images {Fig. 1 including video capture devices 104 capturing video data 106 that is received by edge device 110, to determine object information 116 that includes bounding boxes for common objects, [0020]-[0021], [0026], Fig 6 including receiving step 602 that receives sampled video information (series of sequential images) captured by a video camera and including objects that are detected/determined, steps 604/606, that determine bounding boxes for common objects, [0045]-[0048]};
determining a correlation between the identified one or more common objects detected in two or more sequential images using the bounding boxes for each of the one or more common objects {Fig. 6, steps 608 and/or 610, 612; See also Fig. 4, step 410}.
GB 2585919 A discloses essentially the same concept but on the pixel level using segmentation rather than bounding boxes.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/ Primary Examiner, Art Unit 2667
Prosecution Insights