DETAILED ACTION
This Office Action is responsive to the Applicant’s submission, filed on May 15, 2026, amending claims 1, 6, 7 and 9, and cancelling claims 4 and 5. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 and 6-9 are rejected under 35 U.S.C. 103 as being unpatentable over article entitled, “A semi-automatic system for ground truth generation of soccer video sequences” by D’Orazio et al. (“D’Orazio”), over U.S. Patent No. 12,236,685 to Pan et al. (“Pan”), over the article entitled “Online pedestrian tracking with multi-stage re-identification” by Jiang et al. (“Jiang”), and also over U.S. Patent Application Publication No. 2022/0092348 to Jakobsen et al. (“Jakobsen”).
Regarding claims 1 and 9, D’Orazio describes a semi-automatic system that generates an initial ground truth estimation for image sequences, and then provides a user interface for the manual validation or correction of the ground truth estimations (see e.g. the Abstract). Like claimed, D’Orazio particularly teaches:
detecting a moving object in a sequences of images, and tracking a same moving object in the sequence of images by using a tracker to automatically obtain, as tracking result data, a track that is information representing a time series of the same moving object in the sequence of images (D’Orazio discloses that the system generates the initial ground truth estimates by, in part, processing the sequence of images to detect moving objects therein, and using a tracking algorithm to provide a track for each detected moving object:
In this paper we present a semiautomatic system for ground truth generation that provides video annotation file in XML format, compatible with the ViPER toolkit that can be used for image visualization. The proposed system consists on two steps for the initial generation of ground truth estimations and their manual validation by a human operator. In the first step, the whole sequence is processed applying a background segmentation that detects the moving objects, and a tracking algorithm that provides for each blob the track data. In the second step, by using a user friendly interface, all the data provided in the first step, is validated frame by frame by the human operator. In this way he has only to correct the blob dimensions, if the segmentation algorithm was not precise, and also to solve the group blobs if the tracking algorithm failed. In addition, the operator has the possibility to associate some semantic information to each blob (as the type of interactions among objects), and eventually propagates this label to all the frames in which the same blob remains in the scene. The system was tested on multi-view soccer image sequences. Experiments demonstrated that the ground truth data were generated faster than by using the ViPER toolkit. The sequence download is available at the authors’ Institute website [13].
(Section I. “Introduction.” Emphasis added.).
The automatic processing of the whole sequences consists on two steps: first of all a foreground segmentation algorithm is applied to detect the moving object in the scene. Then a tracking algorithm, based an the spatial contiguity of the moving blobs, associates the track information to each blob. In this paper we propose a background subtraction and a tracking algorithm that were demonstrated to be effective with static cameras, people moving during all the sequence and varying lighting conditions. Anyway the initial processing could be substituted with any other algorithms that work in different context providing different results.
(Section II. “Automatic Processing.” Emphasis added.).
D’Orazio describes particular algorithms for detecting the moving objects and for tracking the detected moving objects – see section II.A “Moving Object Segmentation” and section II.B “Moving Object Tracking” – but suggests that other algorithms can alternatively be used as well, as is indicated in the above excerpt. The algorithm for tracking the detected moving objects is considered a tracker like claimed, and obtains a track that is information representing a time series of the same moving object in the sequence of images. The track data obtained by the tracker for each detected object is considered tracking result data like claimed.); and
generating labeled training data by giving the track as a label to the sequence of images and based on the tracking result data (D’Orazio describes a user interface that enables an editor to validate or correct the detected objects and tracks:
In this paper we present a semiautomatic system for ground truth generation that provides video annotation file in XML format, compatible with the ViPER toolkit that can be used for image visualization. The proposed system consists on two steps for the initial generation of ground truth estimations and their manual validation by a human operator. In the first step, the whole sequence is processed applying a background segmentation that detects the moving objects, and a tracking algorithm that provides for each blob the track data. In the second step, by using a user friendly interface, all the data provided in the first step, is validated frame by frame by the human operator. In this way he has only to correct the blob dimensions, if the segmentation algorithm was not precise, and also to solve the group blobs if the tracking algorithm failed. In addition, the operator has the possibility to associate some semantic information to each blob (as the type of interactions among objects), and eventually propagates this label to all the frames in which the same blob remains in the scene. The system was tested on multi-view soccer image sequences. Experiments demonstrated that the ground truth data were generated faster than by using the ViPER toolkit. The sequence download is available at the authors’ Institute website [13].
(Section I. “Introduction.” Emphasis added.).
D’Orazio discloses that the user interface presents the results of the object detection and tracking by superimposing bounding boxes and other information on image frames in the sequence of images:
In figure 3 a portion of one of the images after the first automatic processing step of our system is shown. In this image, the segmentation algorithm and the tracking algorithm have suggested the bounding boxes on the players and the lifetime of each blob.
At this point the human operator has to modify those situations in which algorithms provided imprecisions. As it can be seen in figure 4 some blobs have not the correct dimensions since either the shoes or the hands were not correctly segmented. The numbers superimposed on each blob represent the track ID and the corresponding lifetime provided by the tracking algorithm. Generally the lifetime is correctly updated also when there is a temporary failure of the segmentation algorithm. Indeed in these cases the tracking algorithm maintains the information on previous blobs and when a new one appears in the image in close positions compatible with those of previous tracks, it checks their similarity and associates the track parameters between the two blobs. In figure 4 the ground truth editor allows the human operator to draw manually the two bounding boxes that solve the merge blob and to associate at each of them the attribute parameters of previous frames. These bounding boxes chaining can be done both in the backward and in the forward direction and the editor automatically updates all the dynamic attributes of that track (such as the lifetime). In the manual data validation phase the operator has the possibility to associate to each blob also other information such as the belonging team (Team A, Team B, Goalkeeper A, Goalkeeper B, and Referee). This step is done just on one frame and also in this case it is propagated to all the bounding boxes of the same track. In figure 4 each bounding box has also the class attribute that is the same for players of the same team.
(Section IV. “Experimental Results.” Emphasis added.).
The sequence of images having the superimposed track annotations can be considered labeled training data in which a track is given as a label to the sequence of images based on the tracking result data. Alternatively, the sequence of images after the user validation step also can be considered labeled training data like claimed.).
D’Orazio thus teaches a training data generation method similar to that of claim 1. The computing system necessary for performing the object detection and tracking, and for presenting the user interface described by D’Orazio, is considered a training data generation system similar to that of claim 9. D’Orazio further suggests enabling a user to integrate two or more tracks into a single track (e.g. by “chaining” bounding boxes – see e.g. section III. “Manual Data Validation”). However, D’Orazio does not explicitly teach implementing a track integration process like required by claims 1 and 9, and which includes: (i) determining an occurrence of overlapping tracks such that two or more different tracks are given to the same moving object based on an exit from a field of view of a camera and a re-entry to the field of view of the camera; (ii) detecting the overlapping tracks based on the occurrence; (iii) inputting the sequence of images into a feature extraction model to extract a feature amount of each moving object detected in the sequence of images and calculate a degree of similarity between the moving objects based on the extracted feature amount; and (iv) when the degree of similarity between a first moving object of a first track and a second moving object of a second track is higher than a threshold and based on the tracking result data, determining that the first moving object and the second moving object are identical and integrating the first track and the second track into a single track. D’Orazio also does not explicitly teach modifying a result of the track integration process via a human machine interface, and using the generated labeled training data for supervised learning or semi-supervised learning of an object identification model based on machine learning, as is further required by claims 1 and 9.
Pan nevertheless describes a track integration process that involves: (i) determining an occurrence of overlapping tracks such that two or more different tracks are given to the same moving object (e.g. the same pedestrian) (see e.g. column 1, lines 50-59; and column 9, lines 26-52); (ii) detecting the overlapping tracks based on the occurrence (see e.g. column 1, lines 50-59; and column 9, lines 26-52); (iii) inputting a sequence of images into a feature extraction model to extract a feature amount of each moving object (e.g. pedestrian) detected in the sequence of images and calculate a degree of similarity between the moving objects based on the extracted feature amount (see e.g. column 2, line 63-67; column 3, line 28 – column 4, line 18; column 5, lines 7-22; column 7, lines 6-65; and column 8, lines 56-61); and (iv) when the degree of similarity between a first moving object of a first track and a second moving object of a second track is higher than a threshold and based on tracking result data, determining that the first moving object and the second moving object are identical and integrating the first track and the second track into a single track (see e.g. column 2, line 63-67; column 3, line 28 – column 4, line 18; column 5, lines 7-22; column 7, lines 6-65; and column 8, lines 56-61).
It would have been obvious to one of ordinary skill in the art, having the teachings of D’Orazio and Pan before the effective filing date of the claimed invention, to modify the method and system taught by D’Orazio so as to also implement a track integration process like taught by Pan, which includes: (i) determining an occurrence of overlapping tracks such that two or more different tracks are given to the same moving object; (ii) detecting the overlapping tracks based on the occurrence; (iii) inputting the sequence of images into a feature extraction model to extract a feature amount of each moving object detected in the sequence of images and calculate a degree of similarity between the moving objects based on the extracted feature amount; and (iv) when the degree of similarity between a first moving object of a first track and a second moving object of a second track is higher than a threshold and based on the tracking result data, determining that the first moving object and the second moving object are identical and integrating the first track and the second track into a single track. It would have been advantageous to one of ordinary skill to utilize such a track integration process because it would better ensure that the same object is labeled consistently (i.e. with the same track) throughout the video, as is evident from Pan (see e.g. column 1, lines 37-59). D’Orazio generally teaches presenting results of the object detection and tracking to a human checker via a humane machine interface, whereby the user can modify or validate the results (see e.g. section III. “Manual Data Validation”). It thus follows that the presentation of the results of the object detection and tracking would further entail presenting a result of the track integration process to the human checker, whereby the user can modify a result of the track integration process via the human machine interface and feed back the result to the tracking result data (e.g. for storage). Accordingly, D’Orazio and Pan are considered to teach a method similar to that of claim 1 and a system similar to that of claim 9. D’Orazio and Pan, however, do not explicitly disclose that the two or more tracks are given to the same moving object based on an exit from a field of view of a camera and a re-entry to the field of view of the camera, as is required by claims 1 and 9. D’Orazio and Pan also do not explicitly teach using the generated labeled training data for supervised learning or semi-supervised learning of an object identification model based on machine learning, as is further required by claims 1 and 9.
Jiang nevertheless generally teaches that objects being tracked sometimes leave a field of view of a camera, but later re-enter the field of view of the camera (see e.g. the Abstract). Jiang further teaches integrating such object information into a single track (see e.g. section 2.4 “Second re-identification stage”).
It would have been obvious to one of ordinary skill in the art, having the teachings of D’Orazio, Pan and Jiang before the effective filing date of the claimed invention, to modify the method and system taught by D’Orazio and Pan such that the track integration process can also be applied to two or more different tracks given to the same moving object based on an exit from a field of view of a camera and a re-entry to the field of view of the camera, as is taught by Jiang. It would have been advantageous to one of ordinary skill to utilize such a combination because objects sometimes leave the field of view of a camera and return, as is taught by Jiang (see e.g. the Abstract); the resulting combination would be able to address such situations, as would have further been apparent to one of ordinary skill in the art.
Jakobsen generally teaches generating labeled training data from a sequence of images, wherein the labeled training data is used for supervised learning or semi-supervised learning of an object identification model (i.e. a re-identification model) based on machine learning (see e.g. paragraphs 0002-0003, 0006-0008 and 0048-0050).
It would have been obvious to one of ordinary skill in the art, having the teachings of D’Orazio, Pan, Jiang and Jakobsen before the effective filing date of the claimed invention, to modify the method and system taught by D’Orazio, Pan and Jiang so as to use the generated labeled training data for supervised learning or semi-supervised learning of an object identification model based on machine learning, as is taught by Jakobsen. It would have been advantageous to one of ordinary skill to utilize such a combination, because it would provide for a more efficient training, as is suggested by Jakobsen (see e.g. paragraphs 0003 and 0022). Accordingly, D’Orazio, Pan, Jiang and Jakobsen are considered to teach, to one of ordinary skill in the art, a training data generation method like that of claim 1 and a training data generation system like that of claim 9, which generate labeled training data for training an object identification model that is based on machine learning.
As per claim 6, D’Orazio teaches presenting results of the object detection and tracking to a human checker via a user interface (see e.g. section I. “Introduction,” which recites: “In the first step, the whole sequence is processed applying a background segmentation that detects the moving objects, and a tracking algorithm that provides for each blob the track data. In the second step, by using a user friendly interface, all the data provided in the first step, is validated frame by frame by the human operator.”). As described above, it would have been obvious to modify the method taught by D’Orazio so as to also utilize a track integration process like taught by Pan. It thus follows that the presentation of the results of the object detection and tracking would further entail presenting a result of the track integration process to the human checker. Accordingly, the above-described combination of D’Orazio, Pan, Jiang and Jakobsen is further considered to teach a training data generation method like that of claim 6.
As per claim 7, D’Orazio teaches presenting results of the object detection and tracking to a human checker via a user interface (see e.g. section I. “Introduction,” which recites: “In the first step, the whole sequence is processed applying a background segmentation that detects the moving objects, and a tracking algorithm that provides for each blob the track data. In the second step, by using a user friendly interface, all the data provided in the first step, is validated frame by frame by the human operator.”). As described above, it would have been obvious to modify the method taught by D’Orazio so as to also utilize a track integration process like taught by Pan. The track integration process taught by Pan is automatic (see e.g. column 1, lines 37-59). It thus follows that the results of the track integration process can be reflected in the labeled training data. Accordingly, the above-described combination of D’Orazio, Pan, Jiang and Jakobsen is further considered to teach a training data generation method like that of claim 7.
As per claim 8, it would have been obvious, as is described above, to modify the method taught by D’Orazio, Pan and Jiang so as to use the generated labeled training data for supervised learning or semi-supervised learning of an object identification model like taught by Jakobsen. Jakobsen particularly teaches that the object identification model is a human re-identification model (see e.g. paragraphs 0002-0003 and 0006-0008). Accordingly, the above-described combination of D’Orazio, Pan, Jiang and Jakobsen is further considered to teach a training data generation method like that of claim 8.
Claims 2 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of D’Orazio, Pan, Jiang and Jakobsen described above, and also over WIPO Publication No. WO 2021/201774 A1 to Wang et al. (“Wang”).
Regarding claim 2, D’Orazio, Pan, Jiang and Jakobsen teach a training data generation method like that of claim 1, as is described above, and which comprises steps for detecting a moving object in a sequence of images and for tracking a same moving object in the sequence of images by using a tracker. D’Orazio further teaches using a bounding box to represent a location of the detected moving object in the sequence of images (see e.g. Section IV. “Experimental Results,” which recites: “In figure 3 a portion of one of the images after the first automatic processing step of our system is shown. In this image, the segmentation algorithm and the tracking algorithm have suggested the bounding boxes on the players and the lifetime of each blob.”). D’Orazio, Pan, Jiang and Jakobsen, however, do not explicitly disclose that the tracker tracks the same moving object based on a movement of the bounding box, without performing feature extraction, as is further required by claim 2.
Wang nevertheless describes a tracker for tracking a moving object within a sequence of images, wherein a target object label (e.g. a bounding box) is applied to each image in the sequence to represent a location of the moving object in the sequence of images, and the tracker tracks the same moving object in the sequence of images based on a movement of the bounding box, without performing feature extraction (i.e. by detecting the bounding box movement instead of movement of the object itself) (see e.g. page 2, line 20 – page 3, line 6; page 3, lines 18-23; page 20, lines 15-27; and page 21, line 27 – page 22, line 12).
It would have been obvious to one of ordinary skill in the art, having the teachings of D’Orazio, Pan, Jiang, Jakobsen and Wang before the effective filing date of the claimed invention, to modify the method taught by D’Orazio, Pan, Jiang and Jakobsen so as to use the tracker taught by Wang to track the moving objects in the sequence of images, whereby a bounding box represents a location of a detected moving object in the sequence of images, and the tracker tracks the same moving object based on a movement of the bounding box, without performing feature extraction. It would have been advantageous to one of ordinary skill to utilize such a tracker because it can improve the accuracy of the object detection and tracking, as is taught by Wang (see e.g. page 3, lines 1-6). Accordingly, D’Orazio, Pan, Jiang, Jakobsen and Wang are considered to teach, to one of ordinary skill in the art, a training data generation method like that of claim 2.
As per claim 3, it would have been obvious, as is described above, to modify the method taught by D’Orazio, Pan, Jiang and Jakobsen so as to use the tracker taught by Wang to track the moving objects in the sequence of images. Wang particularly teaches that the tracker associates multiple bounding boxes representing the same moving object in the sequence of images with each other (e.g. based on a threshold distance), and provides a track comprising information indicating the multiple bounding boxes representing the same moving object in the sequence of images (see e.g. page 21, line 27 – page 22, line 12; and page 23, lines 1-18). Accordingly, the above-described combination of D’Orazio, Pan, Jiang, Jakobsen and Wang is further considered to teach a method like that of claim 3.
Response to Arguments
The Examiner acknowledges the Applicant’s amendments to claims 1, 6, 7 and 9, and cancellation of claims 4 and 5. In response to these amendments, the 35 U.S.C. § 112(b) rejection presented in the previous Office Action to claim 7 is respectfully withdrawn, as is the 35 U.S.C. § 101 rejection presented in the same Office Action to claims 1-9. Moreover, the double patenting rejection presented in the previous Office Action is respectfully withdrawn in light of the Terminal Disclaimer filed on May 15, 2026.
The Applicant’s arguments concerning the 35 U.S.C. §§ 102 and 103 rejections presented in the previous Office Action have been considered, but are moot in view of the new grounds of rejection presented above, which are required in response to the Applicant’s amendments.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BLAINE T BASOM whose telephone number is (571)272-4044. The examiner can normally be reached Monday-Friday, 9:00 am - 5:30 pm, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached at (571)270-3264 . The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BTB/
7/29/2026
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights