DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This is in response to Applicant’s Arguments/Remarks filed on December 23rd, 2025, which has been entered and made of record.
Response to Arguments
Claim Rejections - 35 USC § 103
Applicant’s arguments filed December 23rd, 2025 have been fully considered, but they are not persuasive.
Regarding claim(s) 1, applicant argues that:
“Fotland does not disclose a mechanism that specifically monitors for a complete failure of a primary tracker to distinctly trigger the user of a secondary tracker’s output for that frame" (Remarks, page 10) and “Dai does not address object tracking fusion logic, nor does it teach the specific “miss-triggered” hand off between two parallel trackers” (Remarks, page 10).
The Examiner respectfully disagrees with Applicant’s premise and conclusion. Fotland was cited as a reference teaching the use of multiple object trackers in parallel to track an object. As recited in the office action dated 09/23/2025, Fotland teaches choosing one tracker’s results over the other in the event of low confidence, however Fotland does not explicitly teach to use a tracking output of the one or more second object tracking based on determining that the first object tracking mechanism has missed a detection of the object wherein the first object tracking mechanism has missed the detection by failing to detect the object in the frame. Dai was cited to teach this limitation in combination with Fotland.
In at least ¶ [0079] Dai teaches, “object is independently tracked in parallel by the MALT 510 and by a TLD module 520”. Similar to [0061] of Fotland teaching, “an example of a maximum rule may be based on selection of the results of the tracking process having the highest level of confidence”, Dai teaches an embodiment where two trackers in parallel compute confidence scores and the results of the tracker with higher confidence is used as the tracking result, see [0079] of Dai teaching, “there are various ways to combine multiple tracking results, e.g., based on scoring and past performance. In one embodiment, the MALTED module 530 is configured to compute a confidence score for each tracking engine and select the tracking results from the tracking engine that provides the higher confidence score”. However, Dai additionally teaches in [0079] that when one tracking engine fails, the other tracking engine can be used to provide object tracking. See [0079], “Responsive to one of the tracking engines failing to perform, the other working tracking engine is used to provide object tracking”, and [0080] “the dual tracking engine model illustrated in FIG. 10 is very useful in instances when one tracking engine, e.g., a secondary object tracking module such as the TLD module 520 fails to track, while the other (e.g., the MALT 510) may still be working”. These passages of Dai illustrate the concept of using a secondary tracker’s result when a first tracker fails to track. It would have been obvious to one of ordinary skill in the art to have modified Fotland to include the teachings of Dai by including using a second tracker’s tracking results over a first tracker’s tracking results not only when a first tracker has a low confidence but also when the first tracker has failed to track. Doing so would improve the accuracy of object tracking by having a fail safe for a first tracker. See the rejection and combination rationale below.
As a result, there are insubstantial differences between the prior art element/teachings and the corresponding element/concept(s) disclosed in Applicant's filed Specification/PG Pub. The breadth of the claims permit the teachings of the prior art reference of Fotland in view of Dai to continue to read upon the claim language as currently stated because Applicant fails to explicitly define and describe subject matter in a way that prohibits the teachings of the Fotland and Dai combination from reading upon the claim language. Thus, the prior art of record does meet the limitation of the claims as disclosed within the rejection below.
Applicant's other arguments/remarks with respect to the current claims have been fully considered and given the appropriate weight, and so are believed to have been fully addressed in the Examiner’s response above and therefore not persuasive because they are fully met by the prior art of record as expressed in the rejection below.
Status of Claims
Claims 1-20 are pending. No claim(s) were amended. No claim(s) were canceled. No new claim(s) were added. Claims 1-20 are considered below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 10-11, 14, 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Fotland (US 20150055821 A1) in view of Dai (US 20140347263 A1).
Regarding Claim 1, representative of Claims 10 and 18, Fotland teaches an apparatus comprising:
at least one processor ([0093]: execution by the processor 702); and
at least one memory storing instructions that ([0093]: first data storage for program instructions for execution by the processor 702), when executed by the at least one processor, cause the apparatus at least to;
use a first object tracking mechanism to detect and associate one or more objects from across frames of a video ([abstract]: system and method for tracking an object using multiple tracking processes, [0014]: each camera can comprise a digital still camera, configured to capture subsequent frames in rapid succession, or a video camera able to capture streaming video);
initiate one or more second object tracking mechanisms ([0035]: an object detection process can first be used to determine the position of a representation of an object of interest in the first images and multiple object tracking processes can be used to track the object of interest in subsequent images…in at least some embodiments, one or more object detection processes can be included as a part of the comprehensive object tracking process) to track the one or more objects detected by the first object tracking mechanism across frames of the video in parallel with the first object tracking mechanism ([abstract]: system and method for tracking an object using multiple tracking processes…the multiple tracking processes can be run in parallel); and
use a tracking output of the one or more second object tracking for a frame of the video based on determining that first object tracking mechanism has missed a detection of the object in the frame of the video ([0061]: multiple object tracking processes can provide a respective estimated position of the object of the interest and a level of confidence, [0061]: minimum rule may be based on selection of the results of the tracking process having the lowest error rate),
Although Fotland teaches choosing one tracker’s results over the other in the event of low confidence, Fotland does not explicitly teach wherein the first object tracking mechanism has missed the detection by failing to detect the object in the frame.
Dai teaches wherein the first object tracking mechanism has missed the detection by failing to detect the object in the frame ([0079]: …responsive to one of the tracking engines failing to perform, the other working tracking engine is used to provide object tracking).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present invention to have modified the teachings of Fotland to include the teachings of Dai. Doing so would improve the accuracy of object tracking by enabling one tracker to provide tracking information for an object of interest when another tracker fails.
Regarding Claim 2, representative of Claims 11 and 19, the Fotland and Dai combination teaches the apparatus of claim 1. In addition, Fotland teaches wherein the first object tracking mechanism is based on deep neural network (DNN)-based object tracking ([0061]: multiple object tracking processes can provide a respective estimated position of the object of the interest, [0062]: determination of a position of an object of interest from multiple tracking processes may be based on classification approaches or estimation approaches. Classification approaches can be based on …neural networks), and wherein the one or more second object tracking mechanisms are based on region of interest object (ROI) tracking ([0015]: the object detection process is adapted for locating the head or face of a person. Here, the object detection process locates the head or face of the user 102 within the image 114 and provides as output the dimensions of a bounded box 112, [0035]: object detection process can first be used to determine the position of a representation of an object of interest in the first images and multiple object tracking processes can be used to track the object of interest).
Regarding Claim 5, representative of Claim 14, the Fotland and Dai combination teaches the apparatus of claim 1. In addition, Fotland teaches wherein a respective second object tracking mechanism of the one or more second object tracking mechanisms is respectively initiated for an individual object of the one or more objects ([0017]: various embodiments include tracking of one or more objects of interest. [0035]: multiple object tracking processes can be used to track the object of interest. Examiner notes the limitation under BRI requires at minimum, one second tracking mechanism initiated for an individual object of at least one object. This limitation as written does not necessitate initiating a plurality of second tracking mechanisms, one for each object of a plurality of objects).
Claim(s) 3-4, 6-7, 12-13, 15-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Fotland (US 20150055821 A1) and Dai (US 20140347263 A1), in view of Sundaresan (US 20190114804 A1).
Regarding Claim 3, representative of Claims 12 and 20, the Fotland and Dai combination teaches the apparatus of claim 2. However, neither Fotland nor Dai explicitly teach the remaining limitations of claim 3. Sundaresan teaches wherein a region of interest object (ROI) to be tracked by the ROI tracking is provided by the deep-neural network-based object tracking on the initiating of the ROI tracking ([0092]: if the confidence level of the bounding box of the object received from the strong object tracker 106 is higher than the confidence level of the bounding box currently being used to track the object…the lightweight object tracker 108 replaces (at block 410) the current tracked bounding box for the object with the newly received bounding box, [0059] The lightweight object tracker 108 is initialized using results from the neural network detection system 104 and the strong object tracker 106).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the teachings of Fotland and Dai to include the teachings of Sundaresan. Doing so would improve object tracking accuracy by initiating a second tracker with the results of a first tracker if it has a higher confidence than the second tracker during the tracking process.
Regarding Claim 4, representative of Claim 13, the Fotland and Dai combination teaches the apparatus of claim 1. However, neither Fotland nor Dai explicitly teach the remaining limitations of claim 4. Sundaresan teaches wherein the at least one memory storing the instructions that, when executed by the at least one processor, cause the apparatus at least to: reinitiate the second object tracking mechanism based on the detection of the object by the first object tracking mechanism in a subsequent frame of the video ([0092]: if the confidence level of the bounding box of the object received from the strong object tracker 106 is higher than the confidence level of the bounding box currently being used to track the object…the lightweight object tracker 108 replaces (at block 410) the current tracked bounding box for the object with the newly received bounding box, [0059] The lightweight object tracker 108 is initialized using results from the neural network detection system 104 and the strong object tracker 106).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present invention, to have modified the Fotland and Dai combination with Sundaresan. Dai teaches that a failure of a first tracker may be substituted by the results of another tracker, however Dai does not explicitly mention if the first tracker will be reinitiated. Sundaresan teaches reinitiation of a tracker based on the results of another tracker. The modification would improve the accuracy of tracking in subsequent frames after one tracker experiences a tracking failure or low confidence tracking result.
Regarding Claim 6, representative of Claim 15, the Fotland and Dai combination teaches the apparatus of claim 1. However, neither explicitly teach the remaining limitations of claim 6. In addition, Sundaresan teaches wherein the at least one memory storing the instructions that, when executed by the at least one processor, cause the apparatus at least to: crop the frame of the video based on a bounding box of the tracking output of the one or more second object tracking mechanisms ([0105]: a ROI in a video frame can be determined based on feedback from object tracking. Examiner interpreting this can be implemented with either tracker. [0105]: an ROI corresponding to a bounding box of a tracked object can be used...such an example, the bounding box itself, or a region surrounding the bounding box, can be used as the region that will be cropped from an input video frame); and perform a re-identification of the one or more objects based on the cropped frame ([0105]: an ROI corresponding to a bounding box of a tracked object can be used to determine a region from a video frame to use for object detection and classification).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the Fotland and Dai combination to include the teachings of Sundaresan. Doing so would improve processing speed by performing processing using a cropped frame rather than the entire video frame.
Regarding Claim 7, representative of Claim 16, the Fotland and Dai combination teaches the apparatus of claim 1. However, neither explicitly teach the remaining limitations of claim 7. In addition, Sundaresan teaches wherein the at least one memory storing the instructions that, when executed by the at least one processor, cause the apparatus at least to: resize or crop the frame for input to the one or more second object tracking mechanisms ([0096]: cropped portion of a second video frame (that is subsequent to the first video frame in a video sequence) can be processed by the neural network detection system 104 for a next iteration).
Claim(s) 8-9, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Fotland (US 20150055821 A1) and Dai (US 20140347263 A1), in view of Gao (T. Gao, H. Pan, Z. Wang and H. Gao, "A CRF-Based Framework for Tracklet Inactivation in Online Multi-Object Tracking," in IEEE Transactions on Multimedia, vol. 24, pp. 995-1007, Mar. 2021, doi: 10.1109/TMM.2021.3062489).
Regarding Claim 8, representative of Claim 17, the Fotland and Dai combination teaches the apparatus of claim 1. However, neither explicitly teach the remaining limitations of Claim 8. Gao teaches wherein the at least one memory storing the instructions that, when executed by the at least one processor, cause the apparatus at least to:
generate a tracklet respectively for the one or more objects based on the first object tracking mechanism, the one or more second object tracking mechanisms, or a combination thereof, wherein the tracklet is a sequence of detections across a plurality of frames of the video ([Section III.A]: detects the tracking objects of interest through a Faster R-CNN detector. With the arrival of each new frame, the latest bounding box of each tracklet is treated as the region proposal, [Section III. B, paragraph 1]: our goal is to make a reasonable judgment …labeling whether the tracklets should be inactivated. Here, we use … ensemble tracking information of the tracklet i at frame t).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the Fotland and Dai combination to include the teachings of Gao. Doing so would improve object tracking efficiency by generating tracklets which can be used in classification to determine which tracklets should be active or inactive, minimizing downstream processing power needed for object tracking.
Regarding Claim 9, the Fotland, Dai, and Gao combination teaches the apparatus of claim 8. In addition, Gao teaches wherein the at least one memory storing the instructions that, when executed by the at least one processor, cause the apparatus at least to: classify the tracklet as active, inactive, tracked, and/or fragmented based on the first object tracking mechanism, the one or more second object tracking mechanisms, or a combination thereof ([Section III. B, paragraph 1]: our goal is to make a reasonable judgment …labeling whether the tracklets should be inactivated. Here, we use … ensemble tracking information of the tracklet i at frame t, [introduction, right column, paragraph 4]: inactivates tracklets by setting a fixed threshold on the classification scores of tracking hypotheses, [Section V. A]: evaluation systems…categories as follows…Mostly Tracked (MT)…Fragments (Frag)).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANICE VAZ whose telephone number is (703)756-4685. The examiner can normally be reached Monday-Friday 9:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANICE E. VAZ/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667