045DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 8/27/25 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1- is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US PG Pub 2021/0312642 to Zhang et al.
Regarding claim 1. Zhang discloses a method for user-assisted object tracking (Abstract, & “It can also be implemented with user interaction” paragraph 15), the method comprising:
receiving a first image frame of a sequence of image frames (“The detection module 102 initializes one or more tracking parameters 120 of the object tracker module 108 by learning a corresponding one or more appearance features of the target object 104 in a frame in which the target object was detected. The frame refers to an image in a sequence of video frames or a sequence of images captured in a video stream and the like”, paragraph 17);
performing object detection using an object tracker to identify an object of interest in the first image frame (“In subsequent frames, the object tracker module 108 searches around the last known position of the target object 104. The last known position of the target object is the position after the initial detection or the position after the last time the target object was successfully tracked in a previous frame”, paragraph 19) , based upon one or more templates associated with the object of interest in a template repository (“if the target object 104 was tracked and successfully identified based on the object tracker module 108 parameter settings, new samples containing the target object 104 can be obtained from the frames in which they are found using the tracking position. The new samples, also referred to as patches that are cropped from the frames, are used to update 118 the object tracking model 122 to accommodate target object deformation”, paragraph 21, “the continuous learning framework 100 improves tracking by continuously accumulating samples on a large scale and storing the accumulated samples in a continuous learning samples database 112”, paragraph 26);
outputting a first indicator associated with a first image portion corresponding to the object of interest (“The tracking result is a tracking response map of the candidate object in the tracked area”, paragraph 33, “the dotted line boundary demarcating a portion of the frame that contains the tracked object”, paragraph 50);
receiving a user input associated with the object of interest, the user input indicating an identified image portion in the first image frame (“an object detection module 102 is initialized with the target object 104. For example, the target object 104 can be selected with a manual interaction or the object detection module 102 can auto-detect the target object 104”, paragraph 28);
generating a retargeted template, based at least in part on the identified image portion; and determining a second image portion associated with the object of interest in a second image frame of the sequence of image frames, using the object tracker, based at least in part on the retargeted template (“if the target object 104 was tracked and successfully identified based on the object tracker module 108 parameter settings, new samples containing the target object 104 can be obtained from the frames in which they are found using the tracking position. The new samples, also referred to as patches that are cropped from the frames, are used to update 118 the object tracking model 122 to accommodate target object deformation”, paragraph 21, “the continuous learning framework 100 improves tracking by continuously accumulating samples on a large scale and storing the accumulated samples in a continuous learning samples database 112”, paragraph 26);
wherein the second image frame is after the first image frame in the sequence of image frames (“at process 208, using the target object 104 the object tracker module 108 tracks potential candidate objects in subsequent frames of the video images, such as during training and/or during subsequent long-term object tracking. The target object 104 can be the initial target object from process 202, or a re-detected target object from process 212. The image(s) 210 in which the target object 104 is tracked includes one or more video images, also referred to as frames, from any one or more views (multi-views) generated by multiple image capture devices”, paragraph 30);
wherein the second image portion is different from the first image portion (“new samples containing the target object 104 can be obtained from the frames in which they are found using the tracking position. The new samples, also referred to as patches that are cropped from the frames, are used to update 118 the object tracking model 122 to accommodate target object deformation”, paragraph 21);
wherein the method is performed using one or more processors (“a computing system such as platform 800 may include a processing means such as one or more processors 810”, paragraph 53).
Regarding claim 2. Zhang discloses generating a second indicator associated with the second image portion (Fig, 6 & 7, “the open-loop results 602b maintain tracking of the street performer as indicated by the dotted line boundary demarcating a portion of the frame that contains the tracked object, i.e. the female street performer. For example, in FRAME 8 of FIG. 6C and FRAME 9 of FIG. 6D, the female street performer is nearly perfectly centered within the dotted line boundary. By FRAME 10 of FIG. 6E the closed-loop results 602a drift further to the right of the female street performer, whereas the open-loop results 602b continue to be nearly perfectly centered within the dotted line boundary. Each of the open-loop results 602b will be stored as continuous learning training samples cropped from FRAME 0, FRAME 3, FRAME 8, FRAME 9 and FRAME 10 in respective FIGS. 6A-6E. The open-loop results 602b as tracking progresses to FRAME 10 in FIG. 6E indicate that the discriminatory classifier 504 benefits from the continuous learning training samples to improve the accuracy of detection and identification during long-term object tracking”, paragraph 50).
Regarding claim 3. Zhang discloses storing the retargeted template to a long-term template repository of the template repository (“the object tracker module 108 can use the re-detected target object 104 to continue long-term tracking without losing the valuable information in the accumulated samples stored in the continuous learning sample database 112”, paragraph 35).
Regarding claim 4. Zhang discloses resetting a short-term template repository of the template repository by removing one or more short-term templates from the short-term template repository (“In one embodiment, a set of five tracking result patches 408 are cropped from the frames that encompass candidate objects representing the tracked referee in various poses and with varying amounts of target deformation as compared to the appearance of the tracked referee in the initial target object 404”, paragraph 39).
Regarding claim 5. Zhang discloses identifying a third image portion associated with the object of interest, using the object tracker, on a third image frame of the sequence of image frames, based at least in part on the retargeted template, wherein the third image frame is after the second image frame in the sequence of image frames; determining a confidence level associated with the third image portion; evaluating whether the confidence level meets one or more predetermined criteria; in response to the confidence level meeting the one or more predetermined criteria: generating a short-term template, based on the third image portion; and adding the short-term template to the short-term template repository (“At decision process 216, if the average distance between the extracted appearance features present in the tracked area and the appearance features present in target object 104 and accumulated samples is smaller than an identity threshold, the tracking result is successful, increasing the likelihood of success in continuing to track the target object 104 without resetting the object tracker module 108, including without having to re-detect the target object 104. The tracking result is a tracking response map of the candidate object in the tracked area. Upon successful tracking, at process 218, the continuous learning module 110 can proceed to crop a sample from frame in which a successfully tracked candidate object appears, and to collect all such cropped samples in a continuous learning sample dataset for the target object 104 for eventual storage in the continuous learning samples database 110”, paragraph 33).
Regarding claim 9. Zhang discloses wherein the user input is a second user input and the identified image portion is a second identified image portion (Fig, 6 & 7, “the open-loop results 602b maintain tracking of the street performer as indicated by the dotted line boundary demarcating a portion of the frame that contains the tracked object, i.e. the female street performer. For example, in FRAME 8 of FIG. 6C and FRAME 9 of FIG. 6D, the female street performer is nearly perfectly centered within the dotted line boundary. By FRAME 10 of FIG. 6E the closed-loop results 602a drift further to the right of the female street performer, whereas the open-loop results 602b continue to be nearly perfectly centered within the dotted line boundary. Each of the open-loop results 602b will be stored as continuous learning training samples cropped from FRAME 0, FRAME 3, FRAME 8, FRAME 9 and FRAME 10 in respective FIGS. 6A-6E. The open-loop results 602b as tracking progresses to FRAME 10 in FIG. 6E indicate that the discriminatory classifier 504 benefits from the continuous learning training samples to improve the accuracy of detection and identification during long-term object tracking”, paragraph 50), and wherein the method further comprises:
receiving a first user input associated with a first identified image portion on an initial image frame of the sequence of image frames (“an object detection module 102 is initialized with the target object 104. For example, the target object 104 can be selected with a manual interaction or the object detection module 102 can auto-detect the target object 104”, paragraph 28,);
generating an initial template based at least in part on the first identified image portion (“The detection module 102 initializes one or more tracking parameters 120 of the object tracker module 108 by learning a corresponding one or more appearance features of the target object 104 in a frame in which the target object was detected. The frame refers to an image in a sequence of video frames or a sequence of images captured in a video stream and the like”, paragraph 17); and initializing the object tracker based at least in part on the initial template (“if the target object 104 was tracked and successfully identified based on the object tracker module 108 parameter settings, new samples containing the target object 104 can be obtained from the frames in which they are found using the tracking position. The new samples, also referred to as patches that are cropped from the frames, are used to update 118 the object tracking model 122 to accommodate target object deformation”, paragraph 21, “the continuous learning framework 100 improves tracking by continuously accumulating samples on a large scale and storing the accumulated samples in a continuous learning samples database 112”, paragraph 26).
Regarding claim 10. Zhang discloses identifying a plurality of objects using a software detector (“FIG. 2 is a schematic block diagram illustration of an architecture 200 for implementing continuous learning for long-term object tracking in accordance with various examples described herein. In one embodiment, at process 202, an object detection module 102 is initialized with the target object 104. For example, the target object 104 can be selected with a manual interaction or the object detection module 102 can auto-detect the target object 104, including re-detecting the target object, the latter typically based on prior manual and/or training detections of the target object 104”, paragraph 28), the software detector including a machine-learning model (“FIG. 1, to begin tracking, an object detection module 102 initializes an object tracker module 108 to track a particular object, referred to herein as a target object 104. The object detection module 102 provides an accurate bounding box for any specified target. It can be implemented using state-of-the-art object detectors, such as FAST R-CNN (Fast Region-Based Convolutional Network), Mask R-CNN (Mask Region-Based Convolutional Network), SSD (Single Shot Detection Neural Network), YOLO (You Only Look Once real-time object detection), etc. It can also be implemented with user interaction”, paragraph 16);
comparing each object of the plurality of objects with the initial template; determining that one object of the plurality of objects matches to the initial template (“Generally, once the target object 104 is initialized with a bounding box, the object tracker module 108 will initialize the object tracking model 122 parameters 120 with a given annotation corresponding to an appearance feature of the target object 104. The object tracker module's parameters 120 depend on the type of object tracking model 122 being used. In this case, the module's parameters 120 correspond to the learned appearance features of the target object 104 being tracked. The learned appearance features can include such characteristics as color, shape and so forth. Such features are used to discriminate one detected object from another using the machine learning classifier”, paragraph 18); and
initializing the software detector, based at least in part on the one object of the plurality of objects (“The detection module 102 initializes one or more tracking parameters 120 of the object tracker module 108 by learning a corresponding one or more appearance features of the target object 104 in a frame in which the target object was detected. The frame refers to an image in a sequence of video frames or a sequence of images captured in a video stream and the like”, paragraph 17, “At process 204, the object tracker module 108 is initialized with the detected (or re-detected) target object 104 and any labeled annotation parameters obtained during detection. The labeled annotation parameters correspond to the learned appearance features of the target object 104. At process 206, the object tracker module 108 is updated with the continuous learning samples database 112 with samples accumulated during previous successful tracking, including any labeled annotations of the target object found in the accumulated samples”, paragraph 29).
Regarding claim 11. Claim 11 is rejected for the same reasons and rational as provided above for claim 1.
Regarding claim 12. Claim 12 is rejected for the same reasons and rational as provided above for claim 2.
Regarding claim 13. Claim 13 is rejected for the same reasons and rational as provided above for claim 3.
Regarding claim 14. Claim 14 is rejected for the same reasons and rational as provided above for claim 4.
Regarding claim 15. Claim 15 is rejected for the same reasons and rational as provided above for claim 5.
Regarding claim 19. Claim 19 is rejected for the same reasons and rational as provided above for claim 9.
Regarding claim 20. Claim 20 is rejected for the same reasons and rational as provided above for claim 1.
Allowable Subject Matter
Claims 6-8 & 16-18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US PG Pub 2020/0160540 to Rastgar discloses an electronic device for object tracking based on a user-specified initialization point is provided. The electronic device stores a sequence of image frames, which includes a first image frame and a second image frame. The electronic device estimates a set of feature correspondences between a first set of features points in the first image frame and a second set of feature points in the second image frame. The electronic device generates different first motion-estimate models for different groups of feature correspondences of the set of feature correspondences and further estimates, from different groups of feature correspondences, a plurality of inlier feature correspondences that correspond to the object of interest in the first image frame and the second image frame. The electronic device generates a second motion-estimate model as an optimal motion-estimate model and tracks the object of interest in the sequence of frames, based on the second motion-estimate model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER D. WAIT, Esq. whose telephone number is (571)270-5976. The examiner can normally be reached Monday-Friday, 9:30- 6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at 571 270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CHRISTOPHER D. WAIT, Esq.
Primary Examiner
Art Unit 2683
/CHRISTOPHER WAIT/Primary Examiner, Art Unit 2683