Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed March 2, 2026 with respect to claims 1-20 have been considered but are moot in view of new grounds of rejection.
Response to Amendment
The Amendment of March 2, 2026 overcomes the following objection:
Objection of claim 16 because of informalities.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-8 and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Lee (US Patent Publication No.: US 2023/0351610 A1), hereinafter Lee, in view of Zheng (PCT Patent Pub. No.: WO 2020001302 A1), hereinafter Zheng, further in view of Mandal (ByteTrack: Multi-Object Tracking by Associating Every Detection Box, Qure.ai Tech Blog, August 8, 2022, https://blog.qure.ai/notes/byte-track-multi-object-tracking), hereinafter Mandal, further in view of Xiao (Simple Baselines for Human Pose Estimation and Tracking, ECCV 2018), hereinafter Xiao.
Regarding claim 1, Lee teaches a system for detecting people, comprising: an electronic sensor, wherein the electronic sensor captures a video (The camera 1130 may capture a photo and/or a video. [0106]) of people around a vehicle (For example, the electronic device 1100 may be implemented as at least a part of a mobile device such as a mobile phone, a smart phone, a PDA, a netbook, a tablet computer or a laptop computer, a wearable device such as a smart watch, a smart band or smart glasses, a computing device such as a desktop or a server, a home appliance such as a television, a smart television or a refrigerator, a security device such as a door lock, or a vehicle such as an autonomous vehicle or a smart vehicle. [0104]) to generate a plurality of consecutive frames (In another general aspect, an electronic device includes: a camera configured to generate an input image comprising a plurality of image frames. [0024]), and wherein the plurality of consecutive frames comprises an original frame (In response to a target object being determined in a first image frame, a target box 202 corresponding to the target object may be designated (e.g., generated or determined). [0055]) and one or more new frames (Referring to FIG. 3, a plurality of image frames of a search image 300 may be divided into a plurality of sequence groups including a first sequence group 301 and a second sequence group 302. [0058]
PNG
media_image1.png
814
690
media_image1.png
Greyscale
); a person detector and tracker (
PNG
media_image2.png
572
822
media_image2.png
Greyscale
), comprising a boundary box generator (The first tracking result may include first bounding boxes of the first image frames according to the forward object tracking performed on the first image frames. [0008]) that detects an enclosed boundary box within the original frame (In response to a target object being determined in a first image frame, a target box 202 corresponding to the target object may be designated (e.g., generated or determined). [0055]) and one or more new frames (The object tracking apparatus may determine a target box 204 corresponding to the target object in the search region 203 (i.e., one or more new frames) through the regression analysis and generate a tracking result based on box information of the target box 204. [0057]), and a confidence pre-processing procedure for predicting locations of each person identified (determining a confidence of the template candidate using a result of comparing a first tracking result determined by the forward object tracking performed on the first image frames and a second tracking result determined by the backward object tracking performed on the first image frames. [0005]) using a mathematical model (The object tracking apparatus 100 may generate the tracking result 103 using an object tracking model 110. The object tracking model 110 may be or include an artificial intelligence model trained or learned based on machine learning. [0046]); wherein the confidence pre-processing procedure includes a tracker match comparator (The object tracking apparatus may calculate (e.g., determine) a similarity by comparing the template feature map 211 and the search feature map 212. [0056]) which predicts at least one tracker (Referring to FIG. 1, an object tracking apparatus 100 may output a tracking result 103 based on a template image 101 and a search image 102. [0044]
PNG
media_image3.png
768
612
media_image3.png
Greyscale
) based on the original frame and one or more new frames (For example, the template image 101 (i.e., the original frame) may correspond to one of a plurality of image frames included in an input video file, and the search image 102 (i.e., one or more new frames) may correspond to one or more of the image frames after ( e.g., temporally subsequent to) the image frame corresponding to the template image 101. [0045]), compares the boundary box and the at least one tracker between each frame (The object tracking apparatus may determine a target box 204 corresponding to the target object in the search region 203 through the regression analysis and generate a tracking result based on box information of the target box 204. [0057]), and identifies a match between the boundary box and the at least one tracker in the original and each of the one or more new frames (In response to a target object being determined in a first image frame, a target box 202 corresponding to the target object may be designated (e.g., generated or determined). [0055]. The object tracking apparatus may determine a target box 204 corresponding to the target object in the search region 203 through the regression analysis and generate a tracking result based on box information of the target box 204. [0057]
PNG
media_image4.png
542
862
media_image4.png
Greyscale
); a tracker optimizer for optimizing the at least one tracker into an updated tracker (For example, a difference between an appearance of a target object appearing in first image frames in the first sequence group 301 and an appearance of a target object appearing in second image frames of the second sequence group 302 may occur, and a first template 311 for object tracking in the first sequence group 301 may be updated to a second template 321 for object tracking in the second sequence group 302. [0059]) based on detection by the tracker match comparator (For example, the object tracking apparatus may determine the confidence of the template candidate 412 based on a degree of overlap between at least some of corresponding pairs obtained from the first tracking result F, and the second tracking result B. [0067]).
Lee does not teach the following limitations as further recited, but Zheng further teaches wherein the tracker optimizer validates a matched tracker after persistence for at least a first threshold number (M) of consecutive frames (Optionally, the analyzing the validity of the identification number based on data in the consecutive M frames of images includes: Acquiring data of the identification number in consecutive M frames of images, and determining whether M is within a specified numerical range; wherein the data of the identification number in consecutive M frames of images includes: the identification number of consecutive M frames The coordinates and / or timestamp of the central pixel of the portrait in each frame of the image; If M is greater than a preset minimum value Mmin and less than a preset maximum value Mmax, the identification number is determined to be a valid number, and it is retained in the number statistics list. Page 3 4th paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Lee to incorporate the teachings of Zheng for the tracker optimizer to validate a matched tracker after persistence for at least a first threshold number (M) of consecutive frames in order to improve tracking accuracy by counting a valid tracker.
The combination of Lee and Zheng does not teach the following limitations as further recited, but Mandal further teaches retains an unmatched tracker (For the unmatched tracks T_re-remain after the second association, we put them into T_lost. Page 8 2nd paragraph) for up to a second threshold number (K) of consecutive frames before discarding the unmatched tracker (For each track in T_lost , only when it exists for more than a certain number of frames (in paper, this value is 30 frames), we delete it from the tracks T. Page 8 2nd paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Mandal to retain an unmatched tracker for up to a threshold number of consecutive frames before discarding the unmatched tracker in order to avoid nonnegligible true object missing and fragmented trajectories.
The combination of Lee, Zheng and Mandal does not teach the following limitations as further recited, but Xiao further teaches a pose estimator, comprising a backbone network (Our pose estimation is based on a few deconvolutional layers added on a backbone network, ResNet in this work. Page 2 2nd paragraph) for constructing a feature map (Our pose estimation is based on a few deconvolutional layers added on a backbone network, ResNet in this work. It is probably the simplest way to estimate heat maps from deep and low resolution feature maps. Page 2 2nd paragraph) of pose related information (Fig. 2 (d) Pose to be tracked
PNG
media_image5.png
695
1018
media_image5.png
Greyscale
) from the updated tracker (Fig. 2 (e) Shifted tracked pose) and the boundary box (For the processing frame in videos, the boxes from a human detector and boxes generated by propagating joints from previous frames using optical flow are unified using a bounding box Non-Maximum Suppression (NMS) operation. Page 6 last paragraph) using deep learning approaches (Our pose estimation is based on a few deconvolutional layers added on a backbone network, ResNet in this work. Page 2 2nd paragraph), and a keypoint encode and decoder for encoding at least one keypoint location into a 2-D representation (The annotations include 15 body keypoints location, a unique person id and a head bounding box for each person instance. Page 10 3rd paragraph) and decoding maximum values of the 2-D representation using statistical methods (The targeted heatmap
PNG
media_image6.png
38
44
media_image6.png
Greyscale
for joint k is generated by applying a 2D gaussian (i.e., statistical methods) centered (i.e., maximum values) on the kth joint’s ground truth location. Page 2 7th paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Xiao to utilize a pose estimator in order to perform simultaneous pose detection and tracking.
Regarding claim 16, Lee in the combination teaches a method for detecting people, comprising: capturing a video of the one or more people (
PNG
media_image2.png
572
822
media_image2.png
Greyscale
) to generate a plurality of consecutive frames (In another general aspect, an electronic device includes: a camera configured to generate an input image comprising a plurality of image frames. [0024]), wherein the consecutive frames include an original frame (In response to a target object being determined in a first image frame, a target box 202 corresponding to the target object may be designated (e.g., generated or determined). [0055]) and one or more new frames (Referring to FIG. 3, a plurality of image frames of a search image 300 may be divided into a plurality of sequence groups including a first sequence group 301 and a second sequence group 302. [0058]
PNG
media_image1.png
814
690
media_image1.png
Greyscale
); generating a boundary box around each of the one or more detected people within the original frame and one or more new frames (
PNG
media_image2.png
572
822
media_image2.png
Greyscale
); performing a confidence pre-processing procedure by predicting at least one tracker (Referring to FIG. 1, an object tracking apparatus 100 may output a tracking result 103 based on a template image 101 and a search image 102. [0044]
PNG
media_image3.png
768
612
media_image3.png
Greyscale
) based on the original frame and one or more new frames (For example, the template image 101 (i.e., the original frame) may correspond to one of a plurality of image frames included in an input video file, and the search image 102 (i.e., one or more new frames) may correspond to one or more of the image frames after ( e.g., temporally subsequent to) the image frame corresponding to the template image 101. [0045]), comparing the boundary box and the at least one tracker between each frame (The object tracking apparatus may determine a target box 204 corresponding to the target object in the search region 203 through the regression analysis and generate a tracking result based on box information of the target box 204. [0057]), and identifying a match between the boundary box and the at least one tracker in the original and each of the one or more new frames (In response to a target object being determined in a first image frame, a target box 202 corresponding to the target object may be designated (e.g., generated or determined). [0055]. The object tracking apparatus may determine a target box 204 corresponding to the target object in the search region 203 through the regression analysis and generate a tracking result based on box information of the target box 204. [0057]
PNG
media_image4.png
542
862
media_image4.png
Greyscale
).
Zheng in the combination further teaches validating a matched tracker after persistence for at least a first threshold number (M) of consecutive frames (Optionally, the analyzing the validity of the identification number based on data in the consecutive M frames of images includes: Acquiring data of the identification number in consecutive M frames of images, and determining whether M is within a specified numerical range; wherein the data of the identification number in consecutive M frames of images includes: the identification number of consecutive M frames The coordinates and / or timestamp of the central pixel of the portrait in each frame of the image; If M is greater than a preset minimum value Mmin and less than a preset maximum value Mmax, the identification number is determined to be a valid number, and it is retained in the number statistics list. Page 3 4th paragraph).
Mandal in the combination further teaches retaining an unmatched tracker (For the unmatched tracks T_re-remain after the second association, we put them into T_lost. Page 8 2nd paragraph) for up to a second threshold number (K) of consecutive frames prior to discarding the unmatched tracker (For each track in T_lost , only when it exists for more than a certain number of frames (in paper, this value is 30 frames), we delete it from the tracks T. Page 8 2nd paragraph).
Xiao in the combination further teaches performing a pose estimation (Simple Baselines for Human Pose Estimation and Tracking. Title) on the boundary boxes based on the confidence pre-processing (Similar as in [11], we first drop low-confidence detections, which tends to decrease the mAP metric but increase the MOTA tracking metric. Also, since the tracking metric MOT penalizes false positives equally regardless of the scores, we drop low confidence joints first to generate the result as in [11]. We choose the boxes and joints drop threshold in a data-driven manner on validation set, 0.5 and 0.4 respectively. Page 11 4th paragraph)[;].
Claims 3-8 and 15, unamended and are rejected based on the revised combination of Lee, in view of Zheng and further in view of Mandal as applied to claim 1 above, further in view of Xiao. The grounds of rejection established in the last Office Action is fully incorporated herein.
Method claim 17 is drawn to the method of using the corresponding apparatus claimed in claims 4 and 5. Therefore method claim 17 corresponds to apparatus claims 4 and 5 and is rejected for the same reasons of obviousness as used above.
Method claim 18 is drawn to the method of using the corresponding apparatus claimed in claim 6. Therefore method claim 17 corresponds to apparatus claim 6 and is rejected for the same reasons of obviousness as used above.
Method claim 19 is drawn to the method of using the corresponding apparatus claimed in claims 7 and 8. Therefore method claim 17 corresponds to apparatus claims 7 and 8 and is rejected for the same reasons of obviousness as used above.
Claims 2 and 20, unamended and are rejected based on the revised combination of Lee (US Patent Publication No.: US 2023/0351610 A1), hereinafter Lee, in view of Zheng and further in view of Mandal as applied to claim 1 above, further in view of Xiao (Simple Baselines for Human Pose Estimation and Tracking, ECCV 2018), hereinafter Xiao, and further in view of Jun (Monocular Human Depth Estimation via Pose Estimation, IEEE Access (Volume: 9), date of publication November 8, 2021), hereinafter Jun. The ground of rejection established in the last Office Action is fully incorporated herein.
Claims 9-11, unamended and are rejected based on the revised combination of Lee (US Patent Publication No.: US 2023/0351610 A1), hereinafter Lee, in view of Zheng and further in view of Mandal as applied to claim 1 above, further in view of Xiao (Simple Baselines for Human Pose Estimation and Tracking, ECCV 2018), hereinafter Xiao, and further in view of Luffca (SORT: Simple Online and Realtime Tracking, https://www.luffca.com/2023/04/multiple-object-tracking-sort/, 2023-04-08), hereinafter Luffca. The ground of rejection established in the last Office Action is fully incorporated herein.
Claims 12-14, unamended and are rejected based on the revised combination of Lee (US Patent Publication No.: US 2023/0351610 A1), hereinafter Lee, in view of Zheng and further in view of Mandal as applied to claim 1 above, further in view of Xiao (Simple Baselines for Human Pose Estimation and Tracking, ECCV 2018), hereinafter Xiao, and further in view of Sun (Deep High-Resolution Representation Learning for Human Pose Estimation, Proceedings of the IEEE/CVF conference, 2019), hereinafter Sun. The ground of rejection established in the last Office Action is fully incorporated herein.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights