Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant has canceled claims 3 and 8. Application has pending claims 1-2, 4-7, and 9-11.
Response to Arguments
Applicant’s arguments, see Remarks, filed 01/12/2026, with respect to the rejection(s) of claim(s) 1 under 35 U.S.C. 102(a)(1) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of ZHILAN, as a single reference 103, as fully disclosed within the rejection below.
Applicant additionally argues, on page 9 in the Remarks, that ZHILAN merely compares a best possible match, not a determination based on calculation. Examiner respectfully disagrees. ZHILAN discloses using a pose estimation which is used alongside a probability distribution in order to calculate the pose (see ZHILAN page 5-6/41, wherein the pose determining unit calculates a pose estimation). In addition, the “best fit” that ZHILAN calculates is based on the body constraint, which fits the limitation of a human keypoint to fit the at least one constraint of claim 1 (see ZHILAN page 5/41, wherein the pose assessment module calculates a pose estimation using the probability distribution and the body constraint feature). This pose estimation uses a length and depth estimation to ensure consistency of the limb for the pose (see ZHILAN page 5/41, wherein the body part constraint feature for the 2D and 3D lengths and depths for the limbs is disclosed). Based on a broadest reasonable interpretation, one of ordinary skill in the art would understand that the pose determination is performed by a calculation to determine the human body pose.
Thus, this rejection is made FINAL.
Claim Objections
Claim 1 objected to because of the following informalities: “a value based on a function that maximises a likelihood…”. The word “maximises” should be spelled “maximizes”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over HU ZHILAN KR-20140114741-A, hereinafter ZHILAN.
As per claim 1, ZHILAN discloses a method of pose estimation, the method comprising:receiving at least one image frame, wherein the at least one image frame comprises at least one subject (see ZHILAN page 2/41, wherein an image of a human body is received);determining one or more candidate positions for each of a plurality of human keypoints (see ZHILAN top of page 3/41, wherein a candidate body part according to the skeletal points, i.e., human keypoints, is detected from the body part feature detector 220), wherein each candidate position among the one or more candidate positions is associated with a likelihood that a human keypoint among the plurality of human keypoints is located at the candidate position (see ZHILAN top of page 3/41 and FIG. 5A, wherein a predefined threshold, or likelihood, is used to generate a map of each human joint);generating one or more combinations of human keypoints from among the plurality of human keypoints based on the one or more candidate positions (see ZHILAN bottom of page 4/41, wherein the body part estimating unit 230 fuses the result from the body part feature detector to assume a body part); anddetermining a pose of each subject among the at least one subject based on the one or more combinations of human keypoints (see ZHILAN bottom of page 4/41, wherein the pose determining unit 240 calculates the pose estimation using body part estimation from the body part estimating unit, which contains the skeleton comprised of skeletal points),wherein determining the pose comprises determining the pose based on at least one constraint affecting the pose of each subject among the at least one subject (see ZHILAN top of page 5/41, wherein the body part constraint in the pose estimation is disclosed), and wherein the at least one constraint preferably comprises at least one of: limb length, limb angle, or limb movement (see ZHILAN top of page 5/41, wherein the distance and angle of body parts, i.e., limb length and angle, are disclosed), anddetermining the pose of each subject among the at least one subject comprises calculating, for each combination of human keypoints among the one or more combinations of human keypoints, a value based on a function that utilizes a likelihood that the human keypoint occurs and a fit to the at least one constraint (see ZHILAN page 5/41, wherein the pose assessment module calculates a pose estimation using the probability distribution and the body constraint feature), wherein the pose of the subject is the combination with the highest calculated value (see ZHILAN page 5/41, wherein the pose is determined from the pose estimation with the highest probability value).
While ZHILAN does disclose estimating a pose using probability distribution and a body constraint feature, it does not fully disclose calculating a value based on a function that maximizes a likelihood that the human keypoint occurs, but it would have been obvious to do so. The reason is because ZHILAN uses skeletal points of the human body, the keypoints (see ZHILAN page 4/41 for the skeletal points of the body is extracted), in order to estimate the pose of the human body. This pose estimation uses a length and depth estimation to ensure consistency of the limb for the pose (see ZHILAN page 5/41, wherein the body part constraint feature for the limbs is disclosed). Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention by the applicant to utilize the pose estimate corresponding to a highest probability value, using the estimations of the length and depth, of the pose assessment in order to calculate the likelihood of a body part.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over ZHILAN, in further view of STEPHEN BAEK US-20200327465-A1, hereinafter BAEK.
As per claim 2, ZHILAN fails to explicitly disclose where BAEK teaches:The method of claim 1, wherein determining the one or more candidate positions comprises:generating a heatmap for each human keypoint among the plurality of human keypoints, wherein the heatmap represents a likelihood that a human keypoint among the plurality of human keypoints occurs at a pixel location (see BAEK ¶78, wherein a heat map showing the likelihood of body joints is produced for the key point detection);identifying one or more peaks in the heatmap (see BAEK ¶78, wherein the ground truth heat map is generated using the Gaussian peaks); anddetermining coordinates of each peak among the one or more peaks, wherein the coordinates represent a candidate position of the human keypoint (see BAEK ¶78-79, wherein key point location is determined using
M
i
p
=
e
x
p
-
|
p
-
k
i
|
σ
2
, wherein p is the pixel point and the ground truth location of the key point i).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHILAN’s method by using BAEK’s teaching by including a heatmap of the human keypoints to the candidate positions in order to further determine an accurate probability of the human joint locations.
Claims 4-5 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over ZHILAN, in combination with BAEK, in further view of Samuel MENAKER US 20220379170 A1, hereinafter MENAKER.
As per claim 4, ZHILAN, in combination with BAEK, fails to explicitly disclose where MENAKER teaches:The method of claim 1, wherein the at least one constraint comprises limb movement, and wherein the limb movement is based on a maximum movement of each limb between image frames of the at least one image frame (see MENAKER ¶179, wherein the tracker data, which includes 3D biomechanical movements, stores a maximum and minimum value [of the movement]. Motion trackers, which acquire the tracker data, observe a subject across frames disclosed in ¶44-46).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHILAN’s, in combination with BAEK, method by using MENAKER’s teaching by including a maximum limb movement to the constraint in order to further consider the physical capabilities of human movement.
As per claim 5, ZHILAN, in combination with BAEK and MENAKER, discloses the method of claim 4, wherein the at least one constraint comprises at least one generic constraint, and wherein the generic constraint is based on a dataset comprising a general or specific population (see MENAKER ¶97, wherein the measurements of the motion trackers are compared against experts or users closer to a target score, i.e., specific population).
As per claim 9, ZHILAN, in combination with BAEK and MENAKER, discloses the method of claim 1, wherein determining the pose of each subject among the at least one subject is based on one or more vector fields encoding a location and orientation of limbs (see BAEK ¶81-83, wherein a vector field associated with the body key points is disclosed. Further, ¶85-86 discloses vector tracking of the skeletal orientation for model-based pose tracking, which is then reconstructed as disclosed in ¶89).
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over ZHILAN, in combination with BAEK and MENAKER, in further view of Cristian Sminchisescu US-20190371080-A1, hereinafter Cristian.
As per claim 6, ZHILAN, in combination with BAEK, fails to explicitly disclose where Cristian teaches:The method of claim 5, wherein the at least one constraint comprises at least one personal constraint, wherein the at least one personal constraint is unique to each subject among the at least one subject (see Cristian ¶158-161 and FIG. 18, wherein each person model is comprised of fitted spheres inside a superquadric. These superquadric boundaries define the constraint, wherein the fitted square belongs to either the first or second person), and wherein the at least one personal constraint is based on a plurality of poses for each subject determined over a period of time (see Cristian ¶160-161 and FIG. 18, wherein the pose for each subject and their collision constraint).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHILAN’s, in combination with BAEK and MENAKER, method by using Cristian’s teaching by a personal constraint to the constraint in order to avoid merging the poses of multiple people within a set of frames.
As per claim 7, ZHILAN, in combination with BAEK, MENAKER, and Cristian, discloses the method of claim 6, wherein determining the pose comprises:selecting, from the one or more combinations of human keypoints, one or more combinations of human keypoints that fit the at least one constraint (see ZHILAN bottom of page 5/41 and top of page 6/41 step 330, wherein body part estimation is performed by using, or selecting, the skeleton parts of the relevant region. Additionally, an evaluation criteria for the pose includes parameters for that relevant key point region, which includes the direction, length, and angles of the body part(s), i.e., the constraints); andfor each human keypoint, selecting a candidate position with the highest likelihood that the human keypoint is located (see ZHILAN page 3/41, wherein a predefined threshold is used in order to determine each human joint in the candidate body part by the body part feature detection unit), wherein the candidate position is selected from the selected one or more combinations of human keypoints that fit the at least one constraint (see ZHILAN page 5/41, wherein the pose assessment module uses a probability distribution and selects a pose corresponding to a highest probability distribution by utilizing at least one body part constraint feature).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over ZHILAN, in combination with BAEK and MENAKER, in further view of Zhe Cao OpenPose: Realtime Multi-Person 2D Pose Estimation, hereinafter Cao.
As per claim 10, ZHILAN, in combination with BAEK and MENAKER, fails to explicitly disclose where Cao teaches:correcting the value based on one or more vector fields encoding a location and orientation of limbs (see Cao page 6/15 section 3.5 and FIG. 7, wherein redundant PAF connections is used to improve, or correct, the pose estimation in crowded images. See FIG. 1, wherein the location and orientation of limbs is disclosed by the PAF), wherein the pose of the subject is the combination with the highest corrected calculated value (see Cao page 6/15 section 3.5, wherein the algorithm sorts all pose connections using the redundant PAF to avoid wrong connections by using the higher score).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHILAN’s, in combination with BAEK and MENAKER, method by using Cao’s teaching by including the vector field correction to the value in order to more specifically determine the pose of individuals in crowded or noisy images.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over ZHILAN, in combination with BAEK, MENAKER, and Cao, in further view of BERNARDINO EDUARDO MENDEZ IV WO-2022174075-A1, hereinafter MENDEZ.
As per claim 11, ZHILAN, in combination with BAEK, MENAKER, and Cao, fails to explicitly disclose where MENDEZ teaches:The method of claim 10, further comprising generating an alert based on the pose of each subject among the at least one subject (see MENDEZ ¶18, wherein an alert is generated from object detection, which includes person(s) detection and pose estimation).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHILAN’s, in combination with BAEK, MENAKER, and Cao, method by using MENDEZ’s teaching by including an alert to the pose in order to bring attention to the user if any errors or warnings are detected.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bradley Obas Felix whose telephone number is (703)756-1314. The examiner can normally be reached M-F 8-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 5712728243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRADLEY O FELIX/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights