DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 5, 8-10, 13, 14-16, 19 and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by US 11625838 B1 to Narapureddy et al (hereinafter ‘Narapureddy’).
Regarding claim 1, Narapureddy discloses a data processing method (column 1, lines 54-56, a three dimensional (3D) pose estimation and tracking of one or more people seen in an arbitrary number of camera feeds), comprising: obtaining a target image Fig. 2, Input images 202); processing the target image by using a first pose recognition model, to obtain first pose information of a target object in the target image (column 9, lines 21-26, column 5 and lines 61-65, Fig. 2, wherein process 400 may continue to action 412, at which first feature data may be extracted that represents the plurality of frames of image data. In various examples, the first feature data may be extracted from a deep layer of a CNN. For example, as described above, the first feature data, as first pose information, may be extracted from the penultimate layer (or any desired layer) of HRNet, and wherein HRNet may be a CNN that is used as a network 204 for extracting image-based features at each input frame of input sequences 202. HRNet may be a network that generates human pose estimation data); processing the target image by using a second pose recognition model, to obtain second pose information of the target object in the target image (column 9, lines 52-57, wherein the output of the 4D CNN 211 may be a reduced size tesseract feature. These features represent a spatio-temporal descriptor of a person centered around a detection. This bottleneck descriptor (e.g., the second feature data), as the second pose information, is used in both the person tracking network 210 and pose estimation module (FIG. 2).), wherein the first pose information and the second pose information describe a three-dimensional pose of the target object (column 9, lines 28-30 and 35-39, wherein process 400 may continue to action 414, at which the first feature data, as the first pose information, may be projected into 3D feature data in a voxel space, and wherein process 400 may continue to action 418, at which location data, as the second pose information, describing locations of the one or more persons in the voxel space, as the second 3d information regarding the location of the pose of the object, may be generated. The 3D CNN may be trained to generate 3D cuboid bounding boxes that surround the detected persons.), and wherein the second pose information determines two-dimensional projection information of a predicted pose of the target object (column 5, lines 31-34, and Fig. 4, step 420-422, wherein FIG. 1, the top two rows 130, 132 portray the projections of keypoints, inherently as projection of the 3D pose on 2D images, two different camera views, while the bottom row 134 shows the 3D pose tracking over time); and constructing a loss for updating the second pose recognition model based on the first pose information, the second pose information, the two-dimensional projection information, and a corresponding annotation (wherein the phrase “constructing a loss” is given patent weight however the phrase “loss for . . .” is treated as intended use of the constructed loss and is not given patent weight however, column 8, lines 3-33 and Fig. 2 discloses two loss functions may be combined for the pose estimation task: a L1 distance computed on the keypoints, as annotations, positions and a loss on the response of the heatmap at the ground truth joint position).
Regarding claim 2, Narapureddy discloses wherein the first pose recognition model is obtained through training based on a loss constructed based on output pose information and a corresponding annotation (column 8, lines 52-60, wherein the gradient is propagated back to the initial images, including through the HRNet backbone, as the first model, which is shared by the detection module and the tracking+pose estimation modules. As used herein, “end-to-end” refers to training each machine learning component of the system 200 to minimize the sum of the three losses (equation (4)), as opposed to individually training the components based on only the loss function related to that component.).
Regarding claim 5, Narapureddy discloses wherein the target object is a character (column 5, lines 19-21, wherein FIG. 1 is an illustration of multi-person articulated 3D pose tracking, according to various aspects of the present disclosure.).
Regarding claim 8, Narapureddy discloses wherein the annotation is a manual advance annotation, or is obtained by processing the target image by using a pre-trained model (column 8, lines 31-34, wherein two loss functions may be combined for the pose estimation task: a L1 distance computed on the keypoints, as annotations, positions and a loss on the response of the heatmap at the ground truth joint position:).
Regarding claim 9, Narapureddy discloses a training device, comprising at least one processor and a memory coupled to the at least one processor, wherein the memory stores instructions for execution by the at least one processor (column 10, lines 23-30, wherein the processing element 504 may comprise at least one processor. Any suitable processor or processors may be used. For example, the processing element 504 may comprise one or more digital signal processors (DSPs). The storage element 502 can include one or more different types of memory, data storage, or computer-readable storage media devoted to different purposes within the architecture 500.) to: Please refer to the corresponding method claim 1 above for further teachings.
Regarding claims 10, 13 and 14, please refer to the corresponding method claims 2, 5 and 8 above for further teachings.
Regarding claim 15, Narapureddy discloses a computer program product, comprising computer-readable instructions, wherein the computer-readable instructions, when executed by a computer device (column 10, lines 32-36, wherein different portions of the storage element 502, for example, may be used for program instructions for execution by the processing element 504, storage of images or other digital works, and/or a removable storage for transferring data to other devices), instruct the computer device to: Please refer to the corresponding method claim 1 above for further teachings.
Regarding claims 16, 19 and 20, please refer to the corresponding method claims 2, 5 and 8 above for further teachings.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Narapureddy.
Regarding claim 6, Narapureddy does not specifically disclose wherein the method further comprises: processing the target image by using an updated second pose recognition model, to obtain third pose information of the target object in the target image, wherein the third pose information determines a pose of the target object. However, Narapureddy discloses in column 5, lines 10-16, and wherein the parameters (e.g., weights and/or biases) of the machine learning model may be updated to minimize (or maximize) the cost. For example, the machine learning model may use a gradient descent (or ascent) algorithm to incrementally adjust the weights to cause the most rapid decrease (or increase) to the output of the loss function, as inherently generating third, fourth, etc pose information, and additionally column 8, lines 52-60, discloses wherein the gradient is propagated back to the initial images, including through the HRNet backbone which is shared by the detection module and the tracking+pose estimation modules. As used herein, “end-to-end” refers to training each machine learning component of the system 200, inherently as updating all models, to minimize the sum of the three losses (equation (4)), as opposed to individually training the components based on only the loss function related to that component). Therefore, it would have been obvious to one ordinary skill in the art to combine the updating pose recognition models so that to minimize (or maximize) the cost (column 5, lines 11-12).
Regarding claim 7, Narapureddy discloses wherein the method further comprises: sending, to user equipment, the updated second pose recognition model or the pose of the target object obtained by processing the target image by using the updated second pose recognition model (column 5, lines 31-37, wherein FIG. 1, the top two rows 130, 132 portray the projections of keypoints on two different camera views, while the bottom row 134 shows the 3D pose tracking over time. As shown, the tracking of multiple people in a natural setting (e.g., moving around a basketball court) is smooth accounting for moving cameras over a relatively long duration of time (e.g., 200 frames), inherently as displaying outcome images to a user).
Claims 4, 12 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Narapureddy in view of US 11,748,913 B2 to Ali et al (hereinafter ‘Ali’).
Regarding claims 4, 12 and 18, Narapureddy discloses wherein, and the two-dimensional projection information is represented as a location of a two-dimensional projection of the predicted pose in the original image (Column 5, lines 31-33, wherein FIG. 1, the top two rows 130, 132 portray the projections of keypoints on two different camera views, while the bottom row 134 shows the 3D pose tracking over time). However, Narapureddy does not specifically disclose the target image is an image area in which the target object is located in an original image. Ali discloses the target image is an image area in which the target object is located in an original image (column 12, lines 31-34, wherein given an input monocular image capturing a cropped object, as the target object, the neural network(s) can be trained using camera calibration data, object crop size data, crop location information). Narapureddy and Ali are combinable because they both disclose image capturing and pose processing of an object. Therefore, before the effective filing data of the claimed invention, it would have been obvious to one ordinary skill in the art to combine the target image is an image area in which the target object is located in an original image, of Ali’s method/ device/program with Narapureddy’s so that to remove other parts/portions of the image from further processing (column 12, line 65 through column 13, line 2).
Allowable Subject Matter
Claims 3, 11 and 17 are objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: the prior art or the prior art of record specifically, Narapureddy and Ali, does not disclose:
. . . . wherein the constructing a loss further comprises: constructing the loss based on the first body shape information and the second body shape information, of claims 3, 11 and 17 combined with other features and elements of the claims.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHERVIN K NAKHJAVAN whose telephone number is (571)272-5731. The examiner can normally be reached Monday-Friday 9:00-12:00 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sue Lefkowitz can be reached at (571)272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHERVIN K NAKHJAVAN/Primary Examiner, Art Unit 2672
Prosecution Insights