Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 9, 10, 13 and 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koh et al. (US PG Pub 202100235802) in view of Mehta et al. “XNet: Real-time Multi-person 3D Human Pose Estimation …” (cited by applicant) hereafter called Mehta.
Regarding claims 1 and 15, Koh et al. discloses a method and electronic device for estimating a height of a human (Koh steps 108 and 126 in figure 1A), the electronic device comprising: a processor configured to:
obtain an image including at least a part of a representation of the human and reference information (steps 104 and paragraph 98);
input the image to a first neural network (steps 181 and 182 of figure 1D) and obtain as output from the first neural network first information, the first information related to a plurality of keypoints in the body of the human (paragraph 150);
input the image to obtain second information, the second information related to the reference information (reference information in paragraph 98); and
estimate the height of the human based on the first information and the second information (figure 11 steps 1108-1112 imaged data is compared to input data to verify height information); and
an output unit configured to output the estimated height (figure 16 outputs Height information on a phone).
Although Koh fails to specifically disclose that the reference information is feed into a second neural network Mehta does (section 6.2 reference object are used in stage 2). Since both systems are directed towards the same field of endeavor it would have been obvious to one of ordinary skill in the art at the time of filing to combine the teaches of Koh and Mehta to allow for images of multi persons to have their height calculated by multiple neural networks. The more specialized networks used will allow for simpler training and faster results. Metha also discloses many of the other limitations of claims 1 and 15. See Sections 3 and 6.2 of Mehta.
As for claim 2 Koh discloses an electronic device according to claim 1, wherein the second information is information linking the reference information with physical distance information (Koh paragraph 98 reference data is used to link ground truth information figure 2).
As for claims 3 and 16 Koh discloses an electronic device according to claim 1, wherein the first neural network is configured to segment the at least part of the representation of the human into a plurality of body parts, and to predict the plurality of keypoints in the body of the human based on the plurality of body parts (step 204). Also Note Mehta Section 3 and figure 3 and 4.
As for claim 4, Koh and Mehta disclose an electronic device according to claim 3, wherein the information related to the plurality of keypoints comprises coordinate information about at least part of the plurality of keypoints. Koh uses 2D and 3D models which have coordinates for each body part as well as Mehta Section 3 and figure 3 and 4.
As for claim 5 Mehta and Koh disclose an electronic device according to a wherein a keypoint corresponds to one of a list comprising face, shoulder, hip, knee, ankle and heel. Mehta sections 2 and 3. Koh paragraph 91.
Regarding claim 6. Koh discloses an electronic device according to claim 3, wherein the first neural network is configured to identify a predefined number of keypoints, and if at least one keypoint is not identified by the first neural network with at least 50% of detection confidence and at least 50% of visibility, the processor is configured to generate a notification indicating that the height cannot be estimated, and the output unit is configured to output the notification. Koh teaches that confidence is used in figures 1A steps 114 and 124, see paragraphs 101-109.
Regarding claim 9 Koh discloses an electronic device according to claim 1 wherein the reference information includes an object of a known predetermined size, such as an object of the size of a credit card, see Koh paragraph 98.
Regarding claim 10, Koh discloses an electronic device according to claim 9, wherein the second neural network is configured to find contours of the object, recognize the object, and obtain the predetermined size of the object, and wherein the second information comprises information related to the physical size of the object. (Koh para 98 and 150)
Regarding claim 13, both Koh and Mehta disclose electronic device according to claim 1, further comprising an image capturing unit configured to capture the image. (cameras)
Claim(s) 7, 8, 12 and 17 are is/are rejected under 35 U.S.C. 103 as being unpatentable over Koh et al. in view of Mehta et al. as applied to claims above, and further in view of Fedyukov et al. (US PG Pub 2021/0049811).
As shown above the combination of Koh et al in view of Mehta et al teach the system and methods of claims above. However, the combination fails to teach the specifics of the following claims.
Regarding claim 7, Fedyukov discloses an electronic device according to claim 1, wherein the first neural network is a convolutional neural network for human pose estimation implemented with a BlazePose neural network, for which the prediction of the keypoints has been parametrized using mediapipe pose estimation application program interface, and wherein an output of the BlazePose/mediapipe pose solution application interface is passed through a Broyden, Fletcher, Goldfarb, and Shanno, BFGS, optimization algorithm. Paragraph 348
Since all three systems are trainable network used to calculate human pose and size measurements, they are considered all in the same field of endeavor. Therefore, it would have been obvious to one of ordinary skill in the art before the time of filing to combine the BFGS optimization as taught by Fedyukov into the systems of Koh and Mehta to improve pose estimation. One of ordinary skill would use other training/learning algorithms available depending on cost and accessibility.
Regarding claim 8, Fedyukov discloses an electronic device according to claim 3 wherein the processor is further configured to use the first information to compute Euclidean distances between coordinates of the at least part of the plurality of keypoints on the image to calculate a pixel length of the representation of the human in the image. See Paragraph 139.
Regarding claim 12, Fedyukov discloses an electronic device according to claim 1, wherein the second neural network is formed from a convolutional neural network U-Net with EfficientNet-b0 backbone.
Since all three systems are trainable network used to calculate human pose and size measurements, they are considered all in the same field of endeavor. Therefore, it would have been obvious to one of ordinary skill in the art before the time of filing to combine well known U-Net optimization as taught by Fedyukov into the systems of Koh and Mehta to improve pose estimation. One of ordinary skill would use other training/learning algorithms available depending on cost and accessibility.
Regarding claim 17, see paragraphs 552, 674-675 of Fedyukov for the specific of claimed servers.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US PG Pub 20230052613 uses a credit card as a known reference object in 3D pose estimation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER S KELLEY whose telephone number is (571)272-7331. The examiner can normally be reached Mon-Fri 6:30 to 4 pm alternate Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Debbie Reynolds can be reached at 571-272-0734. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER S KELLEY/Supervisory Patent Examiner, Art Unit 2482