DETAILED ACTION
Response to Arguments
Applicant's arguments filed with respect to claims 1, 3-10, and 12-19 have been fully considered but are moot in view of the new ground(s) of rejection. The rejections are necessitated due to claim amendments.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine,
manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-10, and 12-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1, and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1, 3-10, and 12-19 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101.
The Examiner will analyze Claim 1, and similar rationale applies to independent Claims 10 and 19.
The rationale, under MPEP § 2106, for this finding is explained below. The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a Process since the claim is directed to a method of image processing.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of extracting pose features corresponding to the plurality of hand images respectively; determining finger pose information based on the pose features corresponding to the plurality of hand images, respectively; determining palm pose information based on the pose features corresponding to the plurality of hand images, respectively; and determining pose information of the target hand based on the finger pose information and the palm pose information. are directed to an abstract. The claim is related to mental process by a person performing all steps based on an image.
If/when the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The additional claim limitations A method of image processing, comprising: obtaining a plurality of hand images, the plurality of hand images being images of a target hand captured from a plurality of viewing angles considering the claim as a whole, it doesn’t integrate the judicial exception into a practical application.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the claims do not amount to significantly more.
Furthermore, the generic computer components or machine learning algorithm of the processor/memory recited as performing generic computer or machine learning functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
The Examiner finds that Claims 3-9 does not state significantly more since the claim only recites additional steps for analyzing image in order to determine hand pose.
Thus, claims 1, 3-10, and 12-19 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Therefore, all claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (Pub. No. US 2021/0074016) in view of KRUPKA et al. (Pub. No. US 2017/0193334).
Regarding claim 1, Li teaches a method of image processing (three-dimensional hand pose estimation) [Para. 2], comprising:
obtaining a plurality of hand images (pair of images), the plurality of hand images (pair of images) being images of a target hand captured from a plurality of viewing angles (first view and second view) [Para. 49];
extracting pose features (feature maps) corresponding to the plurality of hand images respectively [Para. 54];
determining information based on the pose features (feature maps) corresponding to the plurality of hand images (pair images), respectively [Para. 54].
However, Li doesn’t explicitly teach about finger pose information.
KRUPKA teaches about finger pose information (finger flexion features) [Para. 53].
It would have been obvious to one of ordinary still in the art before the effective filing date to modify Li’s neural network hand pose estimation method by incorporating Krupak’s teaching of finger pose information so that Li’s feature maps and joint location estimates are used to output finger specific pose states. This medication improves Li by adding finger level pose classification to the estimated hand joints, thereby improving recognition of specific hand gestures rerestarted by individual finger states.
Li teaches determining palm pose information based on the pose features (feature maps) corresponding to the plurality of hand images (pair of images), respectively [Para. 54].
However, Li doesn’t explicitly teach about palm pose information.
KRUPKA teaches about palm pose information [Para. 52].
It would have been obvious to one of ordinary still in the art before the effective filing date to modify Li’s neural network hand pose estimation method by incorporating Krupak’s teaching of palm pose information so that Li’s feature maps and joint location estimates are used to output palm specific pose states. This medication improves Li by adding palm level pose classification to the estimated hand joints, thereby improving recognition of specific hand gestures rerestarted by individual palm states.
Li teaches determining pose information (3D hand pose) of the target hand [para. 51].
However, Li doesn’t explicitly teach determining pose information of the target hand based on the finger pose information and the palm pose information [Para. 51, 52 and 53].
It would have been obvious to one of ordinary still in the art before the effective filing date to modify Li’s 3D hand pose prediction module by incorporating Krupka’s teaching that a hand pose is identified from a combination of palm pose information and finger pose information so that Li’s final 3D hand pose is determined using both classes of hand part pose information. This modification improves Li by making the final 3D hand pose decision depend on explicitly finger and palm states, thereby improving discrimination among hand poses with similar joint locations but different palm or finger configurations.
Claims 10 and 19 are rejected for the same reasons as claim 1 above. Furthermore, Li teaches having a device and a memory to perform the claim limitations [fig. 1-3 and related description].
Claims 3-9 and 12-18 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (Pub. No. US 2021/0074016) in view of KRUPKA et al. (Pub. No. US 2017/0193334) further in view of Litvak et al. (Pub. No. US 2013/0236089).
Regarding claims 3 and 12, Li in view of KRUPKA does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining finger pose information (fingertip/joint) based on the plurality of hand images comprises: determining a fused pose feature (descriptors) based on the pose features corresponding to the plurality of hand images respectively [Para. 80]; and determining the finger pose information (hand joints) based on the fused pose features [Para. 82].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of KRUPKA to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 4 and 13, Li in view of KRUPKA does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining palm pose information based on the plurality of hand images comprises: determining key points corresponding to the plurality of hand images respectively based on the pose features corresponding to the plurality of hand images respectively [Para. 18 and 86]; and determining the palm pose information based on the key points corresponding to the plurality of hand images respectively [Para. 20].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of KRUPKA to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 5 and 14, Li in view of KRUPKA does not explicitly teach the claim limitation.
However, Litvak teaches wherein the key points corresponding to the plurality of hand images respectively are 2.5d key points, [Para. 72, “The resulting descriptor is referred to herein as a "2.5D" descriptor” and Para. 4 “two-dimensional matrix of pixels, in which each pixel corresponds to a respective location in scene and has a respective pixel depth value, indicative of the distance from a certain reference location to the respective”] and the palm pose information is 3d pose information [Para. 20 “Estimating the pose may include reconstructing the pose by applying reverse kinematics using at least one of the locations of the landmarks.”; and 109 “assuming a query patch with center location (x, y, z) was found to match a retrieved patch with center location at (X, Y, Z), and the ground-truth location of a given landmark relative to the center of the retrieved patch was at a distance”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of KRUPKA to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 6 and 15, Li in view of KRUPKA does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining finger pose information comprises: determining the finger pose information based on the plurality of hand images and a predetermined pose prediction model [Para. 83 “Descriptors of the types described above are used first in a learning phase, to build database 25, and then in a detection phase.” And 85 and 86].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of KRUPKA to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 7 and 16, Li in view of KRUPKA does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining palm pose information comprises: determining the palm pose information based on the plurality of hand images and the predetermined pose prediction model (database) [Para. 83 “Descriptors of the types described above are used first in a learning phase, to build database 25, and then in a detection phase”; Para. 20 and 86].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of KRUPKA to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 8 and 17, Li in view of KRUPKA does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining pose information of the target hand based on the finger pose information and the palm pose information comprises: determining the pose information of the target hand based on the finger pose information, the palm pose information and the predetermined pose prediction model [Para. 20, and 86].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of KRUPKA to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 9 and 18, Li in view of KRUPKA do not explicitly teach the claim limitation.
However, Litvak teaches wherein the method of image processing further comprises: obtaining a training dataset comprising a plurality of sample hand images, each sample hand image having corresponding finger pose information and palm pose information [Para. 83 “The learning phase uses a large set of training data, comprising depth maps of hands in different poses, with "ground-truth" information for each depth map”; “The ground-truth information may identify actual locations of anatomical landmarks on the hand, such as the fingertips, base of the hand (wrist position), joints, palm plane, and/or other landmarks on the hand ”; Para. 85 “the ground-truth information may be provided in terms of functional features, such as the angles of the finger joints and wrist”]; and determining the predetermined pose prediction model based on the plurality of sample hand images and an initial pose prediction model (database of patch descriptors) [Para. 90, Para. 95 “once the entire database has been processed in this manner, the computer augments the database as follows, in an augmentation step 114, for each of the patches in the database”; and “Add to the database the vector distances to the ground-truth locations of all anatomical landmarks in the full depth map from which the patch was taken”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of KRUPKA to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-8 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 888-786-0101 (IN USA OR CANADA) or 571-272-4000.
/SOLOMON G BEZUAYEHU/
Primary Examiner, Art Unit 2666
Prosecution Insights