DETAILED ACTION
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 10, and 19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter because the claim(s) as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea. The claims are directed to the abstract idea (i.e. determining finger and palm pose based on images, and determining pose information of a hand based on finger pose info and palm pose info), which is a mental process that could be done by human. Obtaining multiple hand images using camera is nothing more than data gather (extra solution activity). The additional element(s) or combination of elements (i.e. processor and memory) in the claim(s) other than the abstract idea per se amount(s) to no more than: recitation of generic computer structure that serves to perform generic computer functions that are well-understood, routine, and conventional activities previously known to the pertinent industry. Viewed as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
None of the dependent claims has a limitation that amounts to significantly more. Therefore, claims 2-9, 11-18 and 20 are also rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (Pub. No. US 2021/0074016) in view of Yao (Patent No. US 9,870,056).
Regarding claim 1, Li teaches a method of image processing, comprising: obtaining a plurality of hand images, the plurality/pair of hand images being images of a target hand captured from a plurality of viewing angles (first and second views) [Para. 49 the hand pose estimation system 122 receives a pair of images from a camera. In some examples, the camera is a stereo camera. The pair of images comprise a first view and a second view of a hand. In some examples the first view of the hand replicates a left eye view of the hand and the second view of the hand replicates a right eye view of the hand”; determining joints information based on the plurality of hand images [Para. 52 and 53]; determining joints information based on the plurality of hand images [Para. 52 and 53]; and determining pose information of the target hand based on the finger pose information and the palm pose information [Para. 51 “the hand pose estimation system 122 identifies a 3D hand pose based on the plurality of sets of joint location coordinates”].
However, Li doesn’t explicitly teach determining hand pose information based on finger pose and palm pose.
Yao teaches determining finger pose information based on the plurality of hand images [Col. 14, lines 47-55]; determining palm pose information based on the plurality of hand images [Col. 20 lines 34-36] and determining pose information of the target hand based on the finger pose information and the palm pose information [Col. 20 lines 29-39 and corresponding description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li to determine a final hand pose information based on finger and palm poses, feature as taught by Yao; because the modification predictably improve consistency of the overall pose representation without changing Li’s core image-based estimation.
Claims 10 and 19 are rejected for the same reasons as claim 1 above. Furthermore, Li teaches having a device and a memory to perform the claim limitations [fig. 1-3 and related description].
Claims 2-9 and 11-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (Pub. No. US 2021/0074016) in view of Yao (Patent No. US 9,870,056) further in view of Litvak et al. (Pub. No. US 2013/0236089).
Regarding claims 2, 11, and 20, Li in view of Yao do not explicitly teach the claim limitation.
However, Litvak teaches wherein the method of image processing further comprises: extracting pose features (descriptors) corresponding to the plurality of hand images (sequence of depth maps) respectively [Abstract “receiving a depth map of a scene containing a human hand,”, “extracting from the depth map respective descriptors”; Para. 32 “receiving the depth map includes receiving a sequence of depth maps, and estimating the pose includes tracking movement of the human hand over multiple frames in the sequence”];
the finger pose information (descriptor) and/or the palm pose information (fingertips/joint) being determined based on the pose features corresponding to the plurality of hand images respectively [Para. 13 “estimating a pose of the human hand based on stored information associated with the matched descriptors”; and Para. 20 “the landmarks include at least one element selected from a group including a fingertip, a joint, a palm plane, and a base of the hand”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 3 and 12, Li in view of Yao does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining finger pose information (fingertip/joint) based on the plurality of hand images comprises: determining a fused pose feature (descriptors) based on the pose features corresponding to the plurality of hand images respectively [Para. 80]; and determining the finger pose information (hand joints) based on the fused pose features [Para. 82].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 4 and 13, Li in view of Yao does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining palm pose information based on the plurality of hand images comprises: determining key points corresponding to the plurality of hand images respectively based on the pose features corresponding to the plurality of hand images respectively [Para. 18 and 86]; and determining the palm pose information based on the key points corresponding to the plurality of hand images respectively [Para. 20].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 5 and 14, Li in view of Yao does not explicitly teach the claim limitation.
However, Litvak teaches wherein the key points corresponding to the plurality of hand images respectively are 2.5d key points, [Para. 72, “The resulting descriptor is referred to herein as a "2.5D" descriptor” and Para. 4 “two-dimensional matrix of pixels, in which each pixel corresponds to a respective location in scene and has a respective pixel depth value, indicative of the distance from a certain reference location to the respective”] and the palm pose information is 3d pose information [Para. 20 “ Estimating the pose may include reconstructing the pose by applying reverse kinematics using at least one of the locations of the landmarks.”; and 109 “assuming a query patch with center location (x, y, z) was found to match a retrieved patch with center location at (X, Y, Z), and the ground-truth location of a given landmark relative to the center of the retrieved patch was at a distance ”] .
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 6 and 15, Li in view of Yao does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining finger pose information based on the plurality of hand images comprises: determining the finger pose information based on the plurality of hand images and a predetermined pose prediction model [Para. 83 “Descriptors of the types described above are used first in a learning phase, to build database 25, and then in a detection phase.” And 85 and 86].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 7 and 16, Li in view of Yao does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining palm pose information based on the plurality of hand images comprises: determining the palm pose information based on the plurality of hand images and the predetermined pose prediction model (database) [Para. 83 “ Descriptors of the types described above are used first in a learning phase, to build database 25, and then in a detection phase”; Para. 20 and 86].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 8 and 17, Li in view of Yao does not explicitly teach the claim limitation.
However, Litvak teaches wherein the determining pose information of the target hand based on the finger pose information and the palm pose information comprises: determining the pose information of the target hand based on the finger pose information, the palm pose information and the predetermined pose prediction model [Para. 20, and 86].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Regarding claims 9 and 18, Li in view of Yao do not explicitly teach the claim limitation.
However, Litvak teaches wherein the method of image processing further comprises: obtaining a training dataset comprising a plurality of sample hand images, each sample hand image having corresponding finger pose information and palm pose information [Para. 83 “The learning phase uses a large set of training data, comprising depth maps of hands in different poses, with "ground-truth" information for each depth map”; “The ground-truth information may identify actual locations of anatomical landmarks on the hand, such as the fingertips, base of the hand (wrist position), joints, palm plane, and/or other landmarks on the hand ”; Para. 85 “the ground-truth information may be provided in terms of functional features, such as the angles of the finger joints and wrist”]; and determining the predetermined pose prediction model based on the plurality of sample hand images and an initial pose prediction model (database of patch descriptors) [Para. 90, Para. 95 “once the entire database has been processed in this manner, the computer augments the database as follows, in an augmentation step 114, for each of the patches in the database”; and “Add to the database the vector distances to the ground-truth locations of all anatomical landmarks in the full depth map from which the patch was taken”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Li in view of Yao to teach the claim limitations, feature as taught by Litvak; because the modification improves hand pose estimation by using learned descriptor matching on depth images to more infer finger joints and palm pose despite occlusions and sensor noise.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-7 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-0101 (IN USA OR CANADA) or 571-272-1000.
/SOLOMON G BEZUAYEHU/ Primary Examiner, Art Unit 2666