DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application is a continuation of PCT PCT/IB2021/056150. Priority to Japan with a priority date of 07/08/2021 is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 01/02/2024, 05/15/2025, and 10/31/2025 have been considered and placed in the application file.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
• “a pre-processing engine” in claim 1, 10, and 19. The corresponding structure disclosed in the specification is “ The pre-processing engine 65 is to identify keypoints from the raw data to generate a keypoint map 105 from the image 100 as shown in Figure 3. In particular, the pre-processing engine 65 is to identify the keypoints that are visible in the raw data image 100 to generate keypoint coordinates in two-dimensions on the raw data image 100. The manner by which the keypoints are identified is not particularly limited. For example, the keypoints may be extracted from the two-dimensional image based on matching a set of predefined keypoint definitions from a known structure of the object.” (¶0022) and “the pre-processing engine 65 may generate a keypoint heatmap for each keypoint. The keypoint heatmap generated by the pre-processing engine 65 is to generally provide representation of the position of a point on the object. In the present example, the keypoint heatmap is a two-dimensional map.” (¶0023).
• “keypoint analysis engine” in claim 1, 10, and 19. The corresponding structure disclosed in the specification is “The keypoint analysis engine 70 is to identify a connector between two keypoints identified by the pre-processing engine 65. The manner by which the keypoint connectors are identified is not particularly limited. For example, the keypoint analysis engine 70 may use the definitions for each of the keypoints identified by the pre-processing engine 65 to assign a connector between two keypoints. The definitions for each keypoint may include information about neighboring keypoints. As a specific example, each keypoint may represent a joint in a human hand. Accordingly, the definition of a joint may include its relative position in the hand and the neighboring joint or joints connected by a bone. In particular, the keypoint 106 may be associated with a distal interphalangeal joint in the hand and defined to be connected to a metacarpophalangeal joint and a distal interphalangeal joint. The keypoint analysis engine 70 will then connect the keypoint 106 to the keypoint 107 associated with the metacarpophalangeal joint and the keypoint 108 associated with the distal interphalangeal joint based. This process may be iterated until all connections in the keypoint definitions have been made based on the keypoint heatmaps generated by the pre-processing engine 65. In other examples, the keypoint analysis engine 70 may use a known structure to identify the keypoint connectors identified by the pre processing engine 65 instead of making a determination at each keypoint.” (¶0030) and “the keypoint analysis engine 70 is to identify keypoint connectors that are completely visible in the image 100. A keypoint connector is deemed visible when the end keypoints of the keypoint connector are both visible. In some examples, the invisible keypoints may be used to estimate a keypoint connector between a visible keypoint and an invisible keypoint.” (¶0031).
• “pose estimation engine” in claim 1, 10, and 19. The corresponding structure disclosed in the specification is “ The pose estimation engine 75 is to generate a vector for each keypoint connector to represent the keypoint connector. The vector includes a direction and is to be normalized to a magnitude, such as one. Each vector is to start from the same reference point, for example, by subtracting its three-dimensional position. Accordingly, the pose estimation engine 75 to generates a three-dimensional representation 110 of a pose with a plurality of vectors as shown in Figure 4. The vectors generated by the pose estimation engine will be a scale-agnostic representation from which the object may be manipulated, such as via animation of the different connectors. The visual representation may be reconstructed using the vectors and any scaling for the object.” (¶0032) and “The pose estimation engine 75 may then scale the keypoint connectors in accordance with a set of bone lengths defined in a template of the object. When employing these realistic scales, the resulting pose will be visually pleasing.” (¶0034).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8, 10-17, and 19 are rejected under 35 U.S.C. 103 as unpatentable over Iqbal et al. (US Patent Pub US 20210117661 A1 hereafter referred to as Iqbal) in view of Litvak et al (US Patent Pub US 20150156716 A1 hereafter referred to as Litvak).
Regarding Claim 1, Iqbal teaches an apparatus (Iqbal ¶0006 discloses a system) comprising:
a communications interface to receive (Iqbal Fig 1D, 155 and ¶0074 disclose an I/O unit configured to transmit and receive communication and receive locations of the keypoints) raw data from an external source (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera), wherein the raw data (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera) includes a representation of an object (Iqbal Fig 1C, ¶0003, ¶0005 discloses the input image representing an object, in this case a hand);
a memory storage unit to store (Iqbal ¶0094 discloses storing data in a memory partition unit) the raw data (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera);
to identify a plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) from the raw data (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera);
a keypoint analysis engine (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) to identify a connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) between a first visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) and a second visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses the palm as the second keypoint) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand); and
a pose estimation engine (Iqbal Fig 2D, 200 discloses an keypoint estimation system, ¶0002 disclose the purpose is to estimate the 3D pose) to represent the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints), has a normalized magnitude (Iqbal Fig 1A 120, and ¶0006, ¶0026 discloses a scale unit which uses a calculated scale factor to normalize the depth value).
Iqbal does not explicitly disclose a pre-processing engine, to generate a vector, wherein the vector.
Litvak is in the same field of hand pose estimation. Further, Litvak teaches a pre-processing engine (Litvak ¶0083-¶0084 discloses a preprocessing step completed on the image) to generate a vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand) wherein the vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Iqbal the step of preprocessing the image and generating a vector to represent length or distance, as well as the rotational point as taught by Litvak to make the invention that can automatically detect the pose of a hand based with increased efficiency and accuracy by making the incoming image increasingly clear and uniform; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to address constraints on pose estimation such as environmental conditions such as sunlight, occlusions, and complexity of non-rigid hand poses present challenges to landmark detection and determination (Litvak ¶0004).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Iqbal in view of Litvak teaches the apparatus of claim 1, wherein the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) is a plurality of joints (Iqbal ¶0003 discloses the fixed set of key points in spaces being joints), wherein each joint (Iqbal ¶0003 discloses the fixed set of key points in spaces being joints) of the plurality of joints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) represents a rotation point of the object (Litvak ¶0106 and ¶0067 discloses that the rotation joint is the wrist point so that the hand base is pointing down and the fingers pointing up so that the position of the hand is normalized). See Claim 1 for rationale (its parent claim).
Regarding Claim 3, Iqbal in view of Litvak teaches the apparatus of claim 1, wherein the keypoint analysis engine (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) uses a keypoint definition (Iqbal ¶0037 discloses defining the key points as the first joint of the index finger and the root) to identify the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints). See Claim 1 for rationale (its parent claim).
Regarding Claim 4, Iqbal in view of Litvak teaches the apparatus of claim 1, wherein the keypoint analysis engine (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) uses a known structure to identify (Iqbal ¶002 discloses lines connecting the keypoints or vertices that represent structural components of an object's skeleton, such as the bones in a hand) the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints). See Claim 1 for rationale (its parent claim).
Regarding Claim 5, Iqbal in view of Litvak teaches the apparatus of claim 1, wherein the keypoint analysis engine (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) identifies a plurality of connectors associated with the object (Iqbal Fig 1B discloses a plurality of lines connecting a plurality of key points) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand). See Claim 1 for rationale (its parent claim).
Regarding Claim 6, Iqbal in view of Litvak teaches the apparatus of claim 1, wherein the pre-processing engine (Litvak ¶0083-¶0084 discloses a preprocessing step completed on the image) estimates a position of an invisible keypoint (Iqbal ¶0066-¶0068 discloses finding the position of the kth key point through estimation calculation). See Claim 1 for rationale (its parent claim).
Regarding Claim 7, Iqbal in view of Litvak teaches the apparatus of claim 6, wherein the keypoint analysis engine (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) estimates a connection between the invisible keypoint (Iqbal ¶0027 discloses estimating the length of the skeletal model without the ground truth of the root joint) and a visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) selected from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand). See Claim 1 for rationale (its parent claim).
Regarding Claim 8, Iqbal in view of Litvak teaches the apparatus of claim 7, wherein the keypoint analysis engine (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) estimates the connection between the invisible keypoint (Iqbal ¶0027 discloses estimating the length of the skeletal model without the ground truth of the root joint) and a visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) based on a kinematic chain (Iqbal ¶0038 discloses the kinematic structure of the hand and using the prior component length distribution to determine the component position). See Claim 1 for rationale (its parent claim).
Regarding Claim 10, Iqbal teaches a method (Iqbal ¶0006 discloses a method) comprising:
receiving raw data from an external source (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera) via a communications interface (Iqbal Fig 1D, 155 and ¶0074 disclose an I/O unit configured to transmit and receive communication and receive locations of the keypoints), wherein the raw data includes a representation of a hand (Iqbal Fig 1C, ¶0003, ¶0005 discloses the input image representing an object, in this case a hand);
storing the raw data (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera) in a memory storage unit(Iqbal ¶0094 discloses storing data in a memory partition unit);
identifying a plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) from the raw data (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera);
identifying a connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) between a first visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) and a second visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses the palm as the second keypoint) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand); and
to represent the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) of the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) with a pose estimation engine (Iqbal Fig 2D, 200 discloses an keypoint estimation system, ¶0002 disclose the purpose is to estimate the 3D pose), has a normalized magnitude (Iqbal Fig 1A 120, and ¶0006, ¶0026 discloses a scale unit which uses a calculated scale factor to normalize the depth value).
Iqbal does not explicitly disclose with a pre-processing engine, generating a vector, a direction, wherein the vector.
Litvak is in the same field of hand pose estimation. Further, Litvak teaches with a pre-processing engine (Litvak ¶0083-¶0084 discloses a preprocessing step completed on the image) generating a vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand) a direction (Litvak ¶0072, ¶0078, ¶0079 discloses the direction of vertical and horizontal edges), wherein the vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Iqbal the step of preprocessing the image and generating a vector to represent length or distance, as well as the rotational point as taught by Litvak to make the invention that can automatically detect the pose of a hand based with increased efficiency and accuracy by making the incoming image increasingly clear and uniform; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to address constraints on pose estimation such as environmental conditions such as sunlight, occlusions, and complexity of non-rigid hand poses present challenges to landmark detection and determination (Litvak ¶0004).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 11, Iqbal in view of Litvak teaches the method of claim 10, wherein the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) is a plurality of joints (Iqbal ¶0003 discloses the fixed set of key points in spaces being joints), wherein each joint (Iqbal ¶0003 discloses the fixed set of key points in spaces being joints) of the plurality of joints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) represents a rotation point of the hand (Litvak ¶0106 and ¶0067 discloses that the rotation joint is the wrist point so that the hand base is pointing down and the fingers pointing up so that the position of the hand is normalized). See Claim 10 for rationale (its parent claim).
Regarding Claim 12, Iqbal in view of Litvak teaches the method of claim 10, wherein identifying (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) comprises using a keypoint definition (Iqbal ¶0037 discloses defining the key points as the first joint of the index finger and the root). See Claim 10 for rationale (its parent claim).
Regarding Claim 13, Iqbal in view of Litvak teaches the method of claim 10, wherein identifying (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) comprises is based on a known structure of the hand (Iqbal ¶002 discloses lines connecting the keypoints or vertices that represent structural components of an object's skeleton, such as the bones in a hand). See Claim 10 for rationale (its parent claim).
Regarding Claim 14, Iqbal in view of Litvak teaches the method of claim 10, further comprising identifying (Iqbal ¶0126- ¶0129 discloses a trained DNN to identify and classify objects) a plurality of connectors associated with the hand (Iqbal Fig 1B discloses a plurality of lines connecting a plurality of key points) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand). See Claim 10 for rationale (its parent claim).
Regarding Claim 15, Iqbal in view of Litvak teaches the method of claim 10, further comprising estimating a position of an invisible keypoint (Iqbal ¶0066-¶0068 discloses finding the position of the kth key point through estimation calculation). See Claim 10 for rationale (its parent claim).
Regarding Claim 16, Iqbal in view of Litvak teaches the method of claim 15, further comprising estimating a connection between the invisible keypoint (Iqbal ¶0027 discloses estimating the length of the skeletal model without the ground truth of the root joint) and a visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) selected from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand). See Claim 10 for rationale (its parent claim).
Regarding Claim 17, Iqbal in view of Litvak teaches the method of claim 16, wherein estimating the connection between the invisible keypoint (Iqbal ¶0027 discloses estimating the length of the skeletal model without the ground truth of the root joint) and a visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) based on a kinematic chain (Iqbal ¶0038 discloses the kinematic structure of the hand and using the prior component length distribution to determine the component position). See Claim 10 for rationale (its parent claim).
Regarding Claim 19, Iqbal teaches encoded with codes (Iqbal ¶0039 discloses a program), wherein the codes are to direct a processor to (Iqbal ¶0039 discloses a processor capable of executing a program):
receiving raw data from an external source (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera) via a communications interface(Iqbal Fig 1D, 155 and ¶0074 disclose an I/O unit configured to transmit and receive communication and receive locations of the keypoints), wherein the raw data includes a representation of a hand (Iqbal Fig 1C, ¶0003, ¶0005 discloses the input image representing an object, in this case a hand);
storing the raw data (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera) in a memory storage unit (Iqbal ¶0094 discloses storing data in a memory partition unit);
identifying a plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) from the raw data (Iqbal ¶0007, ¶0045, ¶0004 discloses obtaining latent pixel coordinate data from the input image taken by the camera)
identifying a connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) between a first visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) and a second visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses the palm as the second keypoint) from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand); and
to represent the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) of the connector (Iqbal ¶0022 and Fig 1B discloses lines connecting the keypoints) with a pose estimation engine (Iqbal Fig 2D, 200 discloses an keypoint estimation system, ¶0002 disclose the purpose is to estimate the 3D pose),
has a normalized magnitude (Iqbal Fig 1A 120, and ¶0006, ¶0026 discloses a scale unit which uses a calculated scale factor to normalize the depth value).
Iqbal does not explicitly disclose a non-transitory computer readable medium, with a pre-processing engine, generating a vector, a direction, wherein the vector.
Litvak is in the same field of hand pose estimation. Further, Litvak teaches a non-transitory computer readable medium (Litvak ¶0060 discloses a non-transitory computer readable medium) with a pre-processing engine (Litvak ¶0083-¶0084 discloses a preprocessing step completed on the image), generating a vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand) a direction (Litvak ¶0072, ¶0078, ¶0079 discloses the direction of vertical and horizontal edges) wherein the vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Iqbal the step of preprocessing the image and generating a vector to represent length or distance, as well as the rotational point as taught by Litvak to make the invention that can automatically detect the pose of a hand based with increased efficiency and accuracy by making the incoming image increasingly clear and uniform; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to address constraints on pose estimation such as environmental conditions such as sunlight, occlusions, and complexity of non-rigid hand poses present challenges to landmark detection and determination (Litvak ¶0004).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 9, 18, and 20 are rejected under 35 U.S.C. 103 as unpatentable over Iqbal in view of Litvak in further view of Chang et al. (J. Y. Chang, G. Moon and K. M. Lee, "V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 5079-5088 (hereafter referred to as Chang)).
Regarding Claim 9, Iqbal in view of Litvak teaches the apparatus of claim 1, further comprising a pose generator (Iqbal Fig 2D, 200 discloses an keypoint estimation system, ¶0002 disclose the purpose is to estimate the 3D pose) to generate a pose of the object (Iqbal ¶0039, ¶0042 discloses generating a 3D pose of a hand) based on the vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand).
Iqbal in view of Litvak does not explicitly disclose to apply a template.
Chang is in the same field of hand pose estimation. Further, Chang teaches to apply a template (Chang Pg 3 Col 1 ¶02 discloses template fitting used in the depth based 3D human pose estimation).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Iqbal in view of Litvak by including a template for the generation of the hand pose as taught by Chang to make the invention that can automatically detect the pose of a hand based with increased accuracy by having a reference guide; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for an hand pose estimation model that provides accurate estimates while running in real-time (Chang, Abstract).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 18, Iqbal in view of Litvak teaches the method of claim 10 further comprising to generate a pose of the object (Iqbal ¶0039, ¶0042 discloses generating a 3D pose of a hand) based on the vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand).
Iqbal in view of Litvak does not explicitly disclose apply a template.
Chang is in the same field of hand pose estimation. Further, Chang teaches apply a template (Chang Pg 3 Col 1 ¶02 discloses template fitting used in the depth based 3D human pose estimation).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Iqbal in view of Litvak by including a template for the generation of the hand pose as taught by Chang to make the invention that can automatically detect the pose of a hand based with increased accuracy by having a reference guide; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for an hand pose estimation model that provides accurate estimates while running in real-time (Chang, Abstract).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 20, Iqbal in view of Litvak teaches the non-transitory computer readable medium (Litvak ¶0060 discloses a non-transitory computer readable medium) claim 19, wherein the codes are to direct the processor (Iqbal ¶0039 discloses a processor capable of executing a program) to:
estimate a connection between the invisible keypoint (Iqbal ¶0027 discloses estimating the length of the skeletal model without the ground truth of the root joint) and a visible keypoint (Iqbal ¶0026, ¶0037, ¶0049, discloses a first joint as the keypoint) selected from the plurality of visible keypoints (Iqbal ¶0006, Fig 2b, ¶0022, ¶0044 discloses a set of 3D key points representing joints in the 2D image of the hand) based on a kinematic chain (Iqbal ¶0038 discloses the kinematic structure of the hand); and
to generate a pose of the hand (Iqbal ¶0039, ¶0042 discloses generating a 3D pose of a hand) based on the vector (Litvak ¶0070, ¶0075, ¶0088 discloses generating a vector based off of the depth values and the distance from the center to landmarks of the hand).
Iqbal in view of Litvak does not explicitly disclose apply a template.
Chang is in the same field of hand pose estimation. Further, Chang teaches apply a template (Chang Pg 3 Col 1 ¶02 discloses template fitting used in the depth based 3D human pose estimation).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Iqbal in view of Litvak by including a template for the generation of the hand pose as taught by Chang to make the invention that can automatically detect the pose of a hand based with increased accuracy by having a reference guide; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need for an hand pose estimation model that provides accurate estimates while running in real-time (Chang, Abstract).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
UA Patent Publication US-20170168586-A1 to Sinha et al. discloses an method and system for hand pose detection using a depth map.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL LYNN ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mistry Oneal can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/Examiner, Art Unit 2674 /ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674