Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant has canceled claims 5 and 15. Application now has pending claims 1, 3-4, 6-11, 13-14 and 16-20.
Response to Arguments
Applicant’s arguments, see Remarks page 10, filed 12/23/2025, with respect to claim 1 has been fully considered and are persuasive. The 112(b) rejection of claim 1 has been withdrawn.
Applicant’s arguments with respect to claims 1 and 11 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Therefore, the new rejection of HUA, in combination of D2 and Ge, disclose the newly amended limitations of claims 1 and 11. Therefore, this action is made FINAL.
Claim Objections
Claims 8 and 18 recite the limitation “a second transformer network” and “a third transformer network” in pages 5 and 8 respectively. As claim 8 and 18 are amended to be dependent on claims 1 and 11, respectively, there is insufficient antecedent basis for this limitation in the claims, as neither claims 1 nor 11 recite a first transformer network; instead claims 6 and 16 disclose it. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 8-11, 13-14, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lu-yan HUA CN-113221738-A, hereinafter HUA, and D2-6875028-B1, hereinafter D2, and Liuhao Ge US-20200184721-A1, hereinafter Ge.
As per claim 1, HUA discloses a method of estimating a pose of an object and a model of the object, the method comprising:acquiring a global feature of an input image and a location code of an object in a template model (see HUA top of page 5/30, wherein the input image comprises image information of a hand, i.e., global feature, and acquires the initial node information, i.e., location code, in the first hand model as further disclosed on page 6/30), the location code comprising location information for a joint point of the object (see HUA page 5/30, wherein the position information of each node, acquired from the initial node information, is acquired) wherein the location information comprises three dimensional coordinates (see HUA bottom of page 6/30, wherein the position information is three-dimensional position information);dividing the global feature into a plurality of sub-features that do not cross each other based on a local area of the object, wherein each of the plurality of sub-features include plural joint points (see HUA page 6/30 and FIG. 4, wherein the node information, acquired from the gesture attitude estimation data of the real-time hand image, i.e., global feature, includes a tree structure model [a tree structure is a non-intersecting structure], wherein the wrist is the root node and each finger is a branch. The sub-features would be the sub-nodes within the branches);determining a local area feature of the object, based on the global feature of the input image and based on the location code of the object in the template model (see HUA page 6/30, wherein each finger, i.e., local area of the hand image, is a branch off the wrist root node. The nodes, which are acquired from the gesture estimation, are structured in a parent-child relationship, and the distance between the nodes are calculated using Euclidean distance based on the position information); anddetermining location information for the joint point of the object in the input image and location information for the model vertex in the input image, based on the local area feature of the object (see HUA top of page 10/30, wherein the position information for the first joint node is adjusted, according to the rotatable angle which was calculated, or determined, in page 9/30, for each finger first node of the hand, i.e., local area feature of the object),wherein the determining of the local area feature of the object is based on the plurality of sub-features and based on the location code of the object in the template model (see HUA page 6/30 and FIGS. 3-4, wherein sub-nodes, or sub-features, of the node information, i.e., location code, in the first hand model are acquired to construct, form, or determine, the parent-child relationship tree structure for the fingers).
PNG
media_image1.png
477
471
media_image1.png
Greyscale
However, HUA fails to explicitly disclose where D2 teaches:location code comprising location information for a joint point of the object and location information for a model vertex, wherein the location information comprises three dimensional coordinates (see D2 page 3/20 and FIGS. 7-8, wherein the distance information to calculate the three-dimensional position of the hand joint position and each vertex label, i.e., location information and location code, is disclosed); anddetermining location information for the joint point of the object in the input image and location information for the model vertex in the input image (see D2 page 3/20, wherein the three-dimensional position of the hand joint position is calculated and the vertex estimation, which calculates three-dimensional position of each vertex label, are performed on the image input of a hand).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify HUA’s method by using D2’s teaching by including a model vertex to the location code in order to obtain a more detailed 3D information of the hand.
However, while HUA, in combination with D2, discloses grouping the plurality of sub-features, each including the plural joint points (see HUA page 6/30 and FIG. 4, wherein each finger is grouped using a parent-child relationship between the sub-nodes and comprises the bone length for their position information. These fingers comprise a plurality of joint points as discussed at the top of page 6/30 with reference to FIG. 3), it fails to explicitly disclose where Ge teaches:grouping the plurality of sub-features, each including the plural joint points, into a plurality of groups of sub-features based on positional relationships among the joint points of the sub-features (see Ge ¶71-72, wherein the set of neighboring features of vertices
N
(
v
i
)
and children vertices, i.e., grouped plurality of sub-features, which includes the joints, to create the hand mesh); anddetermining the location information for the joint point of the object in the input image and the location information for the model vertex in the input image by performing encoding on the basis of the grouping result (see Ge ¶69-71 and ¶79 and FIGS. 5 and 8, wherein the 3D location coordinates of the joints are estimated and correlates to the input image using the grouped vertices in the hand mesh. The estimated 2D heatmaps are encoded onto the feature maps, which contain the hand mesh vertices as disclosed prior in ¶60).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify HUA’s, in combination with D2, by using Ge’s teaching by including the grouping of sub-features to determine the location information of a model vertex in order to obtain a further detailed representation of the object.
As per claim 3, HUA, in combination with D2 and Ge, discloses the method of claim 1, wherein the local area feature comprises a feature representation of the joint point and model vertex of the local area of the object (see HUA page 6/30 and FIG. 4, wherein the digital identifiers in the finger branch are disclosed. Additionally, see Ge ¶79, wherein the skeletal joints [the joint positions are the finger positions, i.e., local area, as disclosed in ¶14] are identified from the 3D hand mesh, which comprises vertices, i.e., model vertex).
As per claim 4, HUA, in combination with D2 and Ge, together discloses the method of claim 3, wherein the determining of the local area feature of the object comprises acquiring the local area feature of the object by connecting the joint point in the local area corresponding to each sub-feature among the plurality of sub-features with coordinates of the model vertex (see Ge ¶69-70 and FIGS. 5 and 8, wherein the 3D coordinates of the vertices are generated and estimates the 3D hand pose from the mesh. See further ¶71, wherein the 3D joint locations are estimated from the 3D mesh, and the root-relative vertices and root-relative 3D hand joint locations are disclosed which come from the root vertex in the model, and see HUA page 6/30 and FIG. 4, wherein the node information of each finger in the first hand model are included in a tree structure parent-child relationship, wherein the sub-nodes are connected back to the wrist root node 0. See HUA top of page 6/30, wherein the joint nodes are disclosed, including the wrist node).
As per claim 8, HUA, in combination with D2 and Ge, discloses the method of claim 1, wherein the determining of the location information for the joint point of the object in the input image and the location information for the model vertex in the input image comprises (see Ge ¶69-71 and ¶79 and FIGS. 5 and 8, wherein the 3D location coordinates of the joints are estimated and correlates to the input image using the grouped vertices in the hand mesh):encoding each group of the plurality of groups between two or more sub-features through a second transformer network (see Ge ¶60, wherein the latent feature vector is input into a graph CNN used to infer the 3D feature of vertices, which includes the children vertices, i.e., sub-features disclosed in ¶71. This is done using a stacked hourglass network, i.e., a transformer network); and acquiring location information for at least one joint point of the object in the input image and location information for at least one model vertex in the input image by encoding the plurality of encoded groups of sub-features through a third transformer network (see Ge ¶79, wherein another machine learning technique, using a residual network encodes the estimated heat map with the feature maps, i.e., the encoded sub-feature group, estimates the hand shape and pose estimation to acquire the 3D hand joint locations).
As per claim 9, HUA, in combination with D2, discloses the method of claim 1, wherein the object comprises at least one of a human body, an animal, a part of the human body, or a part of the animal (see HUA page 5/30, wherein the image to be identified comprises a human hand).
As per claim 10, HUA, in combination with D2, discloses the method of claim 9, wherein the part of the human body comprises a hand part of the human body, and the local area of the object comprises at least one of a palm, a thumb, a forefinger, a middle finger, a ring finger, or a little finger (see HUA page 6/30 and FIG. 4, wherein the hand with 5 fingers is disclosed).
As per claim 11, the rationale provided in claim 1 is incorporated herein. In addition, HUA also discloses a data acquisition device, a feature configuration device, and an estimation device (see HUA page 2/30, wherein gesture recognition device, an adjusting module, and an electronic device comprising a processor are disclosed).
As per claims 13-14, the rationale provided in claims 3-4 are incorporated herein. In addition, the device of claims 13-14 correspond to the method of 3-4.
As per claims 18-20, the rationale provided in claims 8-10 are incorporated herein. In addition, the device of claims 18-20 correspond to the method of 9-10.
Claims 6-7 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over HUA, in combination with D2 and Ge, in further view of Kevin A. HOROWITZ US-20210240280-A1, hereinafter HOROWITZ.
As per claim 6, HUA, in combination with D2 and Ge, fails to explicitly disclose where HOROWITZ teaches: The method of claim 1, wherein the grouping of the sub-features based on the positional relationships among the local areas of the object comprises:encoding each of the sub-features through a first transformer network (see HOROWITZ ¶120 and FIG. 3G, wherein a normalized contour point set of the hand fingers is captured and stored, or encoded, onto a transformation storage such as a data structure, i.e., transformation network); andbased on the positional relationship between the sub-features, acquiring the plurality of groups of sub-features by grouping the encoded local area the sub-features (see HOROWITZ ¶120-123 and FIGS. 3G, 8B, and 9, wherein the different poses of the hand and fingers, i.e., sub-features, are stored and used to observe which best corresponds with the normalized contour set. See more specifically FIG. 3G wherein the different capsule positions, the fingers, have several different positions).
PNG
media_image2.png
509
695
media_image2.png
Greyscale
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify HUA’s, in combination with D2 and Ge, method by using HOROWITZ’s teaching by grouping local areas based on positional relationships of joint points to HUA’s location information in order to better categorize the local areas within the object.
As per claim 7, HUA, in combination with D2, Ge, and HOROWITZ, discloses the method of claim 6, wherein the grouping of the encoded sub-features based on the positional relationships among sub-features comprises:grouping the encoded sub-features according to a predetermined grouping rule based on the positional relationships among the sub-features, or grouping the encoded local area sub-features through a grouping network based on the positional relationships between the local areas sub-features of the object. (see HOROWITZ ¶123-125 and FIGS. 3G, 8B, and 9, wherein a predictive model of the hand poses, i.e., grouping network, contains contour sets wherein observed information of the hand positions are used to determine poses with respect to the reference frame, such that the hand position and fingers match).
As per claim 16-17, the rationale provided in claims 6-7 are incorporated herein. In addition, the device of claims 16-17 correspond to the method of claims 6-7.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bradley Obas Felix whose telephone number is (703)756-1314. The examiner can normally be reached M-F 8-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 5712728243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRADLEY O FELIX/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights