DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-6 and 13-14 of U.S. Patent No. 12,307,599.
Table: 1
Mapping of contending claims in the patent that contains double patenting issues.
Current Application (19182929)
Patent (12,307,599)
1
1
2
1
3
1
4
1
5
1
6
2
7
3
8
4
9
5
10
6
11
13
12
13
13
13
14
13
15
13
16
14
17
14
18
14
19
14
20
14
Table: 2
Current Application (19182929)
Patent (12,307,599)
Claim 1, A method comprising:
obtaining a deformed three-dimensional mesh from a storage device, the deformed three-dimensional mesh generated using a method comprising:
determining via a processor a correspondence between a two-dimensional image of a three-dimensional object and a UV representation of a three-dimensional mesh of the three-dimensional object by training a neural network, the three-dimensional mesh including a plurality of points in three-dimensional space and a plurality of edges between the plurality of points; and
determining via the processor a deformation of the three-dimensional mesh, the deformation displacing one or more of the plurality of points, wherein the deformation is determined so as to reduce loss when mapping points from the two-dimensional images back onto the two-dimensional images through both the UV representation and the three-dimensional mesh.
Claim 1, A method comprising:
. . . storing on a storage device a deformed three-dimensional template mesh.
determining via a processor a correspondence between one or more two-dimensional images of a three-dimensional object and a UV representation of a three-dimensional template mesh of the three-dimensional object by training a neural network, the three-dimensional template mesh including a plurality of points in three-dimensional space and a plurality of edges between the plurality of points;
determining via the processor a deformation of the three-dimensional template mesh, the deformation displacing one or more of the plurality of points, wherein the deformation is determined so as to reduce reprojection consistency loss when mapping points from the two-dimensional images back onto the two-dimensional images through both the UV representation and the three-dimensional template mesh, . . .
Claim 1 is rejected for obviousness type double patenting over claim 1 of U.S. Patent No. 12,307,599 for having similar limitations as described in Table 2. Although the conflicting claims are not identical, they are not patentably distinct from each other because the scope of the inventions is the same. Claim 1 of current application is an obvious variant and anticipated by claim 1 U.S. Patent No. 12,307,599.
The same logic applies to Claims 2-20. They are rejected for obviousness type double patenting under claims 1-6 and 13-14 of U.S. Patent No. 12,307,599.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US 20210287430 A1), and in view of Corazza et al. (US 20160027200 A1).
Regarding Claim 1, Li discloses A method (Abstract reciting “Apparatuses, systems, and techniques to identify a shape or camera pose of a three-dimensional object from a two-dimensional image of the object.”) comprising:
determining via a processor a correspondence between a two-dimensional image of a three-dimensional object and a UV representation of a three-dimensional mesh of the three-dimensional object by training a neural network, (¶2 reciting “a processor comprises one or more arithmetic logic units (ALUs) to perform training and/or inferencing using neural networks. In at least one embodiment, a neural network can be trained to reconstruct three-dimensional meshes of objects from two-dimensional images of objects and object masks.” Further, ¶82 reciting “To reconstruct mesh textures, the texture decoder can output a UV texture flow (Iflow) that may map pixels from the input image to the UV space. A pre-defined mapping function Φ may further map each pixel in the UV space to a point on the mesh surface.” Fig. 13, lower part, showing the correspondence between an input image to a UV texture and to a reconstructed mesh) the three-dimensional mesh including a plurality of points in three-dimensional space and a plurality of edges between the plurality of points; (It is well known in the art that a 3D mesh including a plurality of points in 3D space and a plurality of edges between the plurality of points. Fig. 116 and Fig. 4 disclose the bird mesh have a plurality of points.) and
determining via the processor a deformation of the three-dimensional mesh, the deformation displacing one or more of the plurality of points, wherein the deformation is determined so as to reduce loss when mapping points from the two-dimensional images back onto the two-dimensional images through both the UV representation and the three-dimensional mesh. (¶82 reciting “To reconstruct mesh textures, the texture decoder can output a UV texture flow (Iflow) that may map pixels from the input image to the UV space. A pre-defined mapping function Φ may further map each pixel in the UV space to a point on the mesh surface.” Further, ¶86 reciting “training of neural network 102 includes feedback from loss evaluator 142 that provides feedback according to loss functions. In at least one embodiment, those loss functions are evaluated to loss function values used for updating neural network 102. In at least one embodiment, a loss function is a comparison of an input image and a reconstructed 2D image reconstructed from a 3D mesh, a texture flow array, and a camera pose estimated from that input image.” In addition, ¶111 reciting “once reconstructed, a reconstructed output image can be compared with the input image and its object mask by a comparator that can provide comparison results to a loss evaluator.”; and ¶113 reciting “training seeks to reduce losses from encoding and decoding”)
However, Li does not explicitly disclose obtaining a deformed three-dimensional mesh from a storage device, the deformed three-dimensional mesh generated using the method.
Corazza teaches “systems and methods for automatically determining correspondences between three-dimensional models” (¶3). ¶10 teaches determining correspondence values between 3D models via UV maps. Further, ¶55 recites “the character creation system can create a 2D mesh map from a 3D mesh. For example, the character creation system may create a UV mesh map from the 3D template mesh 200. To illustrate, FIG. 3 shows an example UV map 300 of the 3D template mesh 200 of FIG. 2.” ¶35 teaches creating a character by deforming a predefined template mesh to match the scan of the user's face and/or body, and recites “Using the correspondence values, as described above, the character creation system can create a 3D model of the scan of the user's face as well as allow the user to customize the 3D model (e.g., add a body and clothing, apply a different hair style, add animation).” More specifically, ¶78 teaches retrieving a created character, and recites “The menu toolbar 502 can include options such as creating a new character, opening an existing character, saving the current character, undoing, and redoing. ”
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the method (taught by Li) to use a UV representation of the 3D mesh for 3D model creation and to save and to retrieve the created mesh on a storage (taught by Corazza). The suggestions/motivations would have been to save the created model as a default character for future reference (¶78).
Regarding Claim 2, Li in view of Corazza discloses The method of claim 1, wherein the two-dimensional image includes a proximate two-dimensional image, the proximate two-dimensional image being captured from a proximate virtual camera pose. (Li, ¶85 reciting “to reconstruct a 2D image of the object depicted in the input image, but from a different camera pose.” Further, Fig. 5 showing a reconstructed 2D image generated with an alternative camera pose. ¶109)
Regarding Claim 3, Li in view of Corazza discloses The method of claim 2, wherein a loss value depends in part on a proximate loss value computed for a corresponding pixel in the proximate two-dimensional image a proximate virtual camera pose. (Li, ¶86 reciting “those loss functions are evaluated to loss function values used for updating neural network 102. In at least one embodiment, a loss function is a comparison of an input image and a reconstructed 2D image reconstructed from a 3D mesh, a texture flow array, and a camera pose estimated from that input image.”)
Regarding Claim 4, Li in view of Corazza discloses The method of claim 3, wherein loss is a reprojection consistency loss. (Li, ¶125)
Regarding Claim 5, Li in view of Corazza discloses The method of claim 4, wherein the three-dimensional mesh is a three-dimensional template mesh. (Li, ¶82 reciting ““the 3D reconstruction model may jointly predict the shape and texture of an object, and the camera pose of an image to fully reconstruct a 3D mesh of an object instance from an image. Given an input image, the reconstruction network can extract the image features using an encoder E and jointly predict the mesh shape, camera pose and mesh texture by three decoders Dshape, Dcamera and Dtexture. The mesh shape V may be reconstructed by predicting vertex offsets ΔV to a category-specific shape template V, while the camera pose θ may be represented by a weak perspective transformation.””)
Regarding Claim 6, Li in view of Corazza discloses The method recited in claim 5, wherein training the neural network comprises predicting, for a first location in a designated one of the two-dimensional images, a corresponding second location in the UV representation. (Li, ¶82 reciting “To reconstruct mesh textures, the texture decoder can output a UV texture flow (Iflow) that may map pixels from the input image to the UV space.”)
Regarding Claim 7, Li in view of Corazza discloses The method recited in claim 6, wherein training the neural network further comprises determining a third location in the three-dimensional template mesh by mapping the second location to the third location via UV parameterization. (Li, ¶82 reciting “A pre-defined mapping function Φ may further map each pixel in the UV space to a point on the mesh surface.”)
Regarding Claim 8, Li in view of Corazza discloses The method recited in claim 7, wherein training the neural network further comprises determining a fourth location in the designated two-dimensional image by projecting the third location onto the virtual camera pose associated with the designated two-dimensional image. (Li, ¶128 reciting “As shown in FIG. 13 (lower portion), considering the pixel marked with a yellow cross in the input image, it can be mapped to the mesh surface through the predicted texture flow Iflow along with the pre-defined mapping function Φ. Meanwhile, its mapping on the mesh surface can be re-projected back to the 2D image by the predicted camera pose, as shown by the green cross in FIG. 13 (lower portion).”)
Regarding Claim 9, Li in view of Corazza discloses The method recited in claim 8, wherein training the neural network further comprises determining a reprojection consistency loss value representing a displacement in two-dimensional space between the first location and the fourth location. (Li, Claim 1 reciting “the loss function is based on differences between the reconstructed image and the input image”)
Regarding Claim 10, Li in view of Corazza discloses The method recited in claim 9, wherein training the neural network further comprises updating the neural network based on the reprojection consistency loss value. (Li, Claim 19 reciting “use the loss function to update the neural network”)
Regarding Claim 11, Li in view of Corazza discloses A method comprising:
receiving a two-dimensional image of a three-dimensional object;
determining via a processor a correspondence between the two-dimensional image of the three-dimensional object and a UV representation of a three-dimensional mesh of the three-dimensional object by training a neural network, the three-dimensional mesh including a plurality of points in three-dimensional space and a plurality of edges between the plurality of points;
determining via the processor a deformation of the three-dimensional mesh, the deformation displacing one or more of the plurality of points, wherein the deformation is determined so as to reduce loss when mapping points from the two-dimensional images back onto the two-dimensional images through both the UV representation and the three-dimensional mesh;
(See claim 1 rejections for detailed analysis.)
and
saving the deformed three-dimensional mesh to a storage device. (Corazza, ¶78 teaching storing a created character, and reciting “The menu toolbar 502 can include options such as creating a new character, opening an existing character, saving the current character, undoing, and redoing. ” The suggestions/motivations would have been the same as that of Claim 1 rejections.)
Claim 12, has similar limitations as of Claim(s) 11 and 2, therefore it is rejected under the same rationale as Claim(s) 11 and 2.
Claim 13, has similar limitations as of Claim(s) 11 and 3, therefore it is rejected under the same rationale as Claim(s) 11 and 3.
Claim 14, has similar limitations as of Claim(s) 11 and 4, therefore it is rejected under the same rationale as Claim(s) 11 and 4.
Claim 15, has similar limitations as of Claim(s) 11 and 5, therefore it is rejected under the same rationale as Claim(s) 11 and 5.
Claim 16, has similar limitations as of Claim(s) 11 and 6, therefore it is rejected under the same rationale as Claim(s) 11 and 6.
Claim 17, has similar limitations as of Claim(s) 11 and 7, therefore it is rejected under the same rationale as Claim(s) 11 and 7.
Claim 18, has similar limitations as of Claim(s) 11 and 8, therefore it is rejected under the same rationale as Claim(s) 11 and 8.
Claim 19, has similar limitations as of Claim(s) 11 and 9, therefore it is rejected under the same rationale as Claim(s) 11 and 9.
Claim 20, has similar limitations as of Claim(s) 11 and 10, therefore it is rejected under the same rationale as Claim(s) 11 and 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YI WANG whose telephone number is (571)272-6022. The examiner can normally be reached 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YI WANG/Primary Examiner, Art Unit 2619
