Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 12 is rejected under 35 U.S.C. 101 because claim 12 recites a system comprised of purely software. However, said software is not stored on a medium and the claim as a whole appears to be nothing more than data, thus defining functional descriptive material per se.
Functional descriptive material may be statutory if it resides on a "computer-readable medium or computer-readable memory." The claim(s) indicated above lack structure, and do not define a computer readable medium and are thus non-statutory for that reason (i.e., "When functional descriptive material is recorded on some computer- readable medium it becomes structurally and functionally interrelated to the medium and will be statutory in most cases since use of technology permits the function of the descriptive material to be realized" - Guidelines Annex IV). The scope of the presently claimed invention encompasses products that are not necessarily computer readable, and thus NOT able to impart any functionality of the recited program.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 5, 8, 10-12, 16, and 19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chen et al. (CN 113111861).
With respect to claim 1, Chen et al. disclose a method for reconstructing a three-dimensional face based on occlusion segmentation, comprising: inputting a target face image to a preconstructed parameter prediction model (paragraph 70, Step S101: The user uploads a face image, as shown in Figure 1a), wherein the parameter prediction model comprises an image feature extractor (paragraph 70, After the feature extraction device receives the face image, it first preprocesses the face, including extracting the 2D key points of the face image, to obtain Figure 1b) and an image segmentation decoder (paragraph 71, A face image is semantically segmented to obtain at least one semantic segmentation region and the region of an occlusion), and the parameter prediction model is trained based on a plurality of face training images, face key point information from the plurality of face training images, and a face occlusion segmentation region, until an association loss function between the image feature extractor and the image segmentation decoder reaches a predetermined state (paragraph 73, the normalized face image is input into the trained parametric regression model to obtain the 3D face model coefficients and the extrinsic parameters of the face model relative to the camera coordinate system, paragraph 90, Step S204: Input the 2D information and face texture map into the trained neural network model to obtain a 3D face rendering image, paragraph 98, constructing a loss function L, calculating the derivative of the loss function L with respect to the five parameters α, β, s, R, and t, and adjusting the parameter values according to the gradient using the gradient descent algorithm. Repeat the iterations multiple times to minimize the loss function L, that is, adjust the parameters to the optimal values); and
outputting a target face reconstruction parameter and a target face occlusion region of the target face image based on the parameter prediction model, and performing three-dimensional face reconstruction post-processing based on the target face reconstruction parameter and the target face occlusion region (paragraph 104, Repeat the iterations multiple times to minimize the loss function L, even if the difference between the second silhouette image and the first silhouette image is less than a threshold, that is, adjust the parameters to the optimal parameters, and obtain the face geometric model corresponding to the second silhouette image at the optimal time as the final face geometric model S, paragraph 135, The rendering unit generates a 3D face based on 2D information and the final texture map. The rendering unit may employ a neural network renderer).
With respect to claim 5, Chen et al. disclose the method according to claim 1, wherein outputting the target face reconstruction parameter and the target face occlusion region of the target face image based on the parameter prediction model comprises: acquiring a feature image corresponding to the target face image by outputting the target face image to the image feature extractor (paragraph 123, Step S302: Extract face texture maps from face images based on face geometric models using the face texture feature extraction method of the first aspect), acquiring the target face reconstruction parameter by integrating the feature image, inputting the feature image to the image segmentation decoder (paragraph 130, In the optimized 3D face reconstruction device, the image preprocessing device also includes region segmentation of the face image), and acquiring the target face occlusion region by performing image segmentation via the image segmentation decoder (paragraph 123, If there are occluded areas in the face image, in this step, a face completion algorithm can be used to complete the face image and then extract the texture image).
With respect to claim 8, Chen et al. disclose the method according to claim 1, wherein performing the three-dimensional face reconstruction post-processing based on the target face reconstruction parameter and the target face occlusion region comprises: generating a target three-dimensional face model by performing three-dimensional face reconstruction based on the target face reconstruction parameter, wherein the target three-dimensional face model comprises a target three-dimensional face shape and a target three-dimensional face texture (paragraph 88, Step S202: Using the face texture feature extraction method of the first aspect, extract the face texture map from the face image based on the face geometric model, paragraph 90, Step S204: Input the 2D information and face texture map into the trained neural network model to obtain a 3D face rendering image); and rendering an occlusion region in the target three-dimensional face model based on the target face occlusion region using the target face image (paragraph 123, If there are occluded areas in the face image, in this step, a face completion algorithm can be used to complete the face image and then extract the texture image), and rendering an unocclusion region in the target three-dimensional face model using a target material (paragraph 126, Step S304: Input the 2D information and face texture map into the trained neural network model to obtain a 3D face rendering image).
With respect to claim 10, Chen et al. disclose a device for reconstructing a three-dimensional face based on occlusion segmentation, comprising: a memory and one or more processors, wherein the memory is configured to store one or more programs, and the one or more processors, when loading and running the one or more programs (paragraph 142, The sixth aspect of the present invention provides a 3D face reconstruction device, including a memory, a processor, and a computer program stored in the memory and executable on the processor), are caused to execute the method of claim 1; see rationale for rejection of claim 1.
With respect to claim 11, Chen et al. disclose a non-transitory computer-readable storage medium, storing one or more computer-executable instructions, wherein the one or more computer-executable instructions, when loaded and executed by a processor of a computer (paragraph 143, The sixth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the face texture feature extraction method…or the face 3D face reconstruction method described in the second or third aspect and any possible thereof), cause the processor of the computer to execute the method of claim 1; see rationale for rejection of claim 1.
With respect to claim 12, Chen et al. disclose a computer program product, comprising: one or more instructions, wherein a computer or a processor, when loading and executing the one or more instructions (paragraph 143, The sixth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the face texture feature extraction method…or the face 3D face reconstruction method described in the second or third aspect and any possible thereof), is caused to perform the method for reconstructing the three-dimensional face based on occlusion segmentation as defined in claim 1; see rationale for rejection of claim 1.
With respect to claim 16, Chen et al. disclose the device according to claim 10, wherein the one or more processors, when loading and running the one or more programs, are caused to execute the method of claim 5; see rationale for rejection of claim 5.
With respect to claim 19, Chen et al. disclose the device according to claim 10, wherein the one or more processors, when loading and running the one or more programs, are caused to execute the method of claim 8; see rationale for rejection of claim 8.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (CN 113111861) in view of Miroshnikov et al. (U.S. PGPUB 20220414766) and further in view of Zhang et al. (CN 114821061).
With respect to claim 6, Chen et al. disclose the method according to claim 1. However, Chen et al. do not expressly disclose a parameter dimensionality of the target face reconstruction parameter output by the image feature extractor and a number of paths of the image segmentation decoder correspond to a model computing power configuration of the parameter prediction model.
Miroshnikov et al., who also deal with using machine learning, disclose a method wherein a parameter dimensionality of the target face reconstruction parameter output by the image feature extractor corresponds to a model computing power configuration of the parameter prediction model (paragraph 170, the computing platform may also be configured to constrain the overall number of unknown parameters that are included within the post-processed model object, because as the dimensionality of the unknown parameters included within the post-processed model object increases, the amount of time and/or compute resources needed to produce the different versions of the post-processed model object at block 512 likewise increases).
Chen et al. and Miroshnikov et al. are in the same field of endeavor, namely neural networks.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method wherein a parameter dimensionality of the target face reconstruction parameter output by the image feature extractor corresponds to a model computing power configuration of the parameter prediction model, as taught by Miroshnikov et al., to the Chen et al. system, because when selecting the transformations, the computing platform may be configured to balance between the advantages of using more complex transformations having multiple unknown parameters and the desire to constrain the dimensionality of the unknown parameters included within the post-processed model object to a level that still allows the different versions of the post-processed model object to be produced within an acceptable amount of time (paragraph 170 of Miroshnikov et al.).
Zhang et al., who also deal with using machine learning, disclose a method wherein and a number of paths of the image segmentation decoder corresponds to a model computing power configuration of the parameter prediction model (paragraph 3, The method based on deep convolutional neural networks is currently the mainstream
method for image semantic segmentation, and it adopts an encoder-decoder structure…Considering the reliability of real-world applications, researchers have focused more on lightweight semantic segmentation. Lightweight semantic segmentation uses a lightweight network structure model, which greatly reduces the number of model parameters and shortens the model training and prediction time).
Chen et al., Miroshnikov et al., and Zhang et al. are in the same field of endeavor, namely neural networks.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method wherein and a number of paths of the image segmentation decoder corresponds to a model computing power configuration of the parameter prediction model, as suggested by Zhang et al., to the Chen et al. as modified by Miroshnikov et al. system, because the large number of parameters puts a computational burden on the device and increases the running time. It is not suitable for use on edge devices with limited computing resources (paragraph 3 of Zhang et al.).
With respect to claim 17, Chen et al. as modified by Miroshnikov et al. and Zhang et al. disclose the device according to claim 10 for executing the method of claim 6; see rationale for rejection of claim 6.
Claim(s) 7 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (CN 113111861) in view of Wang et al. (U.S. PGPUB 20240135747).
With respect to claim 7, Chen et al. disclose the method according to claim 1, wherein prior to inputting the target face image to the preconstructed parameter prediction model, the method further comprises: acquiring stretching and translation parameters of the target face image by registering the target face image based on a face key point detector and a template face key point (paragraph 73, Here, the 3D face model coefficients include parameters α related to the face shape and β related to the face expression. The extrinsic parameters of the face model relative to the camera coordinate system include the scale parameter s, the rotation parameter R, and the
translation parameter t). However, Chen et al. do not expressly disclose cropping the target face image based on the stretching and translation parameters, such that the target face image meets standard face dimensions.
Wang et al., who also deal with 3d face reconstruction, disclose a method for cropping the target face image based on the stretching and translation parameters, such that the target face image meets standard face dimensions (paragraph 33, according to the smoothed face frame sequence, images at corresponding positions are cropped from the corresponding frames in the initial video (if a square frame exceeds an image boundary, fill it with 0 pixel), scaled to a predetermined size, and at least one target video is generated by splicing the images that have been scaled according to an order of the corresponding frames, and each of the at least one target video includes face images corresponding to the same object, paragraph 40, the face detection algorithm can be used to extract the face image of each frame in the target video and scale the face image of each frame to a predetermined size).
Chen et al. and Wang et al. are in the same field of endeavor, namely computer graphics.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method of cropping the target face image based on the stretching and translation parameters, such that the target face image meets standard face dimensions, as taught by Wang et al., to the Chen et al. system, because this would ensure an image quality of the face frame (paragraph 33 of Wang et al.).
With respect to claim 18, Chen et al. disclose the device according to claim 10, wherein the one or more processors, when loading and running the one or more programs, are further caused to execute the method of claim 7; see rationale for rejection of claim 7.
Allowable Subject Matter
Claims 2-4, 13-15, and 20-21 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Although Chen et al. disclose a similar training process (paragraph 90, paragraph 136), none of the cited art teaches or suggests the combination of steps and using all the recited data, i.e., using the three-dimensional face prediction image, the face prediction key point information, and the face prediction occlusion segmentation region as prediction samples, calculating the association loss function between the image feature extractor and the image segmentation decoder based on the training samples and the prediction samples, and the training process of the parameter prediction model is completed in a case where the association loss function reaches the predetermined state.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. PGPUB 20230419592 to Wang et al. for a method of training a 3D face reconstruction model
U.S. PGPUB 20230281945 to Cashman et al. for a method of 3D facial reconstruction
CN 116524106 to Kang for a method of using the face occlusion segmentation model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW GUS YANG whose telephone number is (571)272-5514. The examiner can normally be reached M-F 9 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW G YANG/Primary Examiner, Art Unit 2614
8/20/26
Prosecution Insights