DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see Remarks at pages 17-18, filed 14 November 2025, with respect to the objection to the drawings have been fully considered and are persuasive. The objection has been withdrawn.
Applicant’s arguments, see Remarks at page 18, filed 14 November 2025, with respect to the objections to the title, disclosure, and abstract have been fully considered and are persuasive. The title has been updated and the amended Specification and abstract are acceptable. The objections have been withdrawn.
Applicant’s arguments, see Remarks at page 19, filed 14 November 2025, with respect to the rejection of claims 9, 10, and 19 under 35 U.S.C. 112(b) have been fully considered and are persuasive. The rejection has been withdrawn.
Applicant’s arguments, see Remarks at pages 19-22, filed 14 November 2025, with respect to the rejection of claims 1, 3, 4, 11, 13, 14, and 20 under 35 U.S.C. 101 have been fully considered and are persuasive.
Applicant argues on pages 19-21 that claim 1 is not directed to a mental process per Step 2A: Prong One because “claims do not recite a mental process when they do not contain limitations that can practically be performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations”, citing to CyberSource Corp. v. Retail Decisions, Inc. Applicant restates a holding from the Federal Circuit and argues that the lack of a rejection under 35 U.S.C. 101 for claim 2 implies those additional elements cannot practically be performed in the human mind. The Examiner respectfully disagrees.
The claims still recite a mental process because they contain limitations that can be performed in the human mind and/or with a physical aid like pen and paper or a computer. See MPEP 2106.04(a)(2), subsection III.B-C. For example, “acquiring an image-to-be-processed”, under the broadest reasonable interpretation, describes a person viewing an image and forming a mental picture of it in their mind, acquiring a physical image like a page of a magazine or a printed photograph, or viewing an image on a computer screen where the user initiated the acquisition and the computer merely retrieves and displays the image for the user to view. As another example, the PCA algorithm, which includes “acquiring a dimension reduction matrix”, is merely a set of mathematical operations (e.g., linear algebra) that could be calculated by a person using a pen and paper or by using a computer as a calculator to speed up the calculations. Even for large images, computing the principal components by hand is not difficult, just time consuming. The claims do not specify an image size. Thus, for smaller images, the calculations would be faster. Accordingly, the claims are not eligible at Step 2A: Prong One.
Applicant argues on pages 21-22 that the claims recite additional elements that integrate the judicial exception into a practical application per Step 2A: Prong Two because the additional elements amount to improving the functioning of a computer or another technology or field. Applicant argues the additional elements “provide certain advantages such as reducing the sparsity of features and improving the learning efficiency of the parameter prediction model”. Examiner agrees.
Paragraph 82 of the published Specification, cited on page 21 of Remarks, discloses the use of a principle component analysis (PCA) algorithm to reduce the dimensionality of the description information (e.g., grid of vertexes) before splicing. Similar subject matter from claim 2 has been incorporated into the independent claims. However, this subject matter in itself (PCA) only describes an abstract mathematical/logical process to reduce dimensionality. To achieve the purported benefits of reducing the sparsity of features and improving the learning efficiency of the parameter prediction model, the published Specification further discloses in paragraph 90: “S3: Call a to-be-trained parameter prediction model for parameter prediction based on the splicing processing result to obtain training parameters of the morphology controller.” This subject matter (calling a parameter prediction model based on the splicing result) appears in each of the independent claims in combination with the “acquire a dimension reduction matrix” (perform PCA) clause added to each independent claim. Thus, the disclosed technical solution to achieving the disclosed purported benefits is sufficiently reflected in the additional elements of each independent claim. Therefore, the claims are eligible at Step 2A: Prong Two and the rejection is withdrawn.
Applicant’s arguments, see Remarks at pages 22-25, filed 14 November 2025, with respect to rejections under 35 U.S.C. 102(a)(1) and 35 U.S.C. 103 have been fully considered but are not persuasive. With respect to the subject matter of claim 2 incorporated into claim 1, Applicant argues on page 23 that “the cited references do not teach or suggest at least the claimed ‘acquiring a dimension reduction matrix and performing dimension reduction processing on the description information through the dimension reduction matrix, and splicing the description information and the contour information after the dimension reduction processing;’ and ‘calling a parameter prediction model for parameter prediction based on a splicing processing result to obtain a target parameter.’” Examiner respectfully disagrees.
On page 23, Applicant argues Ramadan does not disclose “acquiring a dimension reduction matrix and performing dimension reduction processing on the description information through the dimension reduction matrix, and splicing the description information and the contour information after the dimension reduction processing.” Examiner respectfully disagrees.
PCA is a well-known technique of linear dimensionality reduction1 and as likewise noted by Ramadan as being “a well-known technique in computer vision and image recognition” (pg. 8, section IV.D). Ramadan explicitly discloses applying Principle Component Analysis (PCA) (see Abstract) to a three-dimensional mesh surface (i.e., description information) representing a face (see section III, par. 1). To compress a three-dimensional face mesh, a transform is applied to generate a feature vector representing the mesh. The feature vector is then used with PCA for dimensionality reduction. (See Ramadan at pg. 9, section V.B, “First the spherical wavelet transform is applied to the semiregular mesh of the face image. For further dimensionality reduction PCA is utilized to reduce the size of the feature vector.”). In PCA, the dimension reduction matrix is what projects or transforms the original data into a lower-dimensional subspace, and the set (matrix) of selected eigenvectors (i.e., the principal components) is the dimension reduction matrix. Ramadan indirectly refers to this matrix when stating “further dimensionality reduction PCA is utilized” in section V.B and directly refers to this matrix when stating “an alternate set of orthonormal basis vectors which best represent the data set” in section IV.D. Despite not literally using the phrase “dimension reduction matrix”, Ramadan discloses a dimension reduction matrix.
Kim discloses generating a three-dimensional face mesh for facial detection and Ramadan discloses generating a three-dimensional face mesh for facial recognition. For the same reasons provided in the rejection of claim 1 under 35 U.S.C. 103 below (and provided for the rejection of claim 2 under 35 U.S.C. 103 in the prior Office action), the combination of Ramadan’s disclosure of using PCA for dimensionality reduction in a mesh representation of facial data with Kim’s face detector/tracker teaches the alleged deficiency in the rejection under 35 U.S.C. 103 (i.e., no disclosure of a dimension reduction matrix).
On page 24, Applicant argues, “the cited portions of Ramadan do not teach splicing the feature vector that is reduced after performing the ‘dimensionality reduction PCA.’ Therefore, the cited portions of Ramadan do not teach or suggest the claimed ‘acquiring a dimension reduction matrix and performing dimension reduction processing on the description information through the dimension reduction matrix, and splicing the description information and the contour information after the dimension reduction processing.’” Examiner respectfully disagrees.
Kim discloses acquiring description information (positions of vertexes or feature points of the mesh) and contour information (distances/lines between vertexes forming mesh triangles). See Kim at par. 98. As explained above, Ramadan discloses a dimension reduction matrix. As explained on page 14 of the non-final Office action, the feature vector that represents the face mesh data (i.e., the contour and descriptor information) after PCA is applied is reduced in size. This is a compression operation. Splicing, under the broadest reasonable interpretation, means to join two things together. Each face model joins together the positions of data points that comprise the mesh and the contour information that spans the data points to form the overall mesh surface. In the context of dimensionality reduction, the compression of a face mesh into a lower-dimensional mesh is a splicing operation because the resulting lower-dimensional mesh is also combination of lower-dimensional representations of portions of the original face mesh. When the PCA dimensionality reduction operations finish, the resulting mesh represented by the PCA calculations is retained in a memory or other storage. In that state of preservation, the dimension-reduced face mesh data has been spliced from the original face mesh.
On page 24, Applicant argues, “the Office did not demonstrate how the ‘transformer 122’ is a ‘parameter prediction [model].’” Examiner respectfully disagrees.
Similar to the facial morphology correction of the instant application (See e.g., FIG. 1b), Kim discloses facial morphology correction. For example, FIG. 1b of the instant application shows how an initial morphology of a mouth region is corrected to follow an expected morphology. In a similar manner, FIG. 6 of Kim shows how an initial morphology of an eye region of a 3d face model (15) is corrected to follow an expected morphology (71) of a captured face. This correction is performed by “transformer 122”. See Kim at pars. 61 and 91. The direction and angle of the captured face are needed so that the transformer can determine how to map the captured data into the correct natural appearance of the corrected morphology. See Kim at par. 61. A detected angle and a detected direction are parameters. The transformer 122 is a software-based transformation module that takes a processing result (detected face direction and angle) and determines a mathematical transformation to correct an unnatural eye pose, i.e., alter the appearance of a person as if they are looking into the camera when in reality they are looking away. Any model or representation of a face incorporates some degree of error or uncertainty. A transformation that produces such a model, therefore, is a prediction of the ground truth of the actual face. The mathematical transformation that is calculated by the transformer 122 includes parameters of the overall transformation, where the linear transformation of each point is a parameter for that point. Thus, the transformer, which is a programmed representation of morphological correction (i.e., eye-gaze correction), and is thus a parameter prediction model, which when utilized by a processor, is called (retrieved/accessed/loaded) for use.
On page 24, Applicant argues, “Also, the Office also did not demonstrate how the ‘transformer 122’ transforms the mixed face shape’ by using the splicing processing result.” Examiner respectfully disagrees.
As acknowledged on page 14 of the non-final Office action, the Office conceded that Kim failed to teach “a splicing processing result”, but did teach a processing result, as discussed above with respect to the detected face direction and angle. Also as explained above, Ramadan discloses splicing description and contour information via the dimensionality reduction of PCA. Dimensionality reduction, as disclosed by Ramadan, offers a tradeoff between size and speed. Either a model is large and takes longer to process (which can degrade facial tracking algorithms), or a model is small and takes less time to process. Kim’s overall process includes tracking a face, determining how to adjust the eye region with the transformer, and modifying the eye region of the face to have a more natural appearance. The combination of Kim in view of Ramadan modifies Kim’s process by adding dimensionality reduction (compression) as disclosed by Ramadan. Dimensionality reduction of Kim’s face mesh using the basis vectors (i.e., dimension reduction matrix) of PCA as disclosed by Ramadan would be beneficial to Kim because it would allow a user to find an optimal balance between model size and processing efficiency. Using spherical wavelets like Ramadan is one approach, but non-wavelet based dimensionality reduction methods would be equally applicable.
For the foregoing reasons, the rejections under 35 U.S.C. 102(a)(1) are withdrawn as being necessitated by the amendment to claims 1 and 11 that incorporates subject matter from claims 2 and 12 respectively, which were previously rejected under 35 U.S.C. 103. The rejections under 35 U.S.C. 103 are maintained.
Claim Objections
Claims 1 and 20 are objected to because of the following informalities: “based on a splicing processing result” should be changed to “based on the splicing processing result” to be consistent with claim 11. Since “[splicing/splice] the description information and the contour information after the dimension reduction processing” corresponds to the “result” in “a splicing processing result”, based on the paragraph spanning pages 12 and 13 of the Specification, amending claims 1 and 20 to be consistent with claim 11 would also be consistent with the Specification and avoid ambiguity in claim interpretation when comparing the claimed inventions of the independent claims. Appropriate correction is required.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 11-14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Pat. Appl. Pub. No. 2015/0009123 to Kim et al. (hereinafter “Kim”) in view of 3D Face Compression and Recognition using Spherical Wavelet Parametrization to Ramadan et al. (hereinafter “Ramadan”).
Regarding claim 1, Kim teaches an image processing method, implemented by a computer device (par. [0057], “computer”), comprising:
acquiring an image-to-be-processed (par. [0110], “a display apparatus photographs the shape of a face (S910)”), the image-to-be-processed comprising a target part of an object (e.g., eyes of a face), a morphology of the target part being a first morphology (par. [0058], “in response to the photographing unit 110 being placed at the center of the top edge of the display apparatus 100, the shape of the eyes of the user being inputted into the photographing unit 110 directs downward”), and the first morphology being not matched with an expected morphology (par. [0082], “The 3D face model may be generated by considering error between the 2D photographed user's face shape and the 3D reference face model.”);
acquiring description information corresponding to the first morphology (par. [0082], “the 3D face model 15 in the form of a mesh may be generated from the face shape including the matched feature points 55-1 and 55-n”), and acquiring contour information about the target part in the first morphology (par. [0098], “The feature points and mesh triangles 31 may be formed continuously along the contour 11 of the face shape. Alternatively, the feature points and mesh triangles may be formed on the face shape as a whole. In other words, a face shape in the form of mesh may be generated.”; A 3D triangle has 3 sides, each of which is a contour that represents the shape of a facial part.);
calling a parameter prediction model (par. [0061], “transformer 11”) for parameter prediction based on a processing result (par. [0061], “using the detected direction and angle of the face shape”) to obtain a target parameter (how to change the face region); and
correcting the morphology of the target part based on the processing result in order to correct the morphology of the target part from the first morphology to a second morphology (par. [0091], “The transformer 122 may extract information of an eye portion 72 from the generated 3D face shape 15. The transformer 122 may generate a mixed face shape by projecting the extracted eye portion 72 and by mixing corresponding feature points or mesh triangles (or unit nets) of an eye portion 71 of the photographed face shape 11 a.”), the second morphology being matched with the expected morphology (par. [0094], “the transformer 122 may transform the shape of the mixed face so that the face shape and eyes face forward.”), but does not teach that which is explicitly taught by Ramadan.
Ramadan teaches acquiring a dimension reduction matrix and performing dimension reduction processing on the description information through the dimension reduction matrix (Ramadan, pg. 9, section V.B, “First the spherical wavelet transform is applied to the semiregular mesh of the face image. For further dimensionality reduction PCA is utilized to reduce the size of the feature vector.”; PCA processing uses a dimension reduction matrix. In PCA, the dimension reduction matrix is what projects or transforms the original data into a lower-dimensional subspace, and the set (matrix) of selected eigenvectors (i.e., the principal components) is the dimension reduction matrix. Ramadan indirectly refers to this matrix when stating “further dimensionality reduction PCA is utilized” in section V.B and directly refers to this matrix when stating “an alternate set of orthonormal basis vectors which best represent the data set” in section IV.D.), and splicing the description information and the contour information after the dimension reduction processing (Ramadan, pg. 8, “discrete bi-orthogonal spherical wavelets functions defined on a 3-D mesh”; The mesh is represented by spherical wavelets. After dimensionality reduction, the feature vector size is reduced and the corresponding contour information of the mesh is preserved in the resulting compressed signal, thereby splicing the contour and description information into the compressed signal.).
Kim discloses acquiring facial feature vectors defined on a 3D mesh representing an expression of a face before finding a matching reference face model. Thus, Kim shows that it was known in the art before the effective filing date of the claimed invention to reduce an input image to a more compact representation, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, improving facial expression correction efficiency. Ramadan discloses acquiring facial feature vectors defined on a 3D mesh and performing dimensionality reduction processing before finding a matching reference face model. Thus, Ramadan shows that it was known in the art before the effective filing date of the claimed invention to reduce an input image to a more compact representation using PCA, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, improving facial expression correction efficiency.
A person of ordinary skill in the art would have been motivated to apply the dimensionality reduction processing disclosed by Ramadan to the feature vectors acquired by Kim to thereby reduce the size of the data representing an input face before determining a difference with a target expression and thereby call the parameter prediction model (transformer) for parameter prediction based on a splicing processing result by compressing the reference and input mesh models using spherical wavelets and PCA dimensionality reduction (via a dimension reduction matrix). Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of more computationally efficient facial expression recognition.
Regarding claim 2, Kim in view of Ramadan teaches the method according to claim 1, wherein the target part is bound to a morphology controller (Kim, par. [0065], “controller 120”), and wherein the correcting the morphology comprises:
calling the morphology controller to adjust the description information based on the target parameter, the adjusted description information being used to enable the morphology of the target part to be presented as the second morphology (Kim, par. [0061], “and then transforms the mixed face shape by using the detected direction and angle of the face shape”).
Regarding claim 3, Kim in view of Ramadan teaches the method according to claim 1, wherein the image-to-be-processed is associated with an image grid, the image grid comprises M grid vertexes (The number of vertexes within the photographed face shape 11a shown in FIG. 6 of Kim.), and the target part corresponds to N grid vertexes in the M grid vertexes (Kim - The number of vertexes within the extracted eye portion 71 is a subset of all the vertexes of the entire face.), where M and N are integers larger than 1, and N is less than or equal to M (A subset of a set is smaller than the set.), and
wherein the description information corresponding to the first morphology comprises location information of the N grid vertexes (Kim - positions of each vertex of the extracted eye portion 71).
Regarding claim 4, Kim in view of Ramadan teaches the method according to claim 3 (See FIG. 6 of Kim annotated by examiner below), wherein the target part comprises an inner contour and an outer contour (Kim, par. [0098], “The feature points and mesh triangles 31 may be formed continuously along the contour 11 of the face shape. Alternatively, the feature points and mesh triangles may be formed on the face shape as a whole. In other words, a face shape in the form of mesh may be generated.”; A triangle has 3 sides, each of which is a contour.), the inner contour corresponds to L grid vertexes (Kim - An inner contour includes a single side of an inner triangle of the mesh of the extracted eye portion 71 defined by a distance between two vertexes.), the outer contour corresponds to P grid vertexes (Kim - An outer contour includes a single outer contour of the extracted eye portion 71 relative to the inner contour defined by a distance between two vertexes.), and the sum of L and P is less than N (Kim - 4 (2+2) is less than the total number of vertexes of the extracted eye portion 71.),
wherein the contour information of the target part comprises a first distance between the inner contour and the outer contour (Kim - Distance between vertexes of each contour) and a second distance between every two grid vertexes in the P grid vertexes (Kim - Distance between vertexes of each contour) corresponding to the outer contour (Kim - The P grid vertexes define the outer contour), and
wherein the first distance comprises a distance between the L grid vertexes and grid vertexes having a correspondence relationship from among the P grid vertexes (Kim - The first distance spans between L and P grid vertexes. See annotated FIG. 6 of Kim below.).
PNG
media_image1.png
447
1182
media_image1.png
Greyscale
FIG. 6 of Kim (annotated)
Claims 11-14 substantially correspond to claims 1-4 respectively by reciting an image processing apparatus, comprising at least one non-transitory memory (Kim, par. [0117], “a non-transitory computer-readable storage medium”) configured to store program code, and at least one processor (Kim, par. [0057], “computer”) configured to read the program code and operate as instructed by the program code, the program code corresponding to the steps of the methods of claims 1-4. The rationale for obviousness is the same as provided for claim 1.
Claim 20 substantially corresponds to claim 1 by reciting a non-transitory computer-readable medium (Kim, par. [0117], “a non-transitory computer-readable storage medium”) containing program code that when executed by at least one processor (Kim, par. [0057], “computer”) of an image processing device (Kim, par. [0056], “photographing unit 100”), causes the at least one processor to perform the steps of the method of claim 1. The rationale for obviousness is the same as provided for claim 1.
Allowable Subject Matter
Claims 5-10 and 15-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and overcoming any applicable claim objections for minor informalities.
Conclusion
Applicant's amendment necessitated the new ground of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN P POTTS whose telephone number is (571)272-6351. The examiner can normally be reached M-F, 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN P POTTS/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
1 https://web.archive.org/web/20201217000112/https://en.wikipedia.org/wiki/Principal_component_analysis
Prosecution Insights