Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 3, 14, and 21 are objected to because of a minor informality: in the inputting limitation, each of these claims recites “the initial image encoder” which does not have antecedent basis in the claims, although the meaning and scope of the claim is clear. The Examiner suggests amending this to read --an initial image encoder--. Appropriate correction is required.
Claim 4, 15, and 22 are objected to because of a minor informality: in the last limitation, each of these claims recites “the multi-angle…template set” which does not have antecedent basis in the claims, although the meaning and scope of the claim is clear. The Examiner suggests amending this to read --a multi-angle…template set--. Appropriate correction is required.
Claims 6 and 17 are objected to because of a minor informality: in the last limitation, each of these claim recites “a multi-angle…template set” which creates antecedent basis issues in the claim, although the meaning and scope of the claim is clear. The Examiner suggests amending this to read --the multi-angle…template set--. Appropriate correction is required.
Claims 7 and 18 are objected to because of a minor informality: each of these claims recites “the preset element…sample pair” which does not have antecedent basis in the claims, although the meaning and scope of the claim is clear. The Examiner suggests amending this to read --a preset element…sample pair--. Appropriate correction is required.
Claims 8 and 19 are objected to because of a minor informality: each of these claims recites “the main outline area” and “the non-preset element” which do not have antecedent basis in the claims, although the meaning and scope of the claim is clear. The Examiner suggests amending these to read --a main outline area-- and --a non-preset element--, respectively. Appropriate correction is required.
Claim 11 is objected to because of a minor informality: in the second line, the claim recites “the processor” which does not have antecedent basis in the claims, although the meaning and scope of the claim is clear. The Examiner suggests amending this to read --a processor--. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 5, 7, 8, 10, 11, 13, 14, 16, and 18-21 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Liang et al. (U.S. Patent Application Publication No. 2023/0115765), referred herein as Liang.
Regarding claim 1, Liang teaches a method for image processing, comprising: obtaining an original image of a target object to be processed, wherein a preset element in the original image is displayed in a first display form (figs 3 and 5, original image 301, 502; paragraph 58, lines 1-7; paragraph 60, lines 1-8; paragraph 90, the last 15 lines; an original image of a target object [e.g. the people/faces] is obtained, wherein a preset element [e.g. the mouth area] is displayed in a first form [e.g. smiling]);
inputting the original image into a pre-trained element removal processing model to obtain a preset element removal image for the target object, and matching the preset element removal image with a template image corresponding to the preset element displayed in a second display form based on preset attribute parameters of the target object (figs 3-5, preset element removal image 305, 501’’, template image 401, 501; paragraph 44; paragraph 61, lines 1-11; paragraph 65, lines 1-6; paragraph 90, the last 15 lines; paragraph 91, lines 1-9; the original image is input into a pre-trained model to obtain a preset element removal image of, e.g., the mouth area, which is then matched with a template image showing a corresponding preset element in a second display form [e.g. frowning] based on attributes of the object); and
inputting the preset element removal image, the template image and a mask image of the preset element in the template image into a preset image element migration model to obtain a target image for the target object, wherein the preset element in the target image is displayed in the second display form (figs 3 and 5; mask image 304, 501, target image 508; paragraph 61; paragraph 90, the last 15 lines; paragraph 91, the last 11 lines; the preset element removal image, template image, and a mask image are input into a migration model to obtain an output image displaying the preset element in the second display form).
Regarding claim 2, Liang teaches the method of claim 1, wherein the process of the preset image element migration model performing the image processing on the input image comprises: performing feature fusion on the preset element removed image, the template image, and the mask image by a preset image encoder of the preset image element migration model to obtain a target feature code; and decoding the target feature code by an image decoder of the preset image element migration model to obtain the target image, wherein the image decoder is a pre-trained image generator (paragraph 54, lines 1-15; paragraph 90, the last 15 lines; paragraph 91).
Regarding claim 3, Liang teaches the method of claim 2, wherein a training process of the preset image encoder comprises: combining a sample image without the preset element of a preset object, a preset display form template sample image of the preset element matching the image without the preset element, and a mask sample image of the preset element in the template sample image, to form a model training sample pair (fig 3, sample image 305 w/o preset, template sample image 302, mask sample image 304; paragraph 60, lines 1-8; paragraph 61; paragraph 65, lines 1-6);
inputting the model training sample pair into the initial image encoder to obtain an initial image feature code; and inputting the initial feature code into the image decoder to obtain an initial decoded image, and iteratively updating the initial image encoder according to a loss between the initial decoded image and the template sample image to obtain the preset image encoder (paragraph 90, the last 15 lines; paragraph 91, lines 1-9; paragraph 104; paragraphs 106 and 107; paragraphs 110 and 111).
Regarding claim 5, Liang teaches the method of claim 1, wherein after matching the preset element removal image with the template image corresponding to the preset element displayed in the second display form, the method further comprises: based on a facial geometry of the target object, performing image correction on a facial shape in the template image (paragraphs 32 and 43; paragraph 47; paragraph 54, lines 1-15; paragraph 65).
Regarding claim 7, Liang teaches the method of claim 1, wherein the element removal processing model is a neural network model obtained by training based on the preset element removal image sample pair, wherein the preset element removal image sample pair includes an original sample image of an object containing the preset element, and a sample image corresponding to the original sample image that does not contain the preset element (fig 3, sample pair including sample image 301 containing the preset element and sample image 305 comprising corresponding image without the preset element; paragraph 60, lines 1-8; paragraph 61).
Regarding claim 8, Liang teaches the method of claim 7, wherein the process of obtaining the sample image that does not contain the preset element comprises: identifying a main outline of the preset element in the original sample image; and processing pixel points of the preset element located in the main outline area as pixel points consistent with pixel information of the pixel points of the non-preset element in the main outline area, and processing the pixel points of the preset element located outside the main outline area as pixel points consistent with the pixel information of the pixel points of the non-preset element outside the main outline area to obtain the sample image that does not contain the preset element (fig 3, images 301/302/304 [preset element area], image 305 [non-preset element]; paragraph 43; paragraph 59, lines 1-8; paragraph 60, lines 1-8; paragraph 61; paragraph 91, lines 1-9).
Regarding claim 10, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the processor, storage, and program, which are taught by Liang, fig 10, processor 1001, storage 1008; paragraph 142, lines 1-17); thus they are rejected on similar grounds.
Regarding claim 11, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the medium, program, and processor, which are taught by Liang, fig 10, processor 1001, storage 1008; paragraph 142, lines 1-17); thus they are rejected on similar grounds.
Regarding claims 13 and 14, the limitations of these claims substantially correspond to the limitations of claims 2 and 3, respectively; thus they are rejected on similar grounds as their corresponding claims.
Regarding claims 16, 18, and 19, the limitations of these claims substantially correspond to the limitations of claims 5, 7, and 8, respectively; thus they are rejected on similar grounds as their corresponding claims.
Regarding claims 20 and 21, the limitations of these claims substantially correspond to the limitations of claims 2 and 3, respectively; thus they are rejected on similar grounds as their corresponding claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 6, 15, 17, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Liang, in view of Zhang et al. (U.S. Patent Application Publication No. 2024/02966624), referred herein as Zhang.
Regarding claim 4, Liang teaches the method of claim 1, wherein matching the preset element removal image with the template image corresponding to the preset element displayed in the second display form based on the preset attribute parameters of the target object comprises: identifying head posture data and facial key point information of the target object; and based on the head posture data and the facial key point information, matching the template image corresponding to the preset element removed image in a display image template set in which the preset element is displayed in the second display form (figs 3-5, the sets of images; paragraphs 41 and 42; paragraph 45; paragraph 59, lines 1-9; paragraph 65, lines 1-6; paragraph 91, lines 1-9).
Liang does not explicitly teach a multi-angle display image set.
However, in a similar field of endeavor, Zhang teaches a method for obtaining preset head and facial elements of a target object in a set of display images and inputting the images into a pre-trained processing model to produce a target image comprising the preset elements displayed in a different form (figs 3A and 3B; paragraph 46, lines 1-31; paragraphs 75 and 76; paragraphs 78 and 79), wherein the image set is a multi-angle display image set (paragraphs 90 and 91).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the multi-view image sets of Zhang with the image sets of Liang because this helps generate a more accurately trained model, and thus produce higher quality target image results (see, for example, Zhang, paragraph 77, the last 5 lines; paragraph 79, the last 5 lines; paragraph 103).
Regarding claim 6, Liang in view of Zhang teaches the method of claim 4, wherein the process of establishing the multi-angle display image template set in which the preset element is displayed in the second display form comprises: obtaining a pre-made three-dimensional head model of the preset element displayed in the second display form; and simulating a surround shooting process, photographing and rendering the three-dimensional head model at multiple angles, and obtaining images of the preset element displayed in the second display form at multiple angles to establish a multi-angle display image template set of the preset element displayed in the second display form (Liang, paragraph 61; paragraph 65, lines 1-6; paragraph 90, the last 15 lines; paragraph 91; Zhang, paragraphs 90 and 91; paragraph 98; the motivation to combine is similar to that discussed in the rejection of claim 4 above).
Regarding claims 15 and 17, the limitations of these claims substantially correspond to the limitations of claims 4 and 6, respectively; thus they are rejected on similar grounds as their corresponding claims.
Regarding claim 22, the limitations of this claim substantially correspond to the limitations of claim 4; thus they are rejected on similar grounds.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Wang (U.S. Patent Application Publication No. 2014/0043329); Method of augmented makeover with 3D face modeling and landmark alignment.
Fonte (U.S. Patent Application Publication No. 2015/0055086); Method and system to create products.
Gao (U.S. Patent Application Publication No. 2021/0272245); Sky filter method for panoramic images and portable terminal.
Frey (U.S. Patent Application Publication No. 2022/0301118); Image replacement inpainting.
Zhang (U.S. Patent Application Publication No. 2021/0390710); System for automatic object mask and hotspot tracking.
Liu (U.S. Patent Application Publication No. 2022/0292773); Methods and systems for personalized 3D head model deformation.
Yao (U.S. Patent Application Publication No. 2021/0209423); Method and apparatus for training face fusion model and electronic device.
Lee (U.S. Patent Application Publication No. 2022/0084234); Method and electronic device for identifying size of measurement target object.
Li (U.S. Patent No. 11,804,068); Systems and methods for image feature recognition using a lensless camera.
Kanazawa (U.S. Patent Application Publication No. 2023/0342890); High resolution inpainting with a machine-learned augmentation model and texture transfer.
Xu (U.S. Patent Application Publication No. 2023/0143452); Method and apparatus for generating image, electronic device and storage medium.
Yin (U.S. Patent Application Publication No. 2023/0086552); Image processing method and apparatus, device, storage medium, and computer program product.
Yu (U.S. Patent No. 12,347,116); Generating alpha mattes utilizing deep learning.
Lyu (U.S. Patent Application Publication No. 2024/0265501); Systems and methods for image processing.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached on Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights