Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 6 and 7 are objected to because of the following informalities:
Regarding claim 6, the numbering of “a fourth cartoon image” in line 4 is inconsistent, as the numbering implies that claim 6 is dependent on claim 2, which recites “a third cartoon image” in lines 5-6. For purposes of clarity, the Examiner respectfully suggests using “a third cartoon image” instead of “fourth cartoon image” in line 4, and using “the third cartoon image” instead of “the fourth cartoon image” in line 5.
Similarly, regarding claim 6, the numbering of “a second loss value” in line 5 implies that claim 6 is dependent on claim 2, which states “a first loss value” in line 9. For purposes of clarity, the Examiner respectfully suggests using “a first loss value” instead of “a second loss value” in line 5, and “the first loss value” instead of “the second loss value” in lines 7, 8, and 11.
Regarding claim 7, the numbering of the sub-losses, more specifically “a third sub-loss” in line 4, “a fourth sub-loss” in line 6, and “a fifth sub-loss” in line 8, are inconsistent, as it implies that claim 7 is dependent on claim 3, which recites “a first sub-loss” in line 6 and “a second sub-loss” in line 8. For purposes of clarity, the Examiner respectfully suggests using “first sub-loss”, “second sub-loss”, and “third sub-loss” instead of “third sub-loss”, “fourth sub-loss”, and “fifth sub-loss” respectively.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3, 6-7, 10, 13-14, 17, and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 recites the limitations "the second feature vector" and “the third feature vector” in lines 4 and 5 respectively. There is insufficient antecedent basis for this limitation in the claim. As they are the first instances of “second feature vector” and “third feature vector”, the Examiner interprets these limitations as “a second feature vector” in line 4 and “a third feature vector” in line 5, where “the second feature vector” and “the third feature vector” in lines 6-7 receive their basis from.
Claim 6 recites the limitation "the second threshold" in line 7. There is insufficient antecedent basis for this limitation in the claim. As this is the first instance of “second threshold”, the Examiner interprets this limitation as “a second threshold” in which “the second threshold” in line 11 receives its basis from.
Claims 10 and 17 recite substantially similar limitations to claim 3, therefore, rejected under 112(b) under the same rationale.
Claims 13 and 20 recite substantially similar limitations to claim 6, therefore, rejected under 112(b) under the same rationale.
Dependent claims 7 and 14 incorporate claim deficiency from claims 6 and 13 respectively and thus also rejected under 112(b).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Prasad et al. (US 2024/0046536 A1), in view of Wang et al. (WO 2020/199478 A1, hereinafter Wang).
Regarding claim 1, Prasad teaches a computer-implemented image generation method, the method comprising: ([Abstract] “The embodiments herein provide a system and method for personalized cartoon image generation.”)
wherein each set of the plurality of sets of paired data comprises a first face image and a first cartoon image corresponding to the first face image; ([0038] “According to one embodiment herein, the facial normalized image as an input to the trained AI model corresponds to a digital image depicting a face of a person. The AI model is trained using the supervised approach, in which paired data corresponding to photorealistic images and cartoon images of the photorealistic images are used for training.”)
training a second model based on the plurality of sets of paired data to obtain a trained second model; ([0038] “The AI model is trained using the supervised approach, in which paired data corresponding to photorealistic images and cartoon images of the photorealistic images are used for training.”. Note: the AI model is mapped to a second model)
and inputting a second face image to be processed into the trained second model to obtain a second cartoon image corresponding to the second face image ([0051] “At step 601, an input image is received. Further, at step 602, a cartoon image is generated from the input image (photorealistic image) using a trained AI model.”).
Prasad fails to teach generating a plurality of sets of paired data using a trained first model. However, this is known in the art as taught by Wang.
Wang teaches generating a plurality of sets of paired data using a trained first model ([0057] “Specifically, the preset comic generation algorithm uses an image processing algorithm to preprocess the captured images in the first image set to extract image information from the captured images… and constructs the target comic image corresponding to the captured image based on this image information.”, where “In a first aspect, this application provides a method for training an image generation model, which includes: Acquire a first image set and a second image set, wherein the first image set includes multiple captured images and the second image set includes multiple cartoon images; The captured image is preprocessed according to a preset comic generation algorithm to obtain the target comic image corresponding to the captured image;” [0006-0008], and “The target cartoon image is used as the input to the generator network... The generator network and the discriminator network are trained iteratively and alternately” [0010]. Note: a preset comic generation algorithm is mapped to a trained first model, where the trained generator network is the second trained network).
Wang is analogous to the claimed invention, as both relates to generating a cartoon image from a real image ([0029] “An image generation unit is used to input the target image into an image generation model to generate a corresponding comic image, wherein the image generation model is a model trained using the image generation model training method described above.”). Wang further teaches “the image generation model training method is used to quickly train an image generation model that can generate comic-style images; the image generation method can be applied to servers or terminals, using the image generation model to generate comic-style images from captured images, thereby improving the user experience.” [0045]. Therefore, it would be obvious for one of ordinary skill of the art before the effective filing date of the claimed invention to incorporate the teachings of Wang to Prasad in order to improve the user experience when generating cartoon images.
Regarding claim 8, claim 8 recites substantially similar limitations to claim 1, but in a device form. The combination of Prasad and Wang further teaches a device comprising: (Wang; [0030] “this application also provides a computer device”)
one or more processors; (Wang; [0030] “the computer device including a memory and a processor”)
and a memory coupled to the one or more processors, (Wang; [0141] “Referring to Figure 9, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.”)
the memory storing programs that, when executed by the one or more processors, cause performance of operations comprising: (Wang; [0030] “the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the image generation model training method or image generation method as described above.”).
Wang is analogous to the claimed invention, as both relates to generating a cartoon image from a real image ([0029] “An image generation unit is used to input the target image into an image generation model to generate a corresponding comic image, wherein the image generation model is a model trained using the image generation model training method described above.”). Therefore, it would be obvious for one of ordinary skill of the art before the effective filing date of the claimed invention to incorporate the teachings of Wang to the combination of Prasad and Wang teach that it is known in the art of cartoon image generation from real images to be in a computer device form as claimed.
Regarding claim 15, claim 15 recites substantially similar limitations to claim 1, but in a medium form. The combination of Prasad and Wang further teaches a non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a device, cause the at least one processor to perform a method ([0147] “The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement any of the image generation model training methods or image generation methods provided in the embodiments of this application.”).
Wang is analogous to the claimed invention, as both relates to generating a cartoon image from a real image ([0029] “An image generation unit is used to input the target image into an image generation model to generate a corresponding comic image, wherein the image generation model is a model trained using the image generation model training method described above.”). Therefore, it would be obvious for one of ordinary skill of the art before the effective filing date of the claimed invention to incorporate the teachings of Wang to the combination of Prasad and Wang to teach that it is known in the art of cartoon image generation from real images to be in a computer device form as claimed.
Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Prasad et al. (US 2024/0046536 A1), in view of Wang et al. (WO 2020/199478 A1, hereinafter Wang), and further in view of Lin (CN 115223013 A).
Regarding claim 6, the combination of Prasad and Wang teaches the method of claim 1, wherein training the second model based on the plurality of sets of paired data to obtain the trained second model comprises: for each set of the plurality of sets of paired data, inputting the first face image of the set of paired data into the second model to obtain a fourth cartoon image; (Prasad; [0050] “In an embodiment, a set of training images is collected, that contains photorealistic images of people's faces and digital cartoon images of people's faces at step 501.”)
and calculating a second loss value based on the fourth cartoon image and the first cartoon image of the set of paired data; (Prasad; [0050] “Further, the received input image (photorealistic image) is passed through a series of convolution and up-sampling layers. Finally, the method produces an output image which is the cartoon version of photorealistic images at step 503. Furthermore, at step 504, the generated cartoon images are compared with the targeted cartoon image for the loss.”).
The combination of Prasad and Wang fails to teach in response to the second loss value being greater than the second threshold updating model parameters of the second model based on the second loss value to obtain an updated second model, and continuing to train the updated second model based on a next set of paired data; and in response to the second loss value being less than or equal to the second threshold, determining the second model to be the trained second model.
However, this is known in the art as taught by Lin.
Lin teaches in response to the second loss value being greater than the second threshold updating model parameters of the second model based on the second loss value to obtain an updated second model, and continuing to train the updated second model based on a next set of paired data; ([0045] “S115. Adjust the parameter values contained in the initial image generation network
according to the preset parameter adjustment rules and the loss value.”, where “If the loss value is greater than the loss threshold, it indicates that the model training is insufficient, and the next face image can be obtained to further train the model.” [0051], and “S1153. If the loss value is greater than the loss threshold, randomly select a face image from the face image set as the current face image” [0050]. Note: The “initial image generation network” is mapped to the second model. Using the teaching of Lin to the combination of Prasad and Wang, Lin teaches that it is known in the art to use both face and cartoon images as input to train the second model.)
in response to the second loss value being greater than the second threshold, updating model parameters of the second model based on the second loss value to obtain an updated second model (
where “If the loss value is greater than the loss threshold, it indicates that the model training is insufficient, and the next face image can be obtained to further train the model.” [0051]. Note:).
and in response to the second loss value being less than or equal to the second threshold, determining the second model to be the trained second model. ([0050] “S1151, judging whether the loss value is not greater than the preset loss threshold value; S1152, if the loss value is not greater than the loss threshold value, the current obtained initial image generating network is determined as the first image generating network after training”).
Lin is analogous to the claimed invention, as both relate to training image generation models to creating cartoon images from real face images. Lin further teaches “If the loss value calculated in a certain instance is not greater than the loss threshold, it indicates that the first image generation network currently trained has been able to generate an output image that is close to the original face image.” [0051]. Therefore, it would be obvious for one of ordinary skill to incorporate the teachings of Lin to the combination of Prasad and Wang in order to generate cartoon images that are accurate to the input face image.
Regarding claim 13, claim 13 recites substantially similar limitations to claim 6, therefore, is rejected under the same rationale as claim 6.
Regarding claim 20, claim 20 recites substantially similar limitations to claim 6, therefore, is rejected under the same rationale as claim 6.
Allowable Subject Matter
Claims 2-5, 7, 9-12, 14, and 16-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, the closest prior art of Lin teaches in response to the first loss value being greater than a first threshold, updating model parameters of the first model based on the first loss value to obtain an updated first model, and continuing to train the updated first model based on a next sample image; ([0045] “S115. Adjust the parameter values contained in the initial image generation network according to the preset parameter adjustment rules and the loss value.”, where “If the loss value is greater than the loss threshold, it indicates that the model training is insufficient, and the next face image can be obtained to further train the model.” [0051], and “S1153. If the loss value is greater than the loss threshold, randomly select a face image from the face image set as the current face image” [0050].)
and in response to the first loss value being less than or equal to the first threshold, determining the first model to be the trained first model ([0050] “S1151, judging whether the loss value is not greater than the preset loss threshold value; S1152, if the loss value is not greater than the loss threshold value, the current obtained initial image generating network is determined as the first image generating network after training”).
However, the prior art taken singly or in combination do not teach or suggest the limitation of “wherein a first model comprises a first generation network and a second generation network; the method comprises: obtaining a sample image and a first feature vector corresponding to the sample image, wherein the sample image is a cartoon image; inputting the first feature vector into the first generation network to obtain a third cartoon image; inputting the first feature vector into the second generation network to obtain a third face image; calculating a first loss value based on the sample image, the third cartoon image and the third face image.”
Therefore, claim 2 is considered allowable.
Claims 9 and 16 contain substantially similar limitations to claim 2, therefore, also contains allowable subject matter.
Claims 3-5, 10-12, and 17-19 contain allowable subject matter because it depends on claims 2, 9, and 16 respectively, which contains allowable subject matter.
Regarding claim 7, the closest prior art of Song et al. (US 2024/0135627 A1) teaches calculating a pixel difference between the fourth cartoon image and the first cartoon image to obtain a third sub-loss; ([0087] “On the other hand, the perception loss may correspond to a difference between each pixel of the plurality of pixels of the stylized image to each respective pixel of the plurality of pixels of the avatar image.” Note: the stylized image is mapped to the first cartoon image, and the avatar image is mapped to the fourth cartoon image).
However, the prior art taken singly or in combination do not teach or suggest the limitation of
“wherein calculating the second loss value based on the fourth cartoon image and the first cartoon image of the set of paired data comprises: calculating a square difference between the pixels of the fourth cartoon image and the first cartoon image to obtain a fourth sub-loss; calculating the difference between a feature vector of the fourth cartoon image and a feature vector of the first cartoon image to obtain a fifth sub-loss; and calculating the second loss value based on the third sub-loss, the fourth sub-loss and the fifth sub-loss.”
Therefore, claim 7 is considered allowable.
Claim 14 contains substantially similar limitations to claim 7, therefore, also contains allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALICIA HA whose telephone number is (571)272-3601. The examiner can normally be reached Mon-Thurs 9:30 AM - 6:30 PM, and Fri 9:30 AM - 1:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALICIA HA/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights