Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claim 31 is rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement.
Claim 31 recites a non-transitory computer-readable storage medium implementing a method comprising inputting an original image into a predetermined model, outputting a prediction result for at least one prediction task comprising a key point prediction task, and wherein a loss item of the predetermined model in a training process comprises a first loss constructed based on an error distribution between a first prediction result of the key point prediction task and a key point position label.
The specification provides no written description of a key point prediction task, of a loss constructed based on an error distribution, or of a key point position label.
Applicant is advised to review claim 31 as filed. As presented in the published application, claim 31 is directed to a different invention than the remaining claims and the specification, and appears to be the result of a drafting error. Correction or cancellation is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 13 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Tan et al. (“MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing”) in view of Isola et al. (“Image-to-Image Translation with Conditional Adversarial Networks”).
Regarding claim 13, MichiGAN teaches a method for image processing comprising obtaining an image to be processed, wherein the image to be processed is an image with a preset object, a portion of pixels of the preset object are located in a subject contour region in the image to be processed, and a further portion of pixels of the preset object are located outside the subject contour region (Sec. 3, an input portrait image I in which a semantic hair mask M distinct from the background region of I, and Sec. 4.1, the hair shape is a 2D binary mask; the hair overlaps the head/face contour region and extends beyond it, so hair pixels lie both within and outside the subject contour region),
obtaining a target image by inputting the image to be processed to a preset object removal processing model, wherein the target image is an object removal image corresponding to the image with the preset object (Sec. 3, Eq. (1); when the hair region is reduced or removed, all non-hair pixels are kept and the background image inpainter fills the vacated hair region. Also see section 4.3, Eq. (8), the output portrait);
and the preset object removal processing model is a model obtained by training based on a pre-established set of image sample pairs, wherein each image sample pair comprises an original image and a removal image obtained by processing pixels of the preset object located outside the subject contour region and pixels of the preset object located in the subject contour region, respectively (Sec. 4.3, the background inpainter processes the hair pixels lying outside the retained region while the generator synthesizes content for the hair region; Sec. 5.2 and Sec. 7.1, trained with explicit supervision on image pairs having a ground-truth portrait image, using training pairs).
Isola thus teaches that a model can be trained on pre-established sample pairs, each pair comprising an original input image and a corresponding target image, so that the model produces the target image from the input image (Sec. 3.1, Eqs. (1), (3) and (4); Sec. 3.2).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to train the hair-editing/removal model of Tan on pre-established sample pairs each comprising an original image with the preset object and a corresponding object removal image as taught by Isola. The reason is to enable the model to directly and reliably produce the object removal image from an input image using supervised, paired ground-truth data.
Regarding claim 22, see the rejection of claim 1 and Tan Sec. 7.1; Isola Sec. 6).
Claim(s) 14 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Tan in view of Isola, and further in view of Yu et al. (“Free-Form Image Inpainting with Gated Convolution”).
Regarding claim 14, Tan teaches a construction process of an image sample pair comprising identifying a subject contour region presenting the preset object in an original image, i.e., generating the hair region of the portrait using a semantic segmentation network (Sec. 5.2; Sec. 4.1),
and obtaining a preset object removal image by processing pixels of the preset object located outside the subject contour region as pixels having consistent pixel information of non-preset-object pixels outside the subject contour region (see sec. 4.3, Eq. 8, the background image inpainter filling the hair region with image content from I; and forming the image sample pair using the original image and the removal image (Sec. 5.2, ground-truth image pairs).
Yu teaches an image inpainting model that fills a masked region of an image by synthesizing content consistent with the valid pixels surrounding that region (Sec. 1, Sec. 3.1, Sec. 3.2 using a binary mask to designate the region to be filled and an SN-PatchGAN discriminator together with a pixel-wise L1 reconstruction loss).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to construct the sample-pair removal image of Tan and Isola by inpainting the preset-object pixels located inside and outside the subject contour region with region-consistent content as taught by Yu. The reason is to produce realistic paired ground-truth removal images in which the removed-object regions are seamlessly consistent with their respective surroundings.
Regarding claim 23, see the rejection of claim 14;
Allowable Subject Matter
Claims 15-21 and 24-30 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The prior art of record does not teach obtaining a skull region binary image by inputting an original image into a skull region prediction model, and removing the preset object in two stages as specifically recited in claim 15; nor the training of the skull region prediction model from a matched three-dimensional skull model by planar projection of claims 16 nor the anchor-sampling calibration strategy for the facial skin patching model of claims 21.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure: Zhu et al. (“Barbershop: GAN-based Image Compositing using Segmentation Masks,”) teaches GAN-based hair editing and compositing of portrait images by aligning latent codes to a target segmentation mask, including altering the hair region of the segmentation mask to change or remove hair.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HADI AKHAVANNIK whose telephone number is (571)272-8622. The examiner can normally be reached 9 AM - 5 PM Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HADI AKHAVANNIK/Primary Examiner, Art Unit 2676
Prosecution Insights