DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
Limitations appearing inside of {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Claim 1 is pending in the application.
Information Disclosure Statement
Information Disclosure Statement (IDS) filed on 02/26/2025 has been considered.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The number of styles is unclear in the recited claim limitation "prompts indicating a style of the face images". The claims may be interpreted as multiple prompts, multiple images, and one style. Is there only one style for all of the prompts/face images, or does each face image have a prompt that indicates a style for each face image? To improve clarity, consider revising the limitation. See suggested wording below*. For the purpose of examination the limitation will be interpreted as the suggested wording.
In the claim limitation “a training dataset including paired groups of face images, landmarks, and prompts” it is unclear what "paired groups" consists of. Are there a pair of face images? According to the disclosure the "paired group" consists of a face image and corresponding landmarks (Specification ¶[0008]; dataset of image-landmark pairs). See suggested wording below*. For purpose of examination the limitation will be interpreted according to the specification (¶[0008]).
*Suggested wording: “…paired data, wherein the paired data includes the face images and corresponding landmarks for each of the face images, and prompts indicating a style corresponding to each of the face images”.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over “Dong” (Dong, Xuanyi, et al. "Style aggregated network for facial landmark detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018), as cited in the IDS (02/26/2026) in view of “Skrypnyk” (Skrypnyk et al., US 20240355064 A1).
Dong teaches a method of training a facial landmark detector, comprising:
{receiving, via a data interface}, a training dataset including
paired groups of face images, landmarks (Dong 2018, [§4.1, p5, col 1, ¶1-2]; 300W and AFLW are face datasets with 68 and 21 landmarks, respectively.), and {prompts indicating a style of the face images};
{training a text to image generation model via a reconstruction loss based on the training dataset};
generating, {via the text to image generation model}, a synthetic dataset including paired face images and landmarks (Dong 2018, [§3.1, p4, col 1, ¶2]; We first transfer the original dataset into three different styles by Adobe Photoshop (PS). These three transferred datasets accompanying with the original dataset are regarded as four classes to fine-tune the classification model ); and
training the facial landmark detector using a loss function based on the landmarks of the synthetic dataset and a prediction of the facial landmark detector (Dong [§4.2, p5, col 2, ¶2]; Evaluation. Normalized Mean Error (NME) is usually applied to evaluate the performance for facial landmark predictions [31, 43, 73]. For 300-W dataset, we use the interocular distance to normalize mean error following the same setting as in [46, 31, 7, 43]. For AFLW dataset, we use the face size to normalize mean error [31]. We also use Cumulative Error Distribution (CED) curve to compare the algorithms provided in [45]. Area Under the Curve (AUC) @ 0.08 error is also employed for evaluation [6, 55].),
wherein the prediction of the facial landmark detector is based on the face images of the synthetic dataset (Dong [§3.2, p4, col 2, ¶1]; The facial landmark prediction module leverages the mutual benefit of both the original images and the style-aggregated ones to overcome negative effects caused by style variations.).
Dong does not explicitly disclose receiving, via a data interface, prompts indicating a style of the face images; training a text to image generation model via a reconstruction loss based on the training dataset; and generating, via the text to image generation model, a synthetic dataset.
However, Skrypnyk, a similar field of endeavor of training generative machine learning models for generating face images, teaches
receiving, via a data interface (¶[0035]; multiple user devices, such as a mobile device 114, head-wearable apparatus 116, and a computer client device 118 that are communicatively connected to exchange data and messages), a training dataset including paired groups of face images (Skrypnyk, ¶[0166]; the interaction system 100 collects a diverse dataset of images related to the domain of interest (e.g., faces), landmarks (Skrypnyk, ¶[0200]; The interaction system identifies landmarks on a user's face from a camera feed using facial landmark detection algorithms and/or computer vision techniques.) and prompts indicating a style of the face images (Skrypnyk, ¶[0205]; The user interface enables a user to enter in a prompt of a desired texture or context for the type of augmentation. The user interface displays the prompt “zombie evil dead” 1302);
training a text to image generation model (Skrypnyk, ¶[0167]; the interaction system 100 trains a text-to-image embedding model that converts textual descriptions into embeddings suitable for conditioning the stable diffusion model.) via a reconstruction loss based on the training dataset (Skrypnyk, ¶[0112]; These ground truth responses can include identification of sources of data that can be used to respond to a prompt and/or generated image content or text content that responds to the prompt. The neural network engine 314 can iterate over the training data until a loss function satisfies a stopping criterion; ¶[0171]The denoising score matching objective measures the difference between the model's denoising predictions and the true denoising targets. This loss is minimized during the training process, which encourages the model to learn a better denoising process. By minimizing the denoising score matching objective, the stable diffusion model learns to generate images or populated image templates that closely align with the structure of the image template and the characteristics indicated in the user prompt. The model becomes capable of reversing the noise injection process effectively, resulting in high-quality image generation.); and
generating, via the text to image generation model, a synthetic dataset (Skrypnyk, ¶[0167] The interaction system 100 can train such a text-to-image embedding model using pre-trained language models fine-tuned on the collected dataset to generate embeddings that encode the semantic information of the user prompts. ¶[0107]; The generative machine learning models can be trained to generate a variety of different content …[which] are trained to receive a prompt as input (which can include …text) and to generate an output that responds to the prompt; the generative machine learning models generate an artificial image … that is responsive to the prompt.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include data interface as taught by Skrypnyk to the invention of Dong. The motivation to do so would be to enable the user to interact with a device, such as by entering a prompt.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include prompts indicating a style of the face images as taught by Skrypnyk to the invention of Dong. Dong teaches generating synthetic dataset using Adobe Photoshop to generate synthetic data of different styles, evidencing the need for different styles in training the facial landmark detector. The motivation to do so would be to leverage the power of generative machine learning to automate the creation of a broader range of augmentations based on user prompts.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include training a text to image generation model and using it to generate synthetic data as taught by Skrypnyk to the invention of Dong. The motivation to do so would be to convert textual descriptions into embeddings suitable for conditioning the stable diffusion model.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892 for full citations.
Yi et al., (2022) teaches generating synthetic data using face and landmark pairs and a prompt, however, the prompt is speech not text.
Qiao et al., (2022) teaches text to image using text prompts to generate stylized image data, but does not use landmarks.
Zeng (2023) teaches turning selfies into a cartoon and demonstrates generating synthetic data of different styles using stable diffusion.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHANDHANA PEDAPATI whose telephone number is 571-272-5325. The examiner can normally be reached M-F 8:30am-6pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHANDHANA PEDAPATI/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights