Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 2 and 6 are objected to because of the following informalities:
Regarding claim 2, claim 2 recites the limitations “at least one of a group consisting of a pet head portrait plug-in and a pet full body portrait plug-in” in lines 2-3 and “wherein the image feature of the to-be-processed pet image comprises at least one of a group consisting of a head portrait feature and a full body portrait feature;” in lines 3-4. The wording of the limitation “at least one of a group” is confusing, as it is uncertain whether the group has both a head portrait plug-in and a full body portrait plug-in or not. To advance compact prosecution, the Examiner interprets this limitation has having a group, where the group can have either a head portrait plug-in or a full body portrait plug-in, but it could contain both but is not necessary. This interpretation is also used for “at least one of a group consisting of a head portrait feature and a full body portrait feature” as well, where the group can either a head portrait feature or a full body portrait feature.
Regarding claim 6, claim 6 recites the limitations “and supplementing the first text based on the breed of the target pet to obtain a second text;” in line 3 and “inputting the second text obtained through supplementing the first text into an image generation model.” in lines 5-6. This claim may be more appropriately written by removing “and supplementing the first text based on the breed of the target pet to obtain a second text;” in line 3, and rewriting “inputting the second text” as “inputting a second text” in line 5 in order to improve clarity and remove redundancy.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-7, 9-10, 15-16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. (US 20250329079 A1, hereinafter Zhao), in view of Zhang et al. (CN 117975152 A, hereinafter Zhang, see English Translation).
Zhou teaches a method for generating a pet image, wherein the method comprises: (Fig. 8, where “A method, apparatus, non-transitory computer readable medium, and system for image processing… where the synthetic image depicts the modification to the input image” [Abstract], and “For example, input image 805 depicts a corgi.” [0126])
obtaining a first text input by a user and a to-be-processed pet image input by the user, ([0004] “Aspects for a method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining an input image and a text prompt”)
and the to-be-processed pet image comprises a target pet for generating the target pet image; ([0126] “For example, input image 805 depicts a corgi.”, where “In some cases, the image embedding captures semantic information of the input image. For example, semantic information may include classifying objects (such as a person, animal, vehicle, or any other relevant entities) depicted in the input image.” [0164])
and obtaining and outputting the target pet image output by the image generation model. ([0004] “An aspect further includes generating, using an image generation model, a synthetic image based on the input image and an output embedding of the language generation model, where the synthetic image depicts the modification to the input image.”)
Zhao teaches and process a text input into the image generation model, ([0004] “In one aspect, the language generation model is trained to generate a guidance embedding for the image generation model based on the image embedding of the input image and a text embedding of an input text prompt.”)
and inputting the first text into an image generation model… to generate the target pet image, ([0004] “In one aspect, the image generation model is trained to generate a synthetic image based on the image embedding of the input image and the guidance embedding generated by the language generation model, where the synthetic image includes the element.”)
Zhao teaches wherein the first text indicates a guidance for a to-be-generated target pet image, ([0126] “In one aspect, text prompt 820 is a query that states ‘Can you generate an image of this corgi on a purple rug in a forest?’”, where “For example, based on input image 805 and text prompt 820, guidance embedding 855 may include high-level semantic information such as corgi, purple rug, forest, etc.” [0128]), but fails to teach wherein the first text indicates a requirement for a to-be-generated target pet image. Zhao further fails to teach inputting an image feature of the to-be-processed pet image into a pre-trained first plug-in of the image generation model, wherein the first plug-in is configured to process the image feature, and the image generation model is configured to interact with the first plug-in. However, this is known in the art as taught by Zhang.
Zhang teaches wherein the first text indicates a requirement for a to-be-generated target pet image, ([0039] “after fusing a subset of pet images corresponding to the required pet features and image style with a trained stable diffusion SD model, a pet image is generated according to the requirements”, where “For example, one feature low-rank adapter Lora is trained using a subset of images with pet features such as ‘a Labrador Retriever running fast’ and their annotations, while another feature low-rank adapter Lora is trained using a subset of images with pet features such as ‘a Labrador Retriever with yellow ears’ and their annotations.” [0032]. Note: the “required pet features and image style” is mapped to the requirement.)
inputting an image feature of the to-be-processed pet image into a pre-trained first plug-in of the image generation model, wherein the first plug-in is configured to process the image feature and process a text input into the image generation model, and the image generation model is configured to interact with the first plug-in ([0032] “In one embodiment, there are multiple feature low-rank adapters Lora and multiple style low-rank adapters Lora. They were trained using a subset of images with different key pet characteristics and different image styles.”, where “In one embodiment, the feature low-rank adapter Lora and the style low-rank adapter are used as plugins for the stable diffusion SD model” [0033], and “creating a feature low-rank adapter Lora and a style low-rank adapter Lora for the pet features and image styles” [0009]. Note: the “feature low-rank adapter Lora” is mapped to the first plug-in, where the “pet features” map to the image feature, and the “stable diffusion SD model” is mapped to the image generation model).
Zhang is analogous to the claimed invention, as both relate to generating pet images. Zhang further teaches that this “enabl[es] training with a small amount of data without modifying the diffusion model, achieving customized requirements, and improving image generation efficiency.” [0033]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Zhang to Zhao in order to improve pet image generation with customized requirements with efficient training.
Regarding claim 6, the combination of Zhao and Zhang teaches the method according to claim 1, the method further comprises: identifying the to-be-processed pet image to determine a breed of the target pet; (Zhang; [0017] “The attribute description information includes at least information related to the key features of the pet represented by the pet image”, where “In this embodiment, the attribute description information of a pet image is as follows: Breed” [0018])
and supplementing the first text based on the breed of the target pet to obtain a second text; and the inputting the first text into an image generation model comprises: inputting the second text obtained through supplementing the first text into an image generation model. ([0015] “combining the attribute description information and image style information to generate refined labels for the pet images;”, where “the user, who can then make a selection from these prompts. In one embodiment, users can choose from categories such as "high-definition photos of adult Labrador Retrievers with white textured skin and yellow ears". Note: the “image style information” is mapped to first text, and “attribute description information” is mapped to the second text.).
Zhang is analogous to the claimed invention, as both relate to generating pet images. Zhang further teaches that the invention addresses the problems that “compared to generating photos of humans, there is a wide variety of pets, and even within the same species, there are many different sub-breeds… This results in the generated pet images often falling far short of user requirements.” [0005]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Zhang to the combination of Zhao and Zhang in order to improve pet image generation with customized requirements with efficient training.
Regarding claim 7, the combination of Zhao and Zhang teaches the method according to claim 1, wherein the method further comprises: obtaining an image style selected by the user, ([0009] “a selection module for selecting a training subset of pet features and image styles expected by the user from the training image set”)
and determining a second plug-in corresponding to the image style, ([0009] “a low-rank adapter Lora creation module for creating a feature low-rank adapter Lora and a style low-rank adapter Lora for the pet features and image styles, respectively,”, where “In one embodiment, the feature low-rank adapter Lora and the style low-rank adapter are used as plugins for the stable diffusion SD model” [0033])
wherein the second plug-in is configured to control a style of the target pet image; ([0019] “Image style information describes the overall style and emotional tone of an image.”)
and the image generation model is further configured to: when generating the target pet image, interact with the second plug-in to generate the target pet image. ([0039] “after fusing a subset of pet images corresponding to the required pet features and image style with a trained stable diffusion SD model, a pet image is generated according to the requirements.”)
Zhang is analogous to the claimed invention, as both relate to generating pet images. Zhang further teaches that “[t]his results in a pet image that matches the required pet features and image style, and is particularly good at generating pet images that resemble the pets that the user has raised.” [0039]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Zhang to the combination of Zhao and Zhang in order to effectively generate pet images that are consistent with the required style.
Regarding claim 9, the combination of Zhao and Zhang teaches the method according to claim 1, wherein the to-be-processed pet image is a single pet image. (Zhao; FIG. 8, input image 805, where “For example, input image 805 depicts a corgi.” [0126]).
Regarding claim 10, claim 10 recites substantially similar limitations to claim 1, but in a device form. The combination of Zhao and Yu further teaches an electronic device, (Zhao; [0195] “FIG. 14 shows an example of a computing device 1400 according to aspects of the present disclosure.”)
wherein the device comprises a processor and a memory; (Zhao; [0195] “The example shown includes computing device 1400, processor 1405, memory subsystem 1410”)
and the processor is configured to execute instructions stored in the memory, to enable the device to perform a method for generating a pet image (Zhao; [0196] “In some embodiments, computing device 1400 includes processor 1405 that can execute instructions stored in memory subsystem 1410 to obtain an input image and a text prompt. The instructions further include to generate, using a language generation model, a text response based on the input image and the text prompt, where the text response describes a modification to the input image. The instructions further include to generate, using an image generation model, a synthetic image based on the input image and an output embedding of the language generation model, where the synthetic image depicts the modification to the input image.”).
Regarding claim 15, claim 15 recites substantially similar limitations to claim 6, therefore, rejected under the same rationale as claim 6.
Regarding claim 16, claim 16 recites substantially similar limitations to claim 6, therefore, rejected under the same rationale as claim 7.
Regarding claim 18, claim 18 recites substantially similar limitations to claim 9, therefore, rejected under the same rationale as claim 9.
Regarding claim 19, claim 19 recites substantially similar limitations to claim 1, but in a medium form. The combination of Zhao and Zhang further teaches a non-transitory computer-readable storage medium, wherein the computer-readable storage medium comprises instructions, and the instructions indicate a device to perform a method for generating a pet image (Zhao; [0004] “Aspects for a method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining an input image and a text prompt including an image modification request.”, where “If implemented in software executed by a processor, the functions may be stored in the form of instructions or code on a computer-readable medium.” [0205]).
Regarding claim 20, claim 20 recites substantially similar limitations to claim 2, therefore, rejected under the same rationale.
Claims 2 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Zhang, and further in view of Luo (CN 117351207 A, see English Translation).
Regarding claim 2, the combination of Zhou and Zhang teaches the method according to claim 1, wherein a plug-in compromises: at least one of a group consisting of a pet… portrait (Zhang; [0019] “Image style information includes, but is not limited to, digital portraits and 3D renderings.”, where “In one embodiment, the feature low-rank adapter Lora and the style low-rank adapter are used as plugins for the stable diffusion SD model” [0033], and “creating a feature low-rank adapter Lora and a style low-rank adapter Lora for the pet features and image styles” [0009]). However, the combination of Zhou and Zhang fails to teach wherein the first plug-in comprises: at least one of a group consisting of a pet head portrait plug-in and a pet full body portrait plug-in, wherein the image feature of the to-be-processed pet image comprises at least one of a group consisting of a head portrait feature and a full body portrait feature; the pet head portrait plug-in is configured to process the head portrait feature of the target pet; and the pet full body portrait plug-in is configured to process the full body portrait feature of the target pet. However, this is known in the art as taught by Luo.
Luo teaches wherein the first plug-in comprises: at least one of a group consisting of a pet head portrait plug-in and a pet full body portrait plug-in, ([0041] “Step S202: Input the original image into the pre-trained segmentation model to obtain the head image of the target object.”, where “In some embodiments of this application, the target object can be a pet, such as a kitten, puppy, or piglet. The target object can also be other animals, such as a calf, pony, or lamb.” [0039]. Note: the “pre-trained segmentation model” is mapped to the pet head portrait plug-in.)
wherein the image feature of the to-be-processed pet image comprises at least one of a group consisting of a head portrait feature and a full body portrait feature; ([0039] “Step S203: Based on the shape and color of the head image, select a material image that matches the head image from a preset material library as the body part of the head image.”).
the pet head portrait plug-in is configured to process the head portrait feature of the target pet; and the pet full body portrait plug-in is configured to process the full body portrait feature of the target pet, ([0041] “stitching the head image and the material image together to obtain a stitched image; and generating a target image based on the stitched image and a preset carrier image.”)
Luo is analogous to the claimed invention, as both relate to generating pet images. Luo further teaches that “Users do not need to use other tools to crop the original image to obtain the head image, which reduces the user's time cost and improves the processing efficiency of the original image.” [0054]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Luo into the combination of Zhao and Zhang to efficiently and quickly process the original pet image.
Regarding claim 11, claim 11 recites substantially similar limitations to claim 2, therefore, rejected under the same rationale as claim 2.
Claims 8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Zhang, and further in view of Deng (CN 109448093 A, see English Translation).
Regarding claim 8, the combination of Zhao and Zhang teaches the method according to claim 1, wherein before the inputting the first text into an image generation model, and inputting an image feature of the to-be-processed pet image into a pre- trained first plug-in of the image generation model, the method further comprises: determining that the to-be-processed pet image has the target pet (Zhang; [0023] “If an image contains multiple pets or multiple objects, the system will identify and focus on the target pet object while cropping out the other objects. During this process, various key features of the target pet can also be identified, such as fur color, head, eyes, nose, mouth, torso, and tail.”, which is done in order to “generating attribute description information and image style information for each pet image in the set, wherein the attribute description information is used to characterize at least the key features of the target pet in the pet image” [0015])
Zhang is analogous to the claimed invention, as both relate to generating pet images. Zhang further teaches that “However, there are difficulties in generating pet photos using the diffusion SD model. The inventors noticed that compared to generating photos of humans, there is a wide variety of pets, and even within the same species, there are many different sub-breeds.” [0005]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Zhang into the combination of Zhao and Zhang, as generating images of humans and pets are very different due to the wider variety between pets within the same species.
The combination of Zhao and Zhang fails to teach determining that the to-be-processed pet image does not comprise a human face. However, this is known in the art as taught by Deng.
Deng does not explicitly teach determining that the to-be-processed pet image does not comprise a human face, but does teach determining between human and animal images ([0057] “The recognition model building unit 301 is used to build facial recognition models for humans or various birds and animals, as well as object recognition models for various objects.”), since “Generally, the facial structures and features of humans, birds, and animals each have their own distinctive characteristics… Therefore, in a specific embodiment of the present invention, the facial recognition model is established by creating a facial recognition model based on 72 points on the human face and key facial points of various birds and animals.” [0042]. Deng is also analogous to the claimed invention, as both relates to generating animal images. Therefore, it would be obvious for one of ordinary skill in the art before the effective date of the claimed invention to incorporate the teachings of Deng to the combination of Zhao and Zhang to determine the image to not have a human face as the features between humans and pets are distinctive, similarly to Zhang’s teaching.
Regarding claim 17, claim 17 recites substantially similar limitations to claim 8, therefore, rejected under the same rationale as claim 8.
Allowable Subject Matter
Claims 3-5 and 12-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 3, the closest prior art of Luo teaches wherein the pet head portrait plug-in is obtained by training as follows: training the pet head portrait plug-in by using a first training image, a description text of the first training image, ([0006] “obtaining training data and sample labels corresponding to the training data, wherein the training data is a training image containing the target object; training a preset initial segmentation model”)
wherein the first training image is used as a training label, ([0006] “inputting the training data into a first feature extraction layer of the initial segmentation model to obtain sub-features in the first feature extraction layer, wherein the sub-features include basic features, detail features, spatial features, and semantic features;”, where “the first feature, and the second feature is calculated to obtain the predicted value of the mask of the head of the target object in the training image; and the loss function of the initial segmentation model is calculated based on the predicted value and the sample label;” [0006]).
and the first training image is an image comprising a pet ([0074] “In some embodiments of this application, the training data may be images containing a target object, which may be a pet or other animal.”)
However, Luo fails to teach a first sub-image of the first training image, …and the first sub-image comprises a head image of the first training image or a head segmentation of the first training image.
Furthermore, no prior art of record either alone or in combination teaches the above limitations as a whole. Therefore, claim 3 contains allowable subject matter.
Regarding claim 4, claim 4 recites substantially similar limitations to claim 3, but for “training the pet full body portrait plug-in”. The closest prior art of Luo fails to teach wherein the pet full body portrait plug-in is obtained by training as follows: training the pet full body portrait plug-in by using a second training image, a description text of the second training image, and a second sub-image of the second training image, wherein the second training image is used as a training label, and the second sub-image comprises a full body image of the second training image or a full body segmentation of the second training image.
Furthermore, no prior art of record either alone or in combination teaches the above limitations as a whole. Therefore, claim 4 contains allowable subject matter.
Regarding claim 5, claim 5 recites substantially similar limitations to claim 3, but for training both “the pet head portrait plug-in” and “the pet full body portrait plug-in”. The closest prior art of Luo fails to teach wherein the pet head portrait plug-in and the pet full body portrait plug-in are obtained by training as follows: training the pet head portrait plug-in and the pet full body portrait plug-in by using a third training image, a description text of the third training image, and a third sub-image of the third training image, wherein the third training image is used as a training label, and the third sub-image comprises a full body image of the third training image, a full body segmentation of the third training image, a head image of the third training image, or a head segmentation of the third training image.
Furthermore, no prior art of record either alone or in combination teaches the above limitations as a whole. Therefore, claim 5 contains allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALICIA HA whose telephone number is (571)272-3601. The examiner can normally be reached Mon-Thurs 9:30 AM - 6:30 PM, and Fri 9:30 AM - 1:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/ALICIA HA/Examiner, Art Unit 2611
Prosecution Insights