Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 5/13/2026 have been fully considered but they are not persuasive.
Claim(s) 1, 3-6, 8, 10-13, 15 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chung et al., PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions, arXiv, August 2023 (hereinafter “Chung”) in view of Aggarwal et al. (U.S. Patent Application Publication 2024/0404144 A1, hereinafter “Aggarwal”).
For independent claims 1, 8 and 15, Applicants argue the references fail to disclose receiving a selection of colors displayed in the one or more color interface elements and generating a set of images that each predominantly features the set of colors corresponding to the selection of colors. Examiner respectfully disagrees.
In particular, Chung discloses the user interface includes a palette interface for displaying color interface elements corresponding to selectable colors, e.g. prompt color circles corresponding to ivory, red and brown colors for selection, and further disclosing each prompt has its own color (page 2/Fig. 1; and pages 5-6/Fig. 5). Chung discloses the palette interface receives user input of a selection of colors displayed in the color interface elements, e.g. selection of ivory, red and brown colors from selection of the corresponding prompt circles (page 2/Fig. 1; and pages 5-6/Fig. 5). Chung discloses application of a diffusion model as a generative image model to the generative image text prompt and to colors corresponding to the selection of colors displayed in the color interface elements to generate images that predominantly feature the colors (page 2/Fig. 1; and pages 5-6/Figs. 5-6): “In the language of prompts-as-color, a brush could be loaded with an ‘ocean’ color and applied to the background to be followed by the targeted application of the ‘boat’-color to certain areas.” Thus, Chung discloses an example where a palette interface may be used to select an ‘ocean’ color and a ‘boat’ color to be predominantly applied in the generation of a set of images.
Therefore, the combination of Chung and Aggarwal discloses the limitations of claims 1, 8 and 15.
For dependent claims 5, 12 and 19, Applicants argue the references fail to disclose determining the plurality of selectable colors based on colors used by the user in other images generated by the user within digital canvas. Examiner respectfully disagrees.
In particular, Chung discloses the colors corresponding to the color interface elements are selected based on colors used by the user in previously generating images within the digital canvas (pages 7 and 12): “First, Prompt-Paint visualizes the prompts used in previous generations as paths in the palette interface as in Figure 1c-1.” “The interface showing all previous generations (in the prompt palette in Figure 1c) could help users understand the previous generations they have tried (N=3) and iterate on the prompt intervention.”).
Therefore, the combination of Chung and Aggarwal discloses the limitations of claims 5, 12 and 19.
For the remaining claims, Applicants argue for their allowance based on their dependence to independent claims 1, 8 and 15. It follows the remaining claims are rejected for the reasons discussed above and in the following Detailed Action.
DETAILED ACTION
Claim Objections
Claims 2, 9 and 16 are objected to because of the following informalities:
For claim 2, Examiner believes this claim should be amended in the following manner:
The method of claim 1, wherein, for each image of the set of images, that image includes an above-threshold percentage of pixels within [[the]] that image that includes (i) the set of colors or (ii) colors within a threshold similarity to one or more colors of the set of colors.
For claim 9, Examiner believes this claim should be amended in the following manner:
The non-transitory computer-readable storage medium of claim 8, wherein, for each image of the set of images, that image includes an above-threshold percentage of pixels within [[the]] that image that includes (i) the set of colors or (ii) colors within a threshold similarity to one or more colors of the set of colors.
For claim 16, Examiner believes this claim should be amended in the following manner:
The system of claim 15, wherein, for each image of the set of images, that image includes an above-threshold percentage of pixels within [[the]] that image that includes (i) the set of colors or (ii) colors within a threshold similarity to one or more colors of the set of colors.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 9 and 16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
For dependent claim 2, this establishes “each image” and “that image”. Claim 2 goes on to recite the phrase “the image” and it is unclear and ambiguous to which of the previously established “each image” and “that image” is being referenced by the phrase “the image”. Examiner has suggested amendments in the claim objections discussed above to resolve the ambiguities.
For dependent claim 9, this establishes “each image” and “that image”. Claim 9 goes on to recite the phrase “the image” and it is unclear and ambiguous to which of the previously established “each image” and “that image” is being referenced by the phrase “the image”. Examiner has suggested amendments in the claim objections discussed above to resolve the ambiguities.
For dependent claim 16, this establishes “each image” and “that image”. Claim 16 goes on to recite the phrase “the image” and it is unclear and ambiguous to which of the previously established “each image” and “that image” is being referenced by the phrase “the image”. Examiner has suggested amendments in the claim objections discussed above to resolve the ambiguities.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3-6, 8, 10-13, 15 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chung et al., PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions, arXiv, August 2023 (hereinafter “Chung”) in view of Aggarwal et al. (U.S. Patent Application Publication 2024/0404144 A1, hereinafter “Aggarwal”).
For claim 1, Chung discloses a method (page 1) comprising: accessing a generative image text prompt based on an input from a user (disclosing a user interface to access a generative image text prompt from user input (page 2/Fig. 1; and page 5)); displaying, within a palette interface, one or more color interface elements comprising a plurality of selectable colors (disclosing the user interface includes a palette interface for displaying color interface elements corresponding to selectable colors, e.g. prompt color circles corresponding to ivory, red and brown colors for selection, and further disclosing each prompt has its own color (page 2/Fig. 1; and pages 5-6/Fig. 5)); receiving, from the user, a selection of colors displayed in the one or more color interface elements (disclosing the palette interface receives user input of a selection of colors displayed in the color interface elements, e.g. selection of ivory, red and brown colors from selection of the corresponding prompt circles (page 2/Fig. 1; and pages 5-6/Fig. 5)); applying a generative image model to the generative image text prompt and to a set of colors corresponding to the selection of colors displayed in the one or more color interface elements to generate a set of images that each predominantly features the set of colors (disclosing application of a diffusion model as a generative image model to the generative image text prompt and to colors corresponding to the selection of colors displayed in the color interface elements to generate images that predominantly feature the colors (page 2/Fig. 1; and pages 5-6/Figs. 5-6): “In the language of prompts-as-color, a brush could be loaded with an ‘ocean’ color and applied to the background to be followed by the targeted application of the ‘boat’-color to certain areas.”); and displaying the set of images within a digital canvas to the user (disclosing the user interface displays the generated images within a digital canvas to the user (page 2/Fig. 1; and pages 5-6/Fig. 6)).
Chung does not specifically disclose an image generation system.
However, these limitations are well-known in the art as disclosed in Aggarwal.
Aggarwal similarly discloses a method for image processing through image generation (par. 3). Aggarwal explains the method may be implemented with a system to perform image generation as a computing device (Fig. 15; par. 3 and 154-155). It follows Chung may be accordingly modified with the teachings of Aggarwal to implement its method with an image generation system.
A person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention would find it obvious to modify Chung with the teachings of Aggarwal. Aggarwal is analogous art in dealing with a method for image processing through image generation (par. 3). Aggarwal discloses its use of a computing device is advantageous in appropriately implementing an image generation system to perform a method of image generation (Fig. 15; par. 3 and 154-155). Consequently, a PHOSITA would incorporate the teachings of Aggarwal into Chung for appropriately implementing an image generation system to perform a method of image generation. Therefore, claim 1 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
For claim 3, depending on claim 1, Chung as modified by Aggarwal discloses wherein each image of the set of images includes a different visual style (Chung discloses each image includes a different visual style, e.g. cubist, surrealism, impressionism, cyberpunk and etc. (pages 5-6/Fig. 6 and page 8/Table 1)).
For claim 4, depending on claim 1, Chung as modified by Aggarwal discloses, further comprising, prior to displaying the one or more color interface elements, determining the plurality of selectable colors of the one or more color interface elements based on a subject matter associated with the generative image text prompt (Chung discloses the colors corresponding to the color interface elements are selected based on subject matter associated with the generative image text prompt, e.g. prompts such as “ocean” and “boat” are associated with corresponding colors for image generation (pages 5-6/Fig. 5)).
For claim 5, depending on claim 1, Chung as modified by Aggarwal discloses further comprising, prior to displaying the one or more color interface elements determining wherein the plurality of selectable colors of the one or more color interface elements based on colors used by the user in other images generated by the user within the digital canvas (Chung discloses the colors corresponding to the color interface elements are selected based on colors used by the user in previously generating images within the digital canvas (pages 7 and 12): “First, Prompt-Paint visualizes the prompts used in previous generations as paths in the palette interface as in Figure 1c-1.” “The interface showing all previous generations (in the prompt palette in Figure 1c) could help users understand the previous generations they have tried (N=3) and iterate on the prompt intervention.”).
For claim 6, depending on claim 1, Chung as modified by Aggarwal discloses wherein the generative image text prompt is entered by the user into a text prompt interface element (Chung discloses a + button as a text prompt interface element in its user interface so that the user enters the generative image text prompt (page 2/Fig. 1 and page 5)).
For claim 8, Chung as modified by Aggarwal discloses a non-transitory computer-readable storage medium storing executable instructions that, when executed by a hardware processor (Aggarwal similarly discloses a method for image processing through image generation (par. 3); Aggarwal explains the method may be implemented with a system to perform image generation as a computing device (Fig. 15; par. 3 and 154-155); Aggarwal further explains the computing device includes memory for storing instructions for execution by a hardware processor to perform the functions of the computing device (Fig. 15; par. 156); and it follows Chung may be accordingly modified with the teachings of Aggarwal to implement its method with an image generation system), cause the hardware processor to perform steps corresponding to the method of claim 1 (see above as to claim 1).
For claim 10, depending on claim 8, this claim is a combination of the limitations of claim 8 and claim 3. It follows claim 10 is rejected for the same reasons as to claim 8 and claim 3.
For claim 11, depending on claim 8, this claim is a combination of the limitations of claim 8 and claim 4. It follows claim 11 is rejected for the same reasons as to claim 8 and claim 4.
For claim 12, depending on claim 8, this claim is a combination of the limitations of claim 8 and claim 5. It follows claim 12 is rejected for the same reasons as to claim 8 and claim 5.
For claim 13, depending on claim 8, this claim is a combination of the limitations of claim 8 and claim 6. It follows claim 13 is rejected for the same reasons as to claim 8 and claim 6.
For claim 15, Chung as modified by Aggarwal discloses a system comprising: a hardware processor; and a non-transitory computer-readable storage medium storing executable instructions that, when executed by the hardware processor (Aggarwal similarly discloses a method for image processing through image generation (par. 3); Aggarwal explains the method may be implemented with a system to perform image generation as a computing device (Fig. 15; par. 3 and 154-155); Aggarwal further explains the computing device includes a hardware processor and memory for storing instructions for execution by the hardware processor to perform the functions of the computing device (Fig. 15; par. 156); and it follows Chung may be accordingly modified with the teachings of Aggarwal to implement its method with an image generation system), cause the hardware processor to perform steps corresponding to the method of claim 1 (see above as to claim 1).
For claim 17, depending on claim 15, this claim is a combination of the limitations of claim 15 and claim 3. It follows claim 17 is rejected for the same reasons as to claim 15 and claim 3.
For claim 18, depending on claim 15, this claim is a combination of the limitations of claim 15 and claim 4. It follows claim 18 is rejected for the same reasons as to claim 15 and claim 4.
For claim 19, depending on claim 15, this claim is a combination of the limitations of claim 15 and claim 5. It follows claim 19 is rejected for the same reasons as to claim 15 and claim 5.
Claim(s) 7, 14 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chung in view of Aggarwal further in view of Feng et al., PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation, arXiv, August 2023 (hereinafter “Feng”).
For claim 7, depending on claim 1, Chung as modified by Aggarwal does not disclose a generative image text prompt is determined based on one or more images selected by a user.
However, these limitations are well-known in the art as disclosed in Feng.
Feng similarly discloses a system and method for image generation based on a generative image text prompt (page 1). Feng explains its generative image text prompt may be determined based on images selected by a user (pages 6-7/Fig. 4). It follows Chung and Aggarwal may be accordingly modified with the teachings of Feng to determine its generative image text prompt based one or more images selected by its user.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Chung and Aggarwal with the teachings of Feng. Feng is analogous art in dealing with a system and method for image generation based on a generative image text prompt (page 1). Feng discloses user selection of images is advantageous in facilitating prompt engineering to appropriately refine a generative image text prompt (page 1; and pages 6-7/Fig. 4). Consequently, a PHOSITA would incorporate the teachings of Feng into Chung and Aggarwal for facilitating prompt engineering to appropriately refine a generative image text prompt. Therefore, claim 7 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
For claim 14, depending on claim 8, this claim is a combination of the limitations of claim 8 and claim 7. It follows claim 14 is rejected for the same reasons as to claim 8 and claim 7.
For claim 20, depending on claim 15, this claim is a combination of the limitations of claim 15 and claims 6 and 7. It follows claim 20 is rejected for the same reasons as to claim 15 and claims 6 and 7.
Allowable Subject Matter
Claims 2, 9 and 16 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims, and to address the claim objections raised above in the Detailed Action.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHARLES TSENG whose telephone number is (571)270-3857. The examiner can normally be reached 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHARLES TSENG/ Primary Examiner, Art Unit 2613
Prosecution Insights