DETAILED ACTION
Introduction
This office action is in response to Applicant’s amendment filed on March 11, 2026.
No claims have been amended. Claims 2 and 13 have been previously cancelled. Claims 21 and 22 have been previously added. Claims 1, 3-12 and 14-22 are pending in the application. As such, claims 1, 3-12 and 14-22 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings were received on September 15, 2023. These drawings have been accepted and considered by the Examiner.
Response to Amendments and Arguments
No amendments to the claims have been made.
In view of the arguments, the rejections to claims 1, 3-12 and 14-22 under 35 U.S.C. 103 have been maintained.
Applicant’s arguments regarding the prior art rejections under 35 U.S.C 103, received on March 11, 2026, have been fully considered.
Applicant argues: On pages 1-2, "Benedetto is concerned with selection among prompt-language alternatives, not selection from model-generated outputs."
Examiner response: Examiner is not intending to assert that Benedetto teaches the "user selection" that is from the generated output. Examiner is asserting a PHOSITA could combine the "user selection" already obtained by Ciminelli into the process of Benedetto by substituting the "user selection" already obtained by Ciminelli for the "second text prompt" of Benedetto (Fig. 5B, callout 560).
Without modifying the original mapping maintained below, a more detailed mapping and explanation is provided here, for purposes of clarification.
Ciminelli, as modified above, teaches the user selection, which is from the generated output.
Ciminelli, as modified above, does not teach, however Benedetto teaches
modifying the first text prompt based on the [user selection] to obtain a second text prompt (Benedetto in [0065, Fig 5A, Fig 5B] teaches using text prompt aggregator which can be used to aggregate the first text prompt and the second text prompt (user selection) to generate the aggregated user prompt (second text prompt)).
PNG
media_image1.png
726
449
media_image1.png
Greyscale
Examiner’s position is Benedetto teaches generating a first text prompt, receiving a second text prompt from the user, and then generating a third aggregate prompt. Ciminelli teaches the user selecting a portion of the output. It is the combination of the references which renders the claimed element obvious.
Applicant’s arguments with respect to claims 1, 3-12 and 14-22 have been considered, and are not considered to be persuasive. Therefore, the rejections of claims 1, 3-12 and 14-22 have been maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 9-10, 12-14, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ciminelli et al. (US Patent Pub. No. 20240086051 A1, as supported by provisional application 63/444,841 filed 2/10/2023), hereinafter Ciminelli, in view of Sadr et al. (US Patent No. 11941678 B2), hereinafter Sadr, in view of Benedetto et al. (US Patent Pub. No. 20240193351 A1), hereinafter Benedetto.
Regarding claims 1, 12 and 20, Ciminelli teaches a computer-implemented method, a computing system, and a non-transitory processor-readable medium (Ciminelli in [0044, Fig. 1B] teaches using a personal computer to implement the method),
[claim 12 only] a processor; and a memory coupled to the processor, the memory storing computer-executable instructions that, when executed by the processor, are to cause the processor to (Ciminelli in [0064] teaches using a processor, memory, and instructions which are non-transitory):
[claim 20 only] non-transitory processor-readable medium storing processor-executable instructions that, when executed by a processor, are to cause the processor to (Ciminelli in [0064] teaches using a processor, memory, and instructions which are non-transitory):
comprising:
obtaining [a plurality of outputs] of a generative model based on input of a first text prompt (Ciminelli in [0085] teaches using a generative model and generating a plurality of alternative outputs, and in [0005] teaches receiving textual input in a natural language, and in [0069] teaches the textual input is referred to as a prompt);
presenting [the plurality of outputs] via a user interface (Ciminelli in [0085] teaches presenting the plurality of alternative drafts of the output, in the same display screen, in different display screens, simultaneously, one after the other);
receiving, via the user interface, user selection of a desired portion of the [plurality of outputs, wherein the plurality of outputs comprise multiple different outputs generated via the generative model based on a same text prompt] (Ciminelli in [0006] teaches a selection of a portion of the output may be received from the user);
and
providing the second text prompt as input to the generative model for obtaining a second output (Ciminelli in [0006] teaches using the selected portion to indicate a change desired, implementing the desired change, and generating a new output for the desired change, and in [0008] teaches using a second textual input to indicate the desired change).
Ciminelli teaches the generative model which produces generative output.
Ciminelli does not teach, however Sadr teaches
plurality of outputs, wherein the plurality of outputs comprise multiple different outputs generated via [the generative model] based on a same text prompt (Sadr in [col 3 ln 40-67, Fig. 2, Fig. 3M] teaches using a first search query (prompt) to generate a plurality of first search results (responses), providing the plurality of results to the user in a display, obtaining a text input and an image selection form the user, generating a multi-modal prompt (second prompt) for input into an image-generation model to generate a model-generated image, and providing the second search results to the user on a display).
PNG
media_image2.png
509
884
media_image2.png
Greyscale
PNG
media_image3.png
535
887
media_image3.png
Greyscale
Sadr is considered to be analogous to the claimed invention because it is in the same field of image-generation models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli further in view of Sadr to allow for using a first search query (prompt) to generate a plurality of first search results (responses). Motivation to do so would provide for using a plurality of descriptor user interface elements which allow for more detailed prompt generation (Sadr [col 14 ln 10-37]).
Ciminelli, as modified above, does not teach, however Benedetto teaches
modifying the first text prompt based on the user selection to obtain a second text prompt (Benedetto in [0065, Fig 5A, Fig 5B] teaches using text prompt aggregator which can be used to aggregate the first text prompt and the second text prompt (user selection) to generate the aggregated user prompt (second text prompt)).
Benedetto is considered to be analogous to the claimed invention because it is in the same field of image-generation models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, further in view of Benedetto to allow for using a text prompt aggregator to aggregate the first text prompt and the second text prompt to generate the aggregated user prompt. Motivation to do so would provide for a user prompt to be refined by taking into consideration additional keywords (Benedetto [0038]).
Regarding claims 3 and 14, Ciminelli, as modified above, teaches the method and computing system of claims 1 and 12.
Ciminelli further teaches
wherein the generative model comprises one of a text-to-image model or a large language model (LLM) (Ciminelli in [0069] teaches using a Large Language Model (LLM) as a generative language model).
Regarding claims 9 and 18, Ciminelli, as modified above, teaches the method and computing system of claims 1 and 12.
Ciminelli further teaches
further comprising
receiving, via the user interface, input of user edits of the at least one output (Ciminelli in [0006] teaches a user selects a portion of the first output for modification)
and
wherein the second text prompt is obtained by modifying the first text prompt based on the user selection and the user edits (Ciminelli in [0006] teaches a user selects a portion of the first output for modification, and the desired changes are analyzed to determine the desired changes, then the system generates a new second output incorporating the desired changes).
Regarding claim 10, Ciminelli, as modified above, teaches the method of claim 9.
Ciminelli further teaches
wherein the user edits comprise at least one of:
deletion of a portion of an output;
replacement of a portion of an output (Ciminelli in [0203] teaches at least part of the base version of the design may be replaced and/or modified based on the mathematical object to generate the new version of the design);
or
addition of text or image (Ciminelli in [0029] teaches user inputs may be text fields and/or image carousels).
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ciminelli, in view of Sadr, in view of Benedetto, in view of Mathewson et al. (US Patent Pub. No. 20240184982 A1, as supported by provisional application 63/429,532 filed 12/1/2022), hereinafter Mathewson.
Regarding claims 4 and 15, Ciminelli, as modified above, teaches the method and computing system of claims 1 and 12.
Ciminelli, as modified above, does not teach, however Mathewson teaches
wherein receiving the user selection of a desired portion comprises:
performing text processing of a text output for obtaining a list of one or more tokens (Mathewson in [0032] teaches using a language model neural network to generate a sequence of tokens);
presenting the one or more tokens via the user interface (Mathewson in [0083] teaches using a user interface for presenting the current output sequence for editing);
and
receiving selection of at least one of the one or more tokens (Mathewson in [0083] teaches presenting the current output sequence for editing, and allowing the user to add, modify, or delete text tokens).
Mathewson is considered to be analogous to the claimed invention because it is in the same field of LLMs. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, further in view of Mathewson to allow for the user to add, modify, or delete text tokens. Motivation to do so would allow for performing generation steps in an interactive manner which allows the user to refine the final content of the textual work without requiring the user to compose the actual textual work (Mathewson [0052]).
Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ciminelli, in view of Sadr, in view of Benedetto, in view of Cohen et al. (US Patent Pub. No. 20240169502 A1), hereinafter Cohen.
Regarding claims 5 and 16, Ciminelli, as modified above, teaches the method and computing system of claims 1 and 12.
Ciminelli, as modified above, does not teach, however Cohen teaches
wherein receiving the user selection of a desired portion comprises:
performing object detection of an image output for identifying one or more objects (Cohen in [0085, Fig. 2] teaches identification of various objects in an image);
graphically representing the one or more objects via the user interface (Cohen in [0205, Fig. 8C] teaches selecting an object and moving it to a different position in the image (moving the graphical representation);
and
receiving selection of at least one of the one or more objects (Cohen in [0091, Fig. 2] teaches detecting a user selection of an object in an image).
Cohen is considered to be analogous to the claimed invention because it is in the same field of digital image object selection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, further in view of Cohen to allow for selecting an object and moving it to a different position in an image. Motivation to do so would allow for flexible and intuitive editing of digital images while efficiently reducing the user interactions typically required to make such edits (Cohen [0002]).
Claims 6-8, 17 and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Ciminelli, in view of Sadr, in view of Benedetto, in view of Quek et al. (US Patent Pub. No. 20090015869 A1), hereinafter Quek.
Regarding claims 6 and 17, Ciminelli, as modified above, teaches the method and computing system of claims 1 and 12.
Ciminelli, as modified above, teaches in claim 1 the user interface, and graphically representing the user selection.
Ciminelli, as modified above, does not teach, however Quek teaches
further comprising
displaying, via the user interface, a sandbox region for graphically representing the user selection (Quek in [0085] teaches using a region for displaying multiple images which have been dragged and dropped there),
wherein the sandbox region is dynamically updated based on selections of desired portions across multiple different outputs (Quek in [0085] teaches using a region for displaying multiple images which have been dragged and dropped there, and also teaches that the style of the interface can be adjusted, in addition to the addition of new images (this maps to dynamic)).
Quek is considered to be analogous to the claimed invention because it is in the same field of digital image object selection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, further in view of Quek to allow for selecting an object and moving it to a different interface. Motivation to do so would allow for intuitive visual abstractions and mechanisms for efficiently and quickly creating image collages (Quek [0027]).
Regarding claim 7, Ciminelli, as modified above, teaches the method of claim 6.
Ciminelli, as modified above, does not teach, however Quek teaches
further comprising:
receiving, via the user interface, input for changing a property of a selected desired portion of the at least one output in the sandbox region (Quek in [0087] teaches the user can use a computer I/O device to adjust the positions, the orientations, or the sizes of the images);
and
updating the user interface to represent the inputted change of the property (Quek in [0087] teaches user can also add a border to the image, or enhance the colors of the image by change contrast, color saturation, or tones (e.g. from color to sepia or black-and-white) [these changes correspond to updating]).
Quek is considered to be analogous to the claimed invention because it is in the same field of digital image object selection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, as modified above, further in view of Quek to allow for adjust the positions, the orientations, or the sizes of the images. Motivation to do so would allow for intuitive visual abstractions and mechanisms for efficiently and quickly creating image collages (Quek [0027]).
Regarding claim 8, Ciminelli, as modified above, teaches the method of claim 7.
Ciminelli, as modified above, does not teach, however Quek teaches
wherein the property of the selected desired portion comprises one of
location (Quek in [0087] teaches the user can use a computer I/O device to adjust the positions, the orientations, or the sizes of the images),
scale (Quek in [0087] teaches the user can use a computer I/O device to adjust the positions, the orientations, or the sizes of the images),
color (Quek in [0087] teaches user can also add a border to the image, or enhance the colors of the image by change contrast, color saturation, or tones (e.g. from color to sepia or black-and-white),
or
language.
Quek is considered to be analogous to the claimed invention because it is in the same field of digital image object selection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, further in view of Quek to allow for adjust the positions, the colors, or the sizes of the images. Motivation to do so would allow for intuitive visual abstractions and mechanisms for efficiently and quickly creating image collages (Quek [0027]).
Regarding claims 21 and 22, Ciminelli, as modified above, teaches the method and computing system of claims 6 and 17.
Ciminelli, as modified above, teaches in claims 6 and 17 above, the sandbox region and using the sandbox region for displaying multiple images which have been dragged and dropped there.
Ciminelli, as modified above, does not teach, however Sadr teaches
wherein the [sandbox region] is progressively populated responsive to selections of preferred content portions across the multiple different outputs that are received via the user interface (Sadr in [col 3 ln 40-67] teaches using a first search query (prompt) to generate a plurality of first search results (responses), providing the plurality of results to the user in a display, obtaining a text input and an image selection form the user, generating a multi-modal prompt (second prompt) for input into an image-generation model to generate a model-generated image, and providing the second search results to the user on a display, and in [col 3 ln 37-53] teaches FIG. 10 depicts an illustration of an example user collection interface according to example embodiments of the present disclosure. For example, the user collection interface can provide a landing experience that provides various inspiration bucket and personalization methods. In particular, a “recommended for you” panel can be provided in the interface that may be determined based on previous creations, based on preferences, and/or based on historical data (e.g., purchase history, search history, and/or browsing history). The user collection interface can include a dream closet that displays a plurality of previous model-generated datasets. The user may interact with a preferences interface to select varying preferences that can be utilized for further “recommended for you” suggestions, which can include automated prompt generation and processing for automatically generated model-generated datasets).
PNG
media_image4.png
642
895
media_image4.png
Greyscale
Sadr is considered to be analogous to the claimed invention because it is in the same field of image-generation models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, further in view of Sadr to allow for a user to make multiple selections for further processing and analysis. Motivation to do so would provide for using a plurality of descriptor user interface elements which allow for more detailed prompt generation (Sadr [col 14 ln 10-37]).
Claims 11 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ciminelli, in view of Sadr, in view of Benedetto, in view of Shlens et al. (US Patent No. 10535164 B2), hereinafter Shlens.
Regarding claims 11 and 19, Ciminelli, as modified above, teaches the method and computing system of claims 1 and 12.
Ciminelli, as modified above, teaches in claim 1 the second text prompt, and providing it as input to the generative model for obtaining the second output.
Ciminelli, as modified above, does not teach, however Shlens teaches
further comprising:
receiving user input of an adherence weight value representing a desired level of adherence to the user selection (Shlens in [col 3 ln 10-17] teaches allowing a user to select a combination of multiple image styles and to specify a weight that should be applied to each),
wherein [the second text prompt and] the adherence weight value are provided as input [to the generative model for obtaining the second output] (Shlens in [col 3 ln 10-17] teaches allowing a user to select a combination of multiple image styles and to specify a weight that should be applied to each).
Shlens is considered to be analogous to the claimed invention because it is in the same field of digital image modification. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ciminelli, as modified above, further in view of Shlens to allow for specify weights that should be applied to each modification desired. Motivation to do so would allow for an output image which can be provided to a user more efficiently i.e. in a shorter amount of time, providing other related advantages such as a reduced screen-on time, with further benefits in terms of the power consumption of the device (Shlens [col 2 ln 12-26]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL J. MUELLER whose telephone number is (571)272-1875. The examiner can normally be reached M-F 9:00am-5:00pm (Eastern).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel C. Washburn can be reached at 571-272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
PAUL MUELLER
Examiner
Art Unit 2657
/PAUL J. MUELLER/Examiner, Art Unit 2657
/DANIEL C WASHBURN/Supervisory Patent Examiner, Art Unit 2657
Prosecution Insights