DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see pages 11-13, filed 11 June 2026, with respect to the rejection(s) of claim(s) 1 and similar claims in substance under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Stable Diffusion Art (“Inpainting: A complete guide”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 9, 10, 15, 16, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Costin et al. (US 2024/0135611 A1) in view of Stable Diffusion Art (“Inpainting: A complete guide”) archived December 9, 2023.
Regarding claim 1, Costin discloses a method implemented by one or more processors (Paragraph 0089, processor), the method comprising: receiving user input associated with a client device of a user (Paragraph 0030, user device), the user input including visual content (Paragraph 0047, initial description from a user to generate an original image), and the user input including a request to modify the visual content; (Paragraph 0049, user description on an object to be introduced into the original image for generating a scene image) determining visual content editing instructions for the visual content (Paragraph 0049, user description on the object to be introduced into the original image for generating the scene image) and at least one bounding box associated with a portion of the visual content that is to be modified (Paragraph 0077, bounding box for placing the object into the image), wherein determining the visual content editing instructions and the at least one bounding box associated with the portion of the visual content that is to be modified comprises: processing, using a generative model (GM), GM input to generate GM output, the GM input including at least the user input and the visual content (Paragraphs 0057-0058, diffusion models for editing images using text guidance). Costin does not clearly disclose determining, based on the GM output, the visual content editing instructions for the visual content and the at least one bounding box associated with the portion of the visual content that is to be modified; generating a modified version of the visual content that is responsive to the user input, wherein generating the modified version of the visual content comprises: processing, using an additional GM that is in addition to the GM, additional GM input to generate additional GM output, the additional GM input including at least the visual content, the visual content editing instructions for the visual content, and the at least one bounding box associated with the portion of the visual content that is to be modified; and determining based on the additional GM output, the modified version of the visual content, the modified version of the visual content including a modified portion of the visual content within the bounding box associated with the portion of the visual content. Stable Diffusion Art discloses determining, based on the GM output (A basic example of inpainting, Step-by-step workflow: txt2img generates an image using a generative model that can be edited because a generated face is too small), the visual content editing instructions for the visual content and the at least one bounding box associated with the portion of the visual content that is to be modified; (A basic example of inpainting, Step-by-step workflow: because the face is too small, a mask can be created for the face area in the generated image as an area to be edited using a prompt, see Inpainting settings explained, Prompt) generating a modified version of the visual content that is responsive to the user input (Inpainting settings explained, Prompt: generation of a desired consistent generic face), wherein generating the modified version of the visual content comprises: processing, using an additional GM that is in addition to the GM, additional GM input to generate additional GM output, the additional GM input including at least the visual content, the visual content editing instructions for the visual content, and the at least one bounding box associated with the portion of the visual content that is to be modified; (Inpainting models: other Stable Diffusion models that can be used for inpainting the mask area to add the generated face, see A basic example of inpainting, Step-by-step workflow) and determining based on the additional GM output, the modified version of the visual content, the modified version of the visual content including a modified portion of the visual content within the bounding box associated with the portion of the visual content (See images in the document illustrating various generated faces for the mask area).
Stable Diffusion Art’s technique of generating a face with a separate model to inpaint a masked area in a generated image with another model would have been recognized by one of ordinary skill in the art to be applicable to the generation of an image and inserting objects into the image using a bounding box of Costin and the results would have been predictable in the generation of objects with one model to inpaint a masked area of a generated image of another model defined by a bounding box. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 2, Costin discloses wherein the visual content includes at least image content (Paragraph 0047, original image).
Regarding claim 3, Costin discloses determining that the request to modify the visual content is a request to modify one or more portions of the image content; (Paragraph 0049, user changes to the original image) and in response to determining that the request is a request to modify one or more of the portions of the image content: determining at least one image seed for the image content that preserves one or more additional portions of the image content that the user did not request be modified (Paragraph 0040, seed value for helping retain features of the previously generated image).
Regarding claim 4, Stable Diffusion Art discloses wherein the additional GM input further includes the at least one image seed, and wherein the additional GM output preserves the one or more additional portions of the image content that the user did not request be modified based on processing the additional GM input that further includes the at least one image seed (See image for inpainting settings where a seed of -1 is used which is only used to inpaint the areas masked off while the remaining areas of the image remains unchanged).
Regarding claim 5, Costin discloses wherein the at least one image seed is a corresponding lower-level representation of the image content (Paragraph 0040, the seed is a numerical value used for the random generation of images).
Regarding claim 9, Costin discloses determining that the request to modify the visual content is a request to add textual content that is related to the image content; (Paragraphs 0042 and 0071, user manipulation to add characters into the scene image) and in response to determining that the request is a request to add textual content that is related to the image content: determining at least one image seed for the image content that preserves the image content (Paragraph 0081, storing the seed to retain characteristics of the original image when manipulations are applied to other portions of the image).
Regarding claim 10, Costin discloses wherein the additional GM input further includes the at least one image seed, and wherein the additional GM output preserves the image content based on processing the additional GM input that further includes the at least one image seed (See image for inpainting settings where a seed of -1 is used which is only used to inpaint the areas masked off while the remaining areas of the image remains unchanged).
Regarding claim 15, Costin discloses prior to processing the additional GM input to generate the additional GM output and using the additional GM: determining at least one seed for a portion of the visual content based on the request included in the user input, wherein the GM input further includes the one or more seeds for the visual content as the visual content for the GM input (Costin, paragraphs 0039-0040, regenerating images using stored seed values and Stable Diffusion Art, see image for inpainting settings where a seed of -1 is used which is only used to inpaint the areas masked off while the remaining areas of the image remains unchanged).
Regarding claim 16, Costin in view of Stable Diffusion Art discloses wherein processing the additional GM input to generate the additional GM output and using the additional GM comprises: updating, in a learned embedding space, the at least one seed based on the request included in the user input; (Costin, paragraph 0079, user manipulation of the seed to change the image that is regenerated) and processing, using image generation capabilities of the additional GM or video generation capabilities of the additional GM, and based on updating the at least one seed in the learned embedding space, the modified version of the visual content as the GM output (Stable Diffusion Art, see image for inpainting settings where the seed can be manipulated by the user is used which is only used to inpaint the areas masked off).
Regarding claim 18, Costin in view of Stable Diffusion Art discloses wherein processing the additional GM input to generate the additional GM output and using the additional GM comprises: processing, using image generation capabilities of the additional GM or video generation capabilities of the GM, the visual content editing instructions to generate a modified portion of the visual content, (Costin, paragraphs 0049 and 0077, the object described by the user is added into the scene using the bounding box that can be inpainted into the image, see Stable Diffusion Art).
Regarding claims 19 and 20, similar reasoning as discussed in claim 1 is applied. Furthermore, Costin discloses at least one processor; and memory storing instructions executed by the at least one processor (Paragraph 0194, processors executing instructions stored in memory).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Costin et al. (US 2024/0135611 A1) in view of Stable Diffusion Art (“Inpainting: A complete guide”) archived December 9, 2023 and further in view of Chen et al. (US 2025/0265829 A1).
Regarding claim 6, Costin in view of Stable Diffusion Art discloses all limitations as discussed in claim 5. Costin in view of Stable Diffusion Art does not clearly disclose wherein the at least one image seed is a corresponding image embedding in a learned embedding space.
Chen discloses encoding a seed into an embedding that can be used to reconstruct a synthetic image from the embedding (Paragraph 0056).
Chen’s technique of encoding a seed into an embedding that can be used to reconstruct a synthetic image from the embedding would have been recognized by one of ordinary skill in the art to be applicable to the seed used for repeated image generation with modifications of Costin in view of Stable Diffusion Art and the results would have been predictable in the encoding of a seed used for repeated image generation with modifications into an embedding that can be used to reconstruct an image. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 7 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Costin et al. (US 2024/0135611 A1) in view of Stable Diffusion Art (“Inpainting: A complete guide”) archived December 9, 2023 and further in view of Edson (US 2024/0273796 A1).
Regarding claim 7, Costin in view of Stable Diffusion Art discloses all limitations as discussed in claim 2. Costin further discloses determining the request to modify the visual content; (Paragraph 0049, various types of manipulations can be made to the original image based on the user’s input) and in response to determining the request is a request to modify one or more of the portions of the image content: determining at least one image seed for the image content that preserves one or more additional portions of the image content that the user did not request be animated (Paragraph 0081, storing the seed value to retain characteristics in the image during regeneration with the manipulations). Costin in view of Stable Diffusion Art does not clearly disclose the request is a request to animate one or more portions of the image content; and in response to determining that the request is a request to animate one or more of the portions of the image content: determining at least one image seed for the image content that preserves one or more additional portions of the image content that the user did not request be animated. Edson discloses modifying an image by animating portions of the image (Paragraph 0073). Edson’s technique of modifying an image by animating portions of the image would have been recognized by one of ordinary skill in the art to be applicable to the generative model for regenerating an image from an original image using a stored seed with applied manipulations according to user input of Costin in view of Stable Diffusion Art and the results would have been predictable in the regeneration of an image from an original using a stored seed with animation of portions of the original image while retaining some characteristics of the original image. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 8, Stable Diffusion Art discloses wherein the additional GM input further includes the at least one image seed, and wherein the additional GM output preserves the one or more additional portions of the image content that the user did not request be animated based on processing the additional GM input that further includes the at least one image seed (See image for inpainting settings where a seed of -1 is used which is only used to inpaint the areas masked off while the remaining areas of the image remains unchanged).
Claim(s) 11 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Costin et al. (US 2024/0135611 A1) in view of Stable Diffusion Art (“Inpainting: A complete guide”) archived December 9, 2023 and further in view of Mann et al. (US 2024/0193890 A1).
Regarding claim 11, Costin in view of Stable Diffusion Art discloses all limitations as discussed in claim 1. Costin further discloses determining the request to modify the visual content; (Paragraph 0049, various types of manipulations can be made to the original image based on the user’s input) and in response to determining the request is a request to modify one or more of the portions of the image content: determining at least one image seed for the image content that preserves the image content (Paragraph 0081, storing the seed value to retain characteristics in the image during regeneration with the manipulations). Costin in view of Stable Diffusion Art does not clearly disclose the request is a request to add video content that is related to the image content; and in response to determining that the request is a request to add video content that is related to the image content: determining at least one image seed for the image content that preserves the image content. Mann discloses blending video frames with image frames (Paragraph 0068). Mann’s technique of blending video frames with image frames would have been recognized by one of ordinary skill in the art to be applicable to the introduction of objects into images of Costin in view of Stable Diffusion Art and the results would have been predictable in the introduction and blending of video frames with portions of the original image using a stored seed that can retain some characteristics of the original image. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 12, Stable Diffusion Art discloses wherein the additional GM input further includes the at least one image seed, and wherein the additional GM output preserves the image content based on processing the additional GM input that further includes the at least one image seed (See image for inpainting settings where a seed of -1 is used which is only used to inpaint the areas masked off while the remaining areas of the image remains unchanged).
Claim(s) 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Costin et al. (US 2024/0135611 A1) in view of Stable Diffusion Art (“Inpainting: A complete guide”) archived December 9, 2023 and further in view of in view of Tsui et al. (US 2025/0278874 A1).
Regarding claim 13, Costin in view of Stable Diffusion Art discloses all limitations as discussed in claim 2. Costin further discloses determining the request to modify the visual content; (Paragraph 0049, various types of manipulations can be made to the original image based on the user’s input) and in response to determining the request is a request to modify one or more of the portions of the image content: determining at least one image seed for the image content that preserves the image content (Paragraph 0081, storing the seed value to retain characteristics in the image during regeneration with the manipulations). Costin in view of Stable Diffusion Art does not clearly disclose the request is a request to add audible content that is related to the image content. Tsui discloses generation of a combination of image and audio content (Paragraph 0046). Tsui’s technique of generating a combination of image and audio content would have been recognized by one of ordinary skill in the art to be applicable to the regeneration of an image with modifications using a stored seed that can retain some characteristics of the original image of Costin in view of Stable Diffusion Art and the results would have been predictable in the regeneration of the image that is combined with audio data using a stored seed that can retain some characteristics of the image. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 14, Stable Diffusion Art discloses wherein the additional GM input further includes the at least one image seed, and wherein the additional GM output preserves the image content based on processing the additional GM input that further includes the at least one image seed (See image for inpainting settings where a seed of -1 is used which is only used to inpaint the areas masked off while the remaining areas of the image remains unchanged).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Boyd et al. (US 2026/0187680 A1) discloses generative models for inpainting masked areas defined by bounding boxes.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHI HOANG whose telephone number is (571)270-3417. The examiner can normally be reached Mon-Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHI HOANG/Primary Examiner, Art Unit 2619
Prosecution Insights