DETAILED ACTION
Continued Examination under 37 C.F.R. § 1.114
A request for continued examination under 37 C.F.R. § 1.114, including the fee set forth in 37 C.F.R. § 1.17(e), was filed in this application after allowance. Since this application is eligible for continued examination under 37 C.F.R. § 1.114, and the fee set forth in 37 C.F.R. § 1.17(e) has been timely paid, prosecution in this application has been reopened pursuant to 37 C.F.R. § 1.114.
Status of the Claims
Claims 7 and 19 have been canceled.
Claims 1-6, 8-18, and 20 are pending.
Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 10-12 are rejected under 35 U.S.C. § 103 as being unpatentable over Mansour et al., US 2025/0005528 A1, in view of Liu et al., US 2024/0338862 A1.
Regarding claim 10, Mansour discloses a method implemented using one or more processors, comprising:
Processing a first large language model (LLM) prompt using an LLM to generate a first LLM response. Mansour ¶¶ 258-259.
Providing the first LLM response to a client application, wherein the first LLM response is operable by the client application to provide first rendered LLM output. Mansour ¶¶ 259, 261, fig. 6B.
Receiving, from the client application: an indication of a subportion of the first rendered LLM output that has been selected using one or more input devices, and a request for a modified version of the selected subportion of the first rendered LLM output. Mansour ¶ 285, fig. 10A.
Extracting a subportion of the first LLM response that corresponds to the selected subportion of the first rendered LLM output. Mansour ¶ 286.
Assembling, as a second LLM prompt, the selected subportion of the first LLM response with data indicative of the request to modify the selected subportion of the first rendered LLM output. Mansour ¶ 286.
Processing the second LLM prompt using the same LLM or a different LLM to generate a second LLM response. Mansour ¶ 286.
Providing the second LLM response to the client application, wherein the second LLM response is operable by the client application to provide second rendered LLM output that includes at least a part of the first rendered LLM output outside of the selected subportion of the first rendered LLM output and the modified version of the selected subportion of the first rendered LLM output. Mansour ¶ 287, fig. 10B.
Wherein the first LLM response comprises metadata instructions for rendering one or more images, and the selected subportion of the first rendered LLM output comprises one or more rendered images. Mansour teaches that an LLM may generate images. Mansour ¶ 82. Mansour teaches the generative output may include formatting instructions, embedded commands, and calls to retrieve content. Id. ¶ 268.
Mansour does not explicitly disclose, but the combination of Mansour with Liu renders obvious:
Wherein the request for a modified version of the selected subportion of the first rendered LLM output comprises a request to replace one or more of the rendered images with one or more alternative images. Monsour teaches receiving a request to replace selected content with modified content. Mansour ¶ 287, fig. 10B (control 1034). Mansour teaches that content may comprise an image. Id. ¶ 247. Mansour teaches that image content may be edited. Id. ¶ 163. Monsour even teaches that an image may be added to a prompt. Id. ¶ 286. Nevertheless, Mansour does not explicitly disclose an LLM generating an alternative image to replace an image initially generated by an LLM. However, Liu teaches such a feature. Liu ¶¶ 141-146, fig. 5.
It would have been obvious before the effective filing date of the claimed invention to a person with ordinary skill in the art to modify Mansour’s process of replacing rendered LLM content with alterative content with Liu’s process of generating an alternative image to replace an image generated by an LLM. Such a modification would provide a convenient and efficient method of image processing. See Liu ¶ 26.
Regarding claim 11, which depends on claim 10, Liu discloses wherein the request to replace one or more of the rendered images with one or more alternative images comprises a natural language request to retrieve one or more replacement images having specified visual features. Liu ¶¶ 141-146, fig. 5.
Regarding claim 12, Mansour discloses a method implemented using one or more processors, comprising:
Processing a first large language model (LLM) prompt using an LLM to generate a first LLM response. Mansour ¶¶ 258-259.
Providing the first LLM response to a client application, wherein the first LLM response is operable by the client application to provide first rendered LLM output. Mansour ¶¶ 259, 261, fig. 6B.
Receiving, from the client application: an indication of a subportion of the first rendered LLM output that has been selected using one or more input devices, and a request for a modified version of the selected subportion of the first rendered LLM output. Mansour ¶ 285, fig. 10A.
Extracting a subportion of the first LLM response that corresponds to the selected subportion of the first rendered LLM output. Mansour ¶ 286.
Assembling, as a second LLM prompt, the selected subportion of the first LLM response with data indicative of the request to modify the selected subportion of the first rendered LLM output. Mansour ¶ 286.
Processing the second LLM prompt using the same LLM or a different LLM to generate a second LLM response. Mansour ¶ 286.
Providing the second LLM response to the client application, wherein the second LLM response is operable by the client application to provide second rendered LLM output that includes at least a part of the first rendered LLM output outside of the selected subportion of the first rendered LLM output and the modified version of the selected subportion of the first rendered LLM output. Mansour ¶ 287, fig. 10B.
Wherein the first LLM response comprises metadata instructions for rendering one or more images, and the selected subportion of the first rendered LLM output comprises one or more rendered images. Mansour teaches that an LLM may generate images. Mansour ¶ 82. Mansour teaches the generative output may include formatting instructions, embedded commands, and calls to retrieve content. Id. ¶ 268.
Mansour does not explicitly disclose, but the combination of Mansour with Liu renders obvious:
Wherein the request for a modified version of the selected subportion of the first rendered LLM output comprises a natural language request to generate a modified version of one or more of the rendered images, and processing the second LLM prompt using the same LLM or a different LLM comprises processing the natural language request using a text-to-image generative model to generate the modified version of one or more of the rendered images. Monsour teaches receiving a request to replace selected content with modified content. Mansour ¶ 287, fig. 10B (control 1034). Mansour teaches that content may comprise an image. Id. ¶ 247. Mansour teaches that image content may be edited. Id. ¶ 163. Monsour even teaches that an image may be added to a prompt. Id. ¶ 286. Nevertheless, Mansour does not explicitly disclose, in response to receiving a natural language request, using a text-to-image generative model to generate a modified version of an image initially generated by an LLM. However, Liu teaches such a feature. Liu ¶¶ 141-146, fig. 5.
It would have been obvious before the effective filing date of the claimed invention to a person with ordinary skill in the art to modify Mansour’s process of replacing rendered LLM content with alterative content with Liu’s process of generating a modified image in response to a natural language request. Such a modification would provide a convenient and efficient method of image processing. See Liu ¶ 26.
Allowable Subject Matter
Claims 1-6, 8-9, 13-18, and 20 contain allowable subject matter.
Claims 1-6, 8-9, 13-18, and 20 are allowed.
The closest known prior art is Mansour et al., US 2025/0005528 A1. Mansour teaches modifying selected portions of LLM output. However, Mansour does not teach: “formulating a search query based on one or more details of the selected subportion of the first rendered LLM output;” “retrieving, from a search engine, one or more documents that are responsive to the search query;” and “wherein the second LLM prompt incorporates data from the one or more documents that are responsive to the search query.”
Conclusion
Although particular portions of the prior art may have been cited in support of the rejections, the specified citations are merely representative of the teachings. Other passages and figures in the cited prior art may apply. Accordingly, Applicant should consider the entirety of the cited prior art for potentially teaching all or part of the claims.
The following prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Tryzna, US 2024/0338860 A1, discloses using an LLM to generate an image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Asher D Kells whose telephone number is (571)270-7729. The examiner can normally be reached Mon. - Fri., 8 a.m. - 4 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Asher D. Kells
Primary Examiner
Art Unit 2171
/Asher D Kells/Primary Examiner, Art Unit 2171
Prosecution Insights