Prosecution Insights
Last updated: April 19, 2026
Application No. 18/670,819

VISUAL RESPONSES TO USER INPUTS

Non-Final OA §101§103
Filed
May 22, 2024
Examiner
SMITH, SEAN THOMAS
Art Unit
2659
Tech Center
2600 — Communications
Assignee
Amazon Technologies, Inc.
OA Round
1 (Non-Final)
83%
Grant Probability
Favorable
1-2
OA Rounds
2y 8m
To Grant
99%
With Interview

Examiner Intelligence

Grants 83% — above average
83%
Career Allow Rate
5 granted / 6 resolved
+21.3% vs TC avg
Strong +33% interview lift
Without
With
+33.3%
Interview Lift
resolved cases with interview
Typical timeline
2y 8m
Avg Prosecution
37 currently pending
Career history
43
Total Applications
across all art units

Statute-Specific Performance

§101
27.9%
-12.1% vs TC avg
§103
50.7%
+10.7% vs TC avg
§102
12.9%
-27.1% vs TC avg
§112
8.6%
-31.4% vs TC avg
Black line = Tech Center average estimate • Based on career data from 6 resolved cases

Office Action

§101 §103
Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Priority Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Accordingly, the present application has been granted the benefit of the earlier filing date of U.S. Application 17/547,586, filed on December 12th, 2021. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a mental process that can be performed in the human mind or with the aid of pen and paper. This judicial exception is not integrated into a practical application because a computer is invoked merely as a tool to execute an abstract idea. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because an abstract idea is merely applied on a generic computer without any element that would otherwise preclude performance of the abstrac. Regarding claim 1, the claim recites “A computer-implemented method, comprising:receiving first input data corresponding to a natural language input of a dialog;receiving second input data representing user profile data;receiving third input data corresponding to at least one previous input or response of the dialog; andprocessing the first input data, the second input data and the third input data using a machine learning component to determine output data representing visual content to be presented on a screen of a device.” The limitations of “receiving… input data,” and “processing… input data… to determine output data,” as drafted cover mental activities which can be performed in the mind or with the aid of pen and paper. Taken individually, or as a whole, these limitations describe acts which are equivalent to human mental work of answering questions or servicing requests. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be performed mentally, and no additional features in the claims would preclude them from being performed as such. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 2, the claim depends from claim 1, and thus recites the limitations of claim 1, “further comprising: receiving first response data corresponding to a potential response to the natural language input; andreceiving supplemental content data corresponding to the natural language input, wherein the first input data comprises the first response data and the supplemental content data.” Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of answering questions or servicing requests. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 3, the claim depends from claim 2, and thus recites the limitations of claims 1 and 2, “wherein: the potential response corresponds to a first content source; andthe supplemental content data corresponds to a second content source different from the first content source.” Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of servicing requests with different sources. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 4, the claim depends from claim 1, and thus recites the limitations of claim 1, “further comprising: determining first data representing a user input, wherein the first input data comprises the first data.” The limitation of “determining first data representing a user input,” as drafted covers mental activities which can be performed in the mind or with the aid of pen and paper. Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of identifying speech. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 5, the claim depends from claim 4, and thus recites the limitations of claims 1 and 4, “wherein the user input comprises a predicted user input.” Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of servicing anticipated requests. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 6, the claim depends from claim 1, and thus recites the limitations of claim 1, “wherein the output data comprises:first data representing first content to be displayed;second data representing second content to be displayed; andthird data representing placement of the first content and the second content as part of presentation on the screen.” Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of organizing responses. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 7, the claim depends from claim 6, and thus recites the limitations of claims 1 and 6, “wherein: the first content corresponds to a first content source; andthe second content corresponds to a second content source different from the first content source.” Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of organizing responses. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 8, the claim depends from claim 6, and thus recites the limitations of claims 1 and 6, “further comprising: determining the third data based at least in part on layout template information.” Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of organizing responses. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 9, the claim depends from claim 1, and thus recites the limitations of claim 1, “further comprising: receiving fourth input data representing information about the device, wherein the machine learning component further processes the fourth input data to determine the output data.” The limitation of “receiving fourth input data,” as drafted covers mental activities which can be performed in the mind or with the aid of pen and paper. Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of organizing responses. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claim 10, the claim depends from claim 1, and thus recites the limitations of claim 1, “further comprising:determining a condition corresponding to output of first visual content;determining the condition is satisfied; andin response to determining the condition is satisfied, including, in the output data, first data representing the first visual content.” The limitations of “determining a condition…” “determining the condition is satisfied,” and “including, in the output data, first data…” as drafted cover mental activities which can be performed in the mind or with the aid of pen and paper. Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of organizing responses. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible. Regarding claims 11-20, system claims 11-20 and method claims 1-10 are related as a method and system of using the same, with each system element’s function corresponding to the method step. Accordingly, claims 11-20 are similarly rejected under the same rationale as applied to claims 1-10. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1-4, 6-14 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2021/0334067 to Thakkar (hereinafter, "Thakkar") in view of U.S. Patent Application Publication 2021/090575 to Mahmood et al. (hereinafter, "Mahmood"). The Mahmood reference has a common assignee with the instant application. Based upon the earlier effectively filed date of the reference, it constitutes prior art under 35 U.S.C. 102(a)(2). This rejection under 35 U.S.C. 103 might be overcome by: (1) a showing under 37 CFR 1.130(a) that the subject matter disclosed in the reference was obtained directly or indirectly from the inventor or a joint inventor of this application and is thus not prior art in accordance with 35 U.S.C.102(b)(2)(A); (2) a showing under 37 CFR 1.130(b) of a prior public disclosure under 35 U.S.C. 102(b)(2)(B); or (3) a statement pursuant to 35 U.S.C. 102(b)(2)(C) establishing that, not later than the effective filing date of the claimed invention, the subject matter disclosed and the claimed invention were either owned by the same person or subject to an obligation of assignment to the same person or subject to a joint research agreement. See generally MPEP § 717.02. Regarding claims 1 and 11, Thakkar teaches a system and method comprising: receiving first input data corresponding to a natural language input of a dialog (paragraph [0027], "Unlike at least some of the other applications installed on the client device, the digital assistant application can have natural language processing capabilities. Configured with these capabilities, the digital assistant application can acquire and parse an input audio signal detected via a microphone of the client device to recognize words from the input audio signal."); receiving third input data corresponding to at least one previous input or response of the dialog (paragraph [0087], "When the input audio signal is determined to lack an agent identifier, the action handler component 120 can select the action-inventory 140 from the set maintained on the data repository 128 based on the request, the one or more parameters, and an agent usage history. The agent usage history can indicate usage statistics of the agent application 110 on the client device 104 running the digital assistant application 108 from which the input audio signal is received."); and processing the first input data, the second input data and the third input data using a machine learning component to determine output data representing visual content to be presented on a screen of a device (paragraph [0064], "The data processing system 102 can include an application, script, or program installed at the client device 104, such as the instance of the digital assistant application 108 on the client device 104 to communicate input audio signals to the communications interface 112 of the data processing system 102 and to drive components of the client computing device to render output audio signals or visual output."). Thakkar does not explicitly teach “receiving second input data representing user profile data,” and thus, Mahmood is introduced. Mahmood teaches receiving second input data representing user profile data (paragraph [0038], "The NLP system 120 may receive (152) one or more signals representing one or more assistants to be used with respect to the natural language input… A further example signal is a user identifier representing the user 5 that originated the natural language input (as a user profile, associated with a user's identifier, may represent a preferred assistant to be used to respond to natural language inputs originating from the user)."). Thakkar and Mahmood are considered analogous because they are each concerned with natural language assistants. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Thakkar with the teachings of Mahmood for the purpose of improving assistant performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Regarding claims 2 and 12, Thakkar teaches a system and method further comprising: receiving first response data corresponding to a potential response to the natural language input (paragraph [0077], "In fulfilling the request, the digital assistant application 108 can invoke the response selector component 122. The response selector component 122 can select or identify responses phrases using the policies 134 or the response data 136 maintained on the data repository 128."); and receiving supplemental content data corresponding to the natural language input, wherein the first input data comprises the first response data and the supplemental content data (paragraph [0077], "The response selector component 122 can search the policies 134 for generating the output using the request type of the response in fulfilling the request."). Regarding claims 3 and 13, Thakkar teaches a system and method wherein: the potential response corresponds to a first content source (paragraph [0077], "In fulfilling the request, the digital assistant application 108 can invoke the response selector component 122. The response selector component 122 can select or identify responses phrases using the policies 134 or the response data 136 maintained on the data repository 128."); and the supplemental content data corresponds to a second content source different from the first content source (paragraph [0079], "Conversely, from the comparison, the NLP component 116 can determine that the request parsed from the input audio signal does not match any of the list of requests for function of the digital assistant application 108. In response to the determination, the NLP component 116 can determine whether the request corresponds to one of the functions of the agent application 110," and paragraph [0083], "In response to the determination that the request corresponds to the function of the agent application 110, the action handler component 120 can identify or select an action-inventory 140 from the data repository 128 for the action corresponding to the request."). Regarding claims 4 and 14, Thakkar teaches a system and method further comprising: determining first data representing a user input, wherein the first input data comprises the first data (paragraph [0003], "The natural language processor can parse the input audio signal of the data packet to identify a request."). Regarding claims 6 and 16, Thakkar teaches a system and method wherein the output data comprises: first data representing first content to be displayed (paragraph [0003], "The client device can display a graphical user interface of a first application in a foreground process on the client device."); second data representing second content to be displayed (paragraph [0003], "The agent interface can display, on the client device, the user interface component including the output from the second application with the graphical user interface of the first application authorized to be presented with the second application."); and third data representing placement of the first content and the second content as part of presentation on the screen (paragraph [0052], "Furthermore, each user interface component can have one or more properties. The properties can include an element type (e.g., command button, scroll bar, textbox, and image), a size, a location within the user interface component 142, transparency, and shape, among others."). Regarding claims 7 and 17, Thakkar teaches a system and method wherein: the first content corresponds to a first content source (paragraph [0003], "The client device can display a graphical user interface of a first application in a foreground process on the client device."); and the second content corresponds to a second content source different from the first content source (paragraph [0003], "The agent interface can display, on the client device, the user interface component including the output from the second application with the graphical user interface of the first application authorized to be presented with the second application."). Regarding claims 8 and 18, Thakkar teaches a system and method further comprising: determining the third data based at least in part on layout template information (paragraph [0056], "In addition to the address template, each action-inventory 140 maintained by the agent registry component 124 can include information related to the action for the action-inventory 140. The action-inventory 140 can include an interface mode indicator. The indicator can specify whether the user interface component 142 of the agent application 110 is to be displayed in carrying out the action. The user interface component 142 can be a visual element to be rendered in the display 148 of the client device 104. The user interface component 142 can have a size less than a size of the display 148. For example, the user interface component 142 can include a box-sized visual element to occupy a 12.5% to 25% of the display 148. The user interface component 142 can be associated with at least one of the action-inventories 140 maintained on the data repository 128."). Regarding claims 9 and 19, Thakkar teaches a system and method further comprising: receiving fourth input data representing information about the device, wherein the machine learning component further processes the fourth input data to determine the output data (paragraph [0027], "Some requests, however, may not correspond to any of the functions preconfigured on the digital assistant application (e.g., playing a video from an online video platform). For these types of requests, the digital assistant application can identify another application that is capable of carrying out the request and invoke the function of the application to pass parameters."). Regarding claims 10 and 20, Thakkar teaches a system and method further comprising: determining a condition corresponding to output of first visual content (paragraph [0003], "The agent interface can determine that the output of the second application from execution of the action is authorized to be presented with the graphical user interface of the first application based on an authorization policy of the second application."); determining the condition is satisfied (paragraph [0003], "The agent interface can identify the user interface component of the second application for the action-inventory selected from the plurality of action-inventories, responsive to the determination the output of the second application is authorized to be presented with the first application."); and in response to determining the condition is satisfied, including, in the output data, first data representing the first visual content (paragraph [0003], "The agent interface can display, on the client device, the user interface component including the output from the second application with the graphical user interface of the first application authorized to be presented with the second application."). Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Thakkar and Mahmood as applied to claims 1, 4, 11 and 14 above, and further in view of U.S. Patent Application Publication 2020/0380057 to Smit (hereinafter, "Smit"). Regarding claims 5 and 15, the combination of Thakkar and Mahmood does not teach a system or method “wherein the user input comprises a predicted user input,” and thus, Smit is introduced. Smit teaches the user input comprises a predicted user input (paragraph [0040], "Embodiments of the content-service tier can also pre-fetch often-accessed results from the content source(s), in order to optimize the speed of QuestObjects client requests that can be anticipated, and possibly to fill the QuestObjects caches after a restart of the system."). Thakkar, Mahmood and Smit are considered analogous because they are each concerned with content retrieval. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Thakkar and Mahmood with the teachings of Smit for the purpose of improving agent performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: U.S. Patent Application Publication 2015/0348549 to Giulu. U.S. Patent Application Publication 2015/0381702 to Heeter. U.S. Patent Application Publication 2019/0347068 to Khaitan. U.S. Patent Application Publication 2021/0241760 to Whalin. U.S. Patent 11,237,635 to Forsland. U.S. Patent 11,516,434 to Agarwal. U.S. Patent Application Publication 2022/0415307 to Kockerbeck. U.S. Patent 11,626,107 to Hajebi. U.S. Patent 11,670,068 to Kim. U.S. Patent 11,670,285 to Crews. U.S. Patent 11,900,072 to Bossio. U.S. Patent 11,907,676 to Pemberton. Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN T SMITH whose telephone number is (571)272-6643. The examiner can normally be reached Monday - Friday 8:00am - 5:00pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PIERRE-LOUIS DESIR can be reached at (571) 272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /SEAN THOMAS SMITH/Examiner, Art Unit 2659 /PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Read full office action

Prosecution Timeline

May 22, 2024
Application Filed
Jan 13, 2026
Non-Final Rejection — §101, §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12602540
LEVERAGING A LARGE LANGUAGE MODEL ENCODER TO EVALUATE PREDICTIVE MODELS
2y 5m to grant Granted Apr 14, 2026
Patent 12530534
SYSTEM AND METHOD FOR GENERATING STRUCTURED SEMANTIC ANNOTATIONS FROM UNSTRUCTURED DOCUMENT
2y 5m to grant Granted Jan 20, 2026
Study what changed to get past this examiner. Based on 2 most recent grants.

AI Strategy Recommendation

Get an AI-powered prosecution strategy using examiner precedents, rejection analysis, and claim mapping.
Powered by AI — typically takes 5-10 seconds

Prosecution Projections

1-2
Expected OA Rounds
83%
Grant Probability
99%
With Interview (+33.3%)
2y 8m
Median Time to Grant
Low
PTA Risk
Based on 6 resolved cases by this examiner. Grant probability derived from career allow rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month