DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5-6, and 10-12 are rejected under 35 U.S.C. § 103 as being unpatentable over Geller (US 20230244506) in view of Chauhan (US 20230351290).
Claim 1
Geller discloses:
A method implemented by one or more processors, the method comprising: {“According to one embodiment, the techniques herein are performed by computer system 1300 in response to processor 1304 executing at least one sequence of at least one instruction contained in main memory 1306.” [0136]}
processing the input query, using at least one of one or more generative models, to generate both a first pane response and a second pane response, {The system processes client input using NLU module 144, conversational graph 142, trained ML systems, and neural networks to determine intent and produce conversational output. A programmed workflow concurrently produces conversational output 320 in conversational assistant interface 310 and generates presentation instructions for a separate interactive UI 410. [0069], [0071] – [0073], [0088]}
wherein the second pane response differs from the first pane response and wherein the second pane response includes a plurality of interactive graphical elements that are modifiable through pointing-based interaction; {Conversational assistant interface 310 displays conversational output, while interactive UI 410 displays selectable options and widgets. [0079], [0088] – [0092]}
causing the first pane response to be rendered in a first pane of a graphical user interface; {Presentation instructions cause ML generated conversational messages to be rendered in a first window or conversational assistant interface 310. [0078], [0087] – [0088]}
causing the second pane response to be rendered in a second pane of the graphical user interface along with rendering of the first pane response in the first pane of the graphical user interface; {The system concurrently renders conversational assistant interface 310 and the separate interactive UI 410 in the same GUI. [0090], [0097] – [0098]}
in response to detecting, during the monitoring, an instance of natural language input directed to the first pane: processing the instance of natural language input and a representation of the second pane response, using one or more of the generative models, to generate both an additional first pane response and an update to the second pane response; and {The system processes conversational input together with context from the programmed workflow and other GUI information. Then it processes input using ML or conversational components to generate subsequent conversational output and presentation instructions that update the UI according to the workflow. [0073], [0080] – [0081], [0088], [0094] – [0095], [0111]}
processing the one or more updated states and the representation of the second pane response, using one or more of the generative models, to generate an additional first pane response; and {Selected graphical states are supplied as input to the conversational assistant system or machine learning algorithms, which generate subsequent conversational prompts or natural language output based on those selections and the workflow context. [0093] – [0095], 0111], [0120]}
Geller does not disclose, however, Chauhan, in a similar field of endeavor directed to implementing customer engagement applications using a software development kit (SDK) comprising one or more Application Programming Interfaces (APIs), teaches:
receiving an input query that is generated based on user interface input at a client device; {The customer fills out an online form to start a chat, and the UI/SDK invokes handlers based on user input. [0431] – [0433], [0465]] – [0466]}
during rendering of the first pane and the second pane of the graphical user interface: monitoring for occurrence of natural language input that is directed to the first pane and also monitoring for occurrence of pointing-based input that is directed to the second pane and that modifies one or more of the interactive graphical elements; {The system supports text and voice communication through the UI, messages, chat inputs, and events triggered by communications. Also, monitoring user clicks on UI controls, where the click involves SDK handlers and cause state changes. [0360] – [0366]; [0431] – [0433]}
causing the additional first pane response to be rendered in the first pane and causing the second pane response to be updated in accordance with the update to the second pane response; {The system supports UI updates after events, including messages, channel closures, interaction removal, updated context, and materialized data views updated based on events. [0357], [0365], [0503]}
in response to detecting, during the monitoring and while the second pane response, as updated, is rendered in the second pane, an instance of pointing-based input that is directed to the second pane response and that modifies one or more states, of the second pane response as updated, to one or more updated states: {The system supports pointing-based selections that update channel, invite, execution states, etc. examples include Accept, Complete, Pause, adding/closing channels, etc. [0379] – [0390]}
causing the updated second pane response to be further updated to visually reflect the one or more updated states; {The system supports visual UI changes reflecting state updates (e.g., progress spinners, real time data views). [0361] – [0365], [0486] – [0487]}
causing the further first pane response to be rendered in the first pane. {The system supports rendering updated messages, interaction information, and UI responses after workflow processing. [0431], [0623]}
Therefore, it would have been obvious to one of the ordinary skills in the art to modify the automated control of GUIs elements features of Geller to include the generative model prompt processing features of Chauhan, to use retrieved customer context and user input to automatically generate more relevant interface content. (See [0076] of Chauhan).
Claim 5
The combination of Geller and Chauhan teaches the limitations set forth above. Geller further discloses:
causing to be rendered, in the first pane of the graphical user interface and along with the first pane response, a natural language input element; {Conversational assistant interface 310 contains conversational output 320 and an interface for receiving input 330, which may be text input in an editable text field. [0087] – [0088]}
wherein detecting the instance of the natural language input directed to the first pane comprises detecting the instance of the natural language input based on typed input or spoken input and based on the typed input or the spoken input occurring following a pointing-based interaction with the natural language input element rendered in the first pane. {The system supports keyboard text input, editable text fields, and auditory language input captured by a microphone. It further supports pointing device or touchscreen interaction with GUI elements and text input in editable text fields. [0069], [0079], [0088], [0096]}
Claim 6
The combination of Geller and Chauhan teaches the limitations set forth above. Geller further discloses:
a local temporal condition for one or more elements of the second pane response, {The system supports timestamped comments, selected locations on playback timeline, temporal locations in A/V content, and comments associated with corresponding time values. [0106], [0108] – [0109], [0114] – [0118]}
a global temporal condition for all elements of the second pane response, and/or a selection condition that indicates whether an element of the second pane response is currently selected. {The system supports a playback timeline corresponding to the timeline of the video being displayed, and selectable options, selected checkboxes, selected goals, tags, and enabled workflow controls based on selections. [0091] – [0094], [0105], [0111]}
Claim 10
The combination of Geller and Chauhan teaches the limitations set forth above. Geller further discloses:
wherein the input query includes natural language content that is based on the user interface input and/or includes an image that is specified by the user interface input. {Input may be provided through an editable text field or auditory language captured by a microphone. [0069], [0088], [0096]}
Claim 11
The combination of Geller and Chauhan teaches the limitations set forth above. Geller further discloses:
wherein the input query further includes contextual information associated with the user interface input. {The system stores metadata, combines NLU input with other application information, and uses content item metadata when creating tasks. [0068], [0070], [0084]}
Claim 12
The combination of Geller and Chauhan teaches the limitations set forth above. Geller further discloses:
wherein the contextual information includes location information characterizing a location of a client device via which the user interface input is provided, file information characterizing one or more files locally stored at the client device, and/or application information characterizing content from one or more applications of the client device. {The system supports content item storage locally coupled to computer 120 and a file selection widget for selecting a file. [0063], [0098] – [0099]}
Claims 2-4 are rejected under 35 U.S.C. § 103 as being unpatentable over the combination of Geller and Chauhan in view of Vadella (US 20220366154).
Claim 2
While the combination of Geller and Chauhan teaches the limitations set forth above it does not explicitly teach, however, Vadella, in a similar field of endeavor directed to effectively localizing system responses, such that the system responses are grammatical and/or natural in the target language(s), teaches:
processing the one or more updated states and the representation of the second pane response, using one or more of the generative models, to generate generative model output; {The GUI engine processes updated user interactions with the current target language NLG template and generates updated target language output examples reflecting the modified template, where the updated output examples are generated from the current representation of the interactive template. [0035], [0040], [0075] – [0076]}
determining, based on the generative model output, whether to provide the further first pane response; {The system monitors user interactions and, in response to the updated template or other interactions, determines whether to generate and render updated output examples reflecting those interactions. [0074] – [0077]}
wherein determining whether to provide the further first pane response is based on the generative model output characterizing the further first pane response in lieu of characterizing instructions to suppress providing of any further first pane response. {The system automatically generates and renders updated output examples responsive to detected interactions without relying on explicit suppression instructions, instead determining presentation based on the updated generated output. [0074] – [0077]}
Therefore, it would have been obvious to one of the ordinary skills in the art to modify the combination of Geller and Chauan to include the natural language generation features of Vadella, to improve the accuracy and relevance of the GUI by utilizing contextual information and selected entities when generating and presenting UI content. (See [0040] of Vadella).
Claim 3
While the combination of Geller and Chauhan teaches the limitations set forth above it does not explicitly teach, however, Vadella, in a similar field of endeavor directed to effectively localizing system responses, such that the system responses are grammatical and/or natural in the target language(s), teaches:
a conflict portion that includes natural language characterizing a conflict created by the one or more updated states; and {The generated target language output examples reveal grammatical conflicts introduced by user modifications, allowing the user to recognize errors such as incorrect gender, number agreement, or duplicated articles. [0056], [0063], [0065]}
a resolution portion that includes natural language characterizing a candidate resolution to the conflict. {The GUI presents candidate target language primitives and updated output examples that demonstrate corrected results after selecting or previewing an alternate primitive, i.e., providing a candidate resolution to the detected conflict. [0051] – [0058], [0064] – [0066]}
The motivation and rationale to include the additional features of Vadella is the same as set forth previously.
Claim 4
The combination of Geller, Chauhan, and Vadella teaches the limitations set forth above. Vadella further teaches:
wherein the resolution portion is selectable and further comprising: {The candidate target language primitives are selectable through user interaction for incorporation into the current target language template. [0038], [0050] – [0054]}
in response to a user selection of the resolution portion: {The system receives a user selection of a target language primitive via user interface input directed to the interactive template portion. [0054], [0075], [0101] – [0102]}
causing the second pane response to be further updated in accordance with the candidate resolution to the conflict. {In response to the user selecting the alternate target language template and regenerates the target language output examples to reflect the selected candidate resolution. [0057] – [0059], [0075] – [0077]}
The motivation and rationale to include the additional features of Vadella is the same as set forth previously.
Claims 7-9 are rejected under 35 U.S.C. § 103 as being unpatentable over the combination of Geller and Chauhan in view of Ferucci (US 20220261817).
Claim 7
While the combination of Geller and Chauhan teaches the limitations set forth above it does not explicitly teach, however, Ferucci, in a similar field of endeavor directed to a collaborative user support system including a user portal and domain models to receive support requests, render visual aids, and provide suggestions, teaches:
wherein the user interface input is received via interaction with the graphical user interface and when the input query is received the graphical user interface lacks the first pane and the second pane. {The system supports receiving the input query through interaction with a GUI and thereafter rendering responsive visual content. [0157] – [0158], [0165] – [0167], [0172] – [0174]}
Therefore, it would have been obvious to one of the ordinary skills in the art to modify the combination of Geller, Chauhan, and Dmitriev to include the visual feedback corresponding to NL input features of Ferrucci, to improve the clarity of the system’s interpretation and facilitating a shared understanding of the user’s request. (See [0048] od Ferrucci).
Claim 8
The combination of Geller, Chauhan, and Ferrucci teaches the limitations set forth above. Ferucci further teaches:
prior to processing the input query to generate both the first pane response and the second pane response: initially processing the input query to determine, based on the initial processing, that the input query is a candidate for dynamic multi-pane interaction; {NLU module 144 analyzes client input to identify intent before the workflow produces outputs and GUI changes. [0069], [0093], [0111], [0123]}
wherein processing the input query to generate both the first pane response and the second pane response is contingent on determining that the input query is a candidate for dynamic multi-pane interaction. {The assistant identifies intent and determines workflows corresponding to client input. [0069], [0093], [0111]}
The motivation and rationale to include the additional features of Ferrucci is the same as set forth previously.
Claim 9
The combination of Geller, Chauhan, and Ferrucci teaches the limitations set forth above. Ferucci further teaches:
prior to processing the input query to generate both the first pane response and the second pane response, and in response to determining that the input query is a candidate for dynamic multi-pane interaction: {The system supports receiving the input and providing a prompt for additional information before proceeding to generate and present responsive suggestions. [0172], [0175] – [0178]; It further supports determining, based on the input and current session state, whether to generate dialog and render an additional visual component. [0047], [0173]}
causing a prompt to be provided, via the graphical interface, wherein the prompt requests affirmation that dynamic multi-pane interaction is desirable; and {“ the multimodal dialog engine 216 may generate user interface elements to prompt the user to answer questions” [0090]. The user “may be asked to verify whether the each component device was correctly identified” [0052]}
receiving affirmative user interface input responsive to the prompt; {The user “may be prompted by the system to generate training data, including marking generated suggestions with user feedback indicating affirmative or negative (e.g., thumbs up or thumbs down).” [0050]}
wherein processing the input query to generate both the first pane response and the second pane response is in response to receiving the affirmative user interface input responsive to the prompt. {“In response to the user answering a question or advice and/or changing a visual component, the multimodal dialog engine 216 may update the session model to reflect any changes. In response to changes with input scenario, the multimodal dialog engine 216 may update visual presentation of diagnostics data to align with the input scenario.” [0090]}
The motivation and rationale to include the additional features of Ferrucci is the same as set forth previously.
Claims 13-15, and 17 are rejected under 35 U.S.C. § 103 as being unpatentable over the combination of Geller and Chauhan in further view of Dmitriev (US 20250327687).
Claim 13
While the combination of Geller and Chauhan teaches the limitations set forth above it does not explicitly teach, however, Dmitriev, in a similar field of endeavor directed to large language model (LLM) map feedback reporting, teaches:
processing, using one or more of the generative models, a first prompt that includes the input query to generate first generative output; {The user’s natural language map feedback query is processed by the LLM as part of a first operation. [0056], [0077] – [0078], [0134]}
determining, based on the first generative output, an intent reflected by the input query, a plurality of entities for the intent, and a plurality of constraints; {An intent represented by the user’s input is determined, such as reporting incorrect opening hours, a road closure, or an incorrect address; the disclosed locations, business names, and other details constitute multiple entities associated with the determined map feedback intent; the template requirements, permitted values, etc., constrain the structured output. [0054], [0077] – [0078], [0079] –[0092]}
processing, using one or more of the generative models, a second prompt that includes one or more example graphical interface schemas, the intent, the plurality of entities, and the plurality of constraints, to generate second generative output; {The second prompt includes: the intent/error type; entities from the report and address information; and constraints from the template. The LLM then generates a second output formatted according to the template. [0092], [0123] – [0128]}
determining, based on the second generative output, a particular graphical interface schema and a correlation of particular entities, of the entities, to the graphical interface schema; {The system determines a structured data schema or template, and then uses the structured result to determine what map element and controls are displayed. The system correlates particular entities such as graphical map elements. [0080], [0114], [0125], [0166] – [0167]}
and
generating the second pane response based on the graphical interface schema and the correlation of the particular entities to the graphical interface schema. {The displayed map correlation is generated based on the selected error type, the corresponding structured data, and the association between identified roads or POIs and displayed graphical elements. [0114], [0167]}
Therefore, it would have been obvious to one of the ordinary skills in the art to modify the combination of Geller and Chauhan to include the user input LLM processed features of Dmitriev, to allow the user to review, validate, and refine the generated result through the interface. (See [0114] of Dmitriev).
Claim 14
The combination of Geller, Chauhan, and Dmitriev teaches the limitations set forth above. Dmitriev further teaches:
determining, based on the first generative output, entity parameters; {Extracted values (e.g., place name, location, search query) constitute entity parameters used to formulate subsequent requests. [0077] – [0078], [0125], [0134]}
transmitting, via one or more application programming interfaces and to an external system, a request that is generated based on the entity parameters; and {The assistant generates and transmits API requests based on entity parameters extracted from the user input and context. [0149] – [0166], [0168] – [0170]}
receiving, from the external system responsive to the request, the plurality of the entities. {The external systems return multiple entities, including a stadium, road link, and event entities. [0166]}
The motivation and rationale to include the additional features of Dmitriev is the same as set forth previously.
Claim 15
The combination of Geller, Chauhan, and Dmitriev teaches the limitations set forth above. Dmitriev further teaches:
wherein the plurality of entities, received from the external system, include a business location entity that specifies a name of the business location, a location of the business location, and operating hours for the business location. {The DISCOVER result constitutes a business location entity. The returned entity specifies a business. The returned place entity specifies geographic coordinates and may include address information. The business location entity is associated with its name and operating hours. [0134], [0146] – [0149], [0166]}
The motivation and rationale to include the additional features of Dmitriev is the same as set forth previously.
Claim 17
The combination of Geller, Chauhan, and Dmitriev teaches the limitations set forth above. Dmitriev further teaches:
determining, based on the first generative output, the first response; {The initial LLM output is used to determine a natural language response or question presented to the user. [0092] – [0094]}
wherein causing the first pane response to be rendered in the first pane of the graphical user interface comprises causing the first pane response to be rendered prior to generating the second pane response. {The conversational question may be generated and presented before sufficient information exists to generate the completed graphical map correction. [0093] – [0094], [0114], [0167]}
The motivation and rationale to include the additional features of Dmitriev is the same as set forth previously.
Claim 16 is rejected under 35 U.S.C. § 103 as being unpatentable over the combination of Geller, Chauhan, and Dmitriev in further view of Ferrucci (US 20220261817).
Claim 16
While the combination of Geller, Chauhan, and Dmitriev teaches the limitations set forth above it does not explicitly teach, however, Ferrucci, in a similar field of endeavor directed to a collaborative user support system including a user portal and domain models to receive support requests, render visual aids, and provide suggestions, teaches:
receiving, with the input query, an indication of a user account; {The system supports receiving an input query through a user portal while the user is logged into a personal user account associated with user data and stored sessions models. [0025], [0043], [0071] – [0072]}
searching, based on one or more terms of the input query, one or more corpuses for the user account; {The system supports searching “domain text corpora” based on keywords identified form the input query and retrieving stored session data associated with the user account. [0049], [0072], [0074] – [0076]}
determining, based on the searching, one or more responsive information items from the one or more corpuses; and {The system supports determining responsive search results obtained from the searched corpora. [0049], [0076], [0093], [0114]}
including content from the responsive information items in the first prompt that is processed in generating the first generative output. {The system supports including content from the retrieved evidentiary passages and search results as context processed by a generative model to generate an output. [0082], [0095], [0102], [0105] – [0106]}
The motivation and rationale to include the additional features of Ferrucci is the same as set forth previously.
Claims 18 and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over Geller (US 20230244506) in view of Dmitriev (US 20250327687) in further view of Chauhan (US 20230351290).
Claim 18
Geller discloses:
A method implemented by one or more processors, the method comprising: {“According to one embodiment, the techniques herein are performed by computer system 1300 in response to processor 1304 executing at least one sequence of at least one instruction contained in main memory 1306.” [0136]}
receiving an input query that is generated based on user interface input at a client device; {The client computer receives keyboard or spoken input through browser 130 and sends the input to the conversational assistant system. [0069], [0079], [0127] – [0130]}
processing the input query, using at least one of one or more generative models, to generate both a first pane response and a second pane response, {The system processes client input using NLU module 144, conversational graph 142, trained ML systems, and neural networks to determine intent and produce conversational output. A programmed workflow concurrently produces conversational output 320 in conversational assistant interface 310 and generates presentation instructions for a separate interactive UI 410. [0069], [0071] – [0073], [0088]}
wherein the second pane response differs from the first pane response and wherein the second pane response includes a plurality of interactive graphical elements that are modifiable through pointing-based interaction; {Conversational assistant interface 310 displays conversational output, while interactive UI 410 displays selectable options and widgets. [0079], [0088] – [0092]}
causing the second pane response to be rendered in a second pane of the graphical user interface along with rendering of the first pane response in the first pane of the graphical user interface; {The system concurrently renders conversational assistant interface 310 and the separate interactive UI 410 in the same GUI. [0090], [0097] – [0098]}
in response to detecting, during the monitoring, an instance of pointing-based input that is directed to the second pane and that modifies one or more states, of one or more of the interactive graphical elements, to one or more updated states: {Selecting options 402, 404, and 406 changes the state of those GUI elements. Selecting dates, times, tags, or workflow controls similarly modifies GUI or workflow state. [0091] – [0094], [0101] – [0103]}
causing the second pane response to be updated, including causing one or more of the interactive graphical elements to visually reflect the one or more updated states; {The system visually reflects selected states through checkboxes, buttons, filtered comments, and changed GUI content. [0091] – [0095], [0114] – [0115]}
processing a representation of the second pane response including the one or more updated states, using one or more of the generative models, to generate an additional first pane response; and {Selected graphical states are supplied as input to the conversational assistant system or machine learning algorithms, which generate subsequent conversational prompts or natural language output based on those selections and the workflow context. [0093] – [0095], [0111], [0120]}
causing the additional first pane response to be rendered in the first pane; {After user selection in the second pane, the conversational assistant interface displays an additional prompt, or natural language output. [0095], [0111], [0120] – [0121]}
in response to detecting, during the monitoring and subsequent to the additional first pane response being rendered, an instance of natural language input directed to the first pane: {The system supports continued workflow interaction where the user may provide further typed or spoken input through the conversational assistant interface after prior GUI selections and prompts. [0080] – [0081], [0088], [0095] – [0096], [0123]}
processing the instance of natural language input and a current representation of the second pane response at a time of the instance of natural language input, using one or more of the generative models, to generate both a further first pane response and an update to the second pane response; and {The system processes conversational input together with context from the programmed workflow and other GUI information. Then it processes input using ML or conversational components to generate subsequent conversational output and presentation instructions that update the UI according to the workflow. [0073], [0080] – [0081], [0088], [0094] – [0095], [0111]}
causing the additional first pane response to be rendered in the first pane and causing the second pane response to be further updated in accordance with the generated update to the second pane response. {The system supports rendering subsequent conversational output in the conversational output in the conversational assistant interface and updating the interactive UI based on workflow state. [0080] – [0081], [0092] – [0095], [0111], [0120] – [0121]}
Geller does not disclose, however, Dmitriev, in a similar field of endeavor directed to large language model (LLM) map feedback reporting, teaches:
causing the first pane response to be rendered in a first pane of a graphical user interface; {The conversational question LLM response can be rendered in a user interface. [0093] – [0094], [0134]}
The motivation and rationale to include the additional features of Dmitriev is the same as set forth previously.
The combination of Geller and Dmitriev does not teach, however, Chauhan, in a similar field of endeavor directed to implementing customer engagement applications using a software development kit (SDK) comprising one or more Application Programming Interfaces (APIs), teaches:
while rendering the graphical user interface: monitoring for occurrence of natural language input that is directed to the first pane and also monitoring for occurrence of pointing-based input that is directed to the second pane and that modifies one or more states of one or more of the interactive graphical elements; {The system supports text and voice communication through the UI, messages, chat inputs, and events triggered by communications. Also, monitoring user clicks on UI controls, where the click involves SDK handlers and cause state changes. [0360] – [0366]; [0431] – [0433]}
The motivation and rationale to include the additional features of Chauhan is the same as set forth previously.
Claim 20
Geller discloses:
A method implemented by one or more processors, the method comprising: {“According to one embodiment, the techniques herein are performed by computer system 1300 in response to processor 1304 executing at least one sequence of at least one instruction contained in main memory 1306.” [0136]}
processing the input query, using at least one of one or more generative models, to generate both a first pane response and a second pane response, {The system processes client input using NLU module 144, conversational graph 142, trained ML systems, and neural networks to determine intent and produce conversational output. A programmed workflow concurrently produces conversational output 320 in conversational assistant interface 310 and generates presentation instructions for a separate interactive UI 410. [0069], [0071] – [0073], [0088]}
wherein the second pane response differs from the first pane response and wherein the second pane response includes a plurality of interactive graphical elements that are modifiable through pointing-based interaction, and {Conversational assistant interface 310 displays conversational output, while interactive UI 410 displays selectable options and widgets. [0079], [0088] – [0092]}
causing the second pane response to be rendered in a second pane of the graphical user interface along with rendering of the first pane response in the first pane of the graphical user interface; and {The system concurrently renders conversational assistant interface 310 and the separate interactive UI 410 in the same GUI. [0090], [0097] – [0098]}
Geller does not disclose, however, Dmitriev, in a similar field of endeavor directed to large language model (LLM) map feedback reporting, teaches:
wherein processing the input query, using at least one of the one or more generative models, to generate the second pane response comprises: {The LLM processes the user’s natural language map feedback input and produces structured information that is used to generate a graphical map visualization. [0054] – [0056], [0114]}
processing, using one or more of the generative models, a first prompt that includes the input query to generate first generative output; {The user input is supplied to an LLM, which produces an initial output identifying the subject and meaning of the report, e.g., Table 8. [0077] – [0078], [0134]}
determining, based on the first generative output, an intent reflected by the input query, a plurality of entities for the intent, and a plurality of constraints; {An intent represented by the user’s input is determined, such as reporting incorrect opening hours, a road closure, or an incorrect address; the disclosed locations, business names, and other details constitute multiple entities associated with the determined map feedback intent; the template requirements, permitted values, etc., constrain the structured output. [0054], [0077] – [0078], [0079] –[0092]}
processing, using one or more of the generative models, a second prompt that includes the intent, the plurality of entities, and the plurality of constraints, to generate second generative output; {The second prompt includes: the intent/error type; entities from the report and address information; and constraints from the template. The LLM then generates a second output formatted according to the template. [0092], [0123] – [0128]}
determining, based on the second generative output, a particular graphical interface schema and a correlation of particular entities, of the entities, to the graphical interface schema; and {The system determines a structured data schema or template, and then uses the structured result to determine what map element and controls are displayed. The system correlates particular entities such as graphical map elements. [0080], [0114], [0125], [0166] – [0167]}
generating the second pane response based on the graphical interface schema and the correlation of the particular entities to the graphical interface schema; {The displayed map correlation is generated based on the selected error type, the corresponding structured data, and the association between identified roads or POIs and displayed graphical elements. [0114], [0167]}
causing the first pane response to be rendered in a first pane of a graphical user interface; {The conversational question LLM response can be rendered in a user interface. [0093] – [0094], [0134]}
while rendering the graphical user interface: monitoring for occurrence of natural language input that is directed to the first pane and also monitoring for occurrence of pointing-based input that is directed to the second pane and that modifies one or more states of one or more of the interactive graphical elements. {The system monitors or receives natural language input during interaction with the displayed interface [0056], [0066] – [0068]; A user may provide pointing-based input to select the confirmation control associated with the graphical map visualization [0167], [0200]. Selecting a confirmation option changes the confirmation status of the displayed correction and may cause submission or updating of the displayed map data [0114], [0167], [0200].
The motivation and rationale to include the additional features of Dmitriev is the same as set forth previously.
The combination of Geller and Dmitriev does not teach, however, Chauhan, in a similar field of endeavor directed to implementing customer engagement applications using a software development kit (SDK) comprising one or more Application Programming Interfaces (APIs), teaches:
receiving an input query that is generated based on user interface input at a client device; {The customer fills out an online form to start a chat, and the UI/SDK invokes handlers based on user input. [0431] – [0433], [0465]] – [0466]}
The motivation and rationale to include the additional features of Chauhan is the same as set forth previously.
Claim 19 is rejected under 35 U.S.C. § 103 as being unpatentable over the combination of Geller, Dmitriev, and Chauhan in further view of Vadella (US 20220366154).
Claim 19
While the combination of Geller, Dmitriev, and Chauhan teaches the limitations set forth above, it does not explicitly teach, however, Vadella, in a similar field of endeavor directed to effectively localizing system responses, that include dynamic information, to target language(s), such that the system responses are grammatical and/or natural in the target language(s), teaches:
processing the one or more updated states and the representation of the second pane response, using one or more of the generative models, to generate generative model output; {The GUI engine processes updated user interactions with the current target language NLG template and generates updated target language output examples reflecting the modified template, where the updated output examples are generated from the current representation of the interactive template. [0035], [0040], [0075] – [0076]}
determining, based on the generative model output, whether to provide the further first pane response; {The system monitors user interactions and, in response to the updated template or other interactions, determines whether to generate and render updated output examples reflecting those interactions. [0074] – [0077]}
wherein determining whether to provide the further first pane response is based on the generative model output characterizing the further first pane response in lieu of characterizing instructions to suppress providing of any further first pane response. {The system automatically generates and renders updated output examples responsive to detected interactions without relying on explicit suppression instructions, instead determining presentation based on the updated generated output. [0074] – [0077]}
The motivation and rationale to include the additional features of Vadella is the same as set forth previously.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure (additional pertinent references can be found on attached form PTO-892):
US 20200042515 A1, which teaches: A method for providing query responses to a user via online chat establishes a first communication connection for online chat between a user interface and an artificial intelligence (AI) entity.
US 20250307897 A1, which teaches: A computer-implemented method comprises receiving a first chat message at a server computer from a chat interface of a reservation application executing on a mobile computing device.
US 20130033414 A1, which teaches: A display environment for a plurality of display devices is described. In one or more implementations, a display environment of an operating system of a computing device is configured to display a plurality of shells that support user interaction with the operating system by launching a first shell for display on a first display device of the computing device and launching a second shell for display on a second display device of the computing device such that the first and second shells are displayable simultaneously by the computing device.
“A Context-Aware Chatbot for tourist destinations” (NPL attached), which teaches: The cultural heritage is one of the most important resources of the territory. It represents one of the scenarios where new technologies can provide more interesting contributions. In particular, adaptive systems and related services may increase the promotion of cultural heritage. In fact, tourists can use several services able to filter the huge amount of data present on the network in order to only provide relevant information. The aim of this paper is to introduce a chatbot based on a Context-Aware System. This chatbot recommends contents and services according to tourist profiles and context. For testing the proposed architecture, a prototype was developed in order to support tourists during a visit to some cultural sites in Campania: Paestum, Pompeii and Herculaneum. The first experimental results are encouraging and show the potential of the proposed approach.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARLOS F MONTALVO whose telephone number is (703)756-5863. The examiner can normally be reached Monday - Friday 8:00AM - 5:30PM; First Fridays OOO.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sarah Monfeldt can be reached at 571-270-1833. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.F.M./Examiner, Art Unit 3629 /SARAH M MONFELDT/Supervisory Patent Examiner, Art Unit 3629
Prosecution Insights