Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claim 20 objected to because of the following informalities: the claim recites the performance of “semantic matching that goes beyond exact matches.” The recited “goes beyond” is a relative term or term of degree with no effective measure—Examiner will consider the recited going beyond as performance of matching which may be exact or not exact. Appropriate correction is required.
Claim Rejections - 35 USC § 112
Claim 2 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. The claim concludes with a semi-colon rather than a period as required; the semi-colon suggests that relevant claim language is missing.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Claim(s) 1, 16, 20 is/are directed to a system, method, etc. for responding to a user query by provision of an answer. The claims rely on well understood, routine, and conventional structures such as a processor, memory, data structure, etc. to instruct the system along methods by which the system returns an answer to a natural language user query by operation of an appropriate portion of a user interface to determine the answer data and provide the answer. The claims are considered a manner by which data resolves more data, in this case a natural language query and response thereto; the claims are also considered a stand in for human behavior as the claims steps are substantially similar to the manner in which a human being might request another to operate an interface on their behalf. As such the claims cannot be considered to integrate the judicial exceptions of an abstract idea such as data per se or programs per se nor the judicial exception of human activity and/or mental processes such as operations performed in the human mind, human activity, human behavior; etc. as the claims do not include substantially more than the performance of such exceptions upon a computer claimed at a high level of generality and based on models intended to mimic or replicate human cognitive processes. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Dependent claims 2-15, 17-19, 20 do not remedy and are similarly rejected as the claims further address additional subject matter which may be seen as the generation of data from data; a stand in for human behavior, and/or human application of agency in concert with assistive instructions, mathematic concepts, AI models, etc.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of copending Application No. 18597397. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims are considered substantially similar as the claims in each application reference recite:
receiving a natural language query input from a user interface;
targeting a plurality of remote resources such as pages, entities, etc. for search, analysis, etc.;
using vector representations of entities, optionally vectorized;
performing semantic type matching of the query and the searched remote resources;
building a graph or graph type data structure of the remote resources and returned results;
outputting a result to a user, etc.
Any remaining distinctions among the claims are considered well-known, routine and/or conventional variants of methods and operations by which a user query, question, etc. is used to resolve a result, answer, etc..
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 rejected under 35 U.S.C. 103 as being unpatentable over Coimbra: 20210326430 hereinafter Coi further in view of Kundel: 20240354321 hereinafter Kun and further in view of Hoang: 20240061832 hereinafter Ho.
Regarding claim 1
Coi teaches:
A computer-implemented method comprising:
receiving a natural language query (Coi: ¶ 14, 70: system receives a user voice query, identifies a trigger, keyword, etc.);
determining a terminus user interface element out of a plurality of user interface elements of an application having a plurality of pages (Coi: ¶ 77, 78, 80, 82: action API generates action data corresponding to the trigger, keyword, etc. to cause performance of an action corresponding thereto),
wherein determining the terminus user interface element comprises, finding a user interface element matching the natural language query via internal representations of the user interface elements wherein user interface element metadata is incorporated into the internal representations of the user interface elements (Coi: ¶ 75-78: action corresponding to, matching, etc. the user query generative of action data structure to cause an application upon a user device to perform the corresponding action wherein user interface element metadata comprise an internal representation incorporated into the action data structure);
navigating to a given page out of the plurality of pages of the application on which the terminus user interface element appears (Coi: ¶ 3, 9-11, 26, 30, 81, 107, 113, 117: the action API determines a type of action corresponding to a keyword, such as a navigation action, such as of an electronic resource and performs corresponding actions upon the electronic resource such as input of data to the electronic resource and resolution of responses thereto);
from the given page, extracting determined parameter data values responsive to the query based on a state of the determined resource for the terminus user interface element (Cio: Abstract; ¶ 3, 9-14: such as by executing actions by determining, inputting, etc. of relevant parameters such by populating fields or otherwise resolving a response based on user intent);
presenting the answer data as an answer to the natural language query (Coi: ¶ 35, 67, 148: in this way a user voice input to the system resolves an answer based thereon).
Coi does not explicitly discuss determining the terminus user interface element comprises, with a large language model, from the given page, extracting answer data using, based on, etc. the LLM and presenting the answer data as an answer to the natural language query.
In a related field of endeavor Kun teaches a system and method for receiving a natural language query associated with a user (Kun: Abstract; Fig 2A); determining a terminus user interface element out of a plurality of user interface elements of an application having a plurality of pages (Kun: Abstract, ¶ 13, 29, 42, etc.; Fig 2A, etc.: generative NL model forms, resolves, and augments a user query based on user ID, contextual data which is communicated to a recipient such as by a terminus user interface element on the client side of the system operable therefore), wherein determining the terminus user interface element comprises, with a large language model (Kun: ¶ 3, etc.: system operates based on an LLM chatbot framework) determining and presenting answer data to a user (Kun: ¶ 48, etc.: system provides natural language responses to the user query). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize an LLM to generate an augmented query as taught or suggested by Kun and thereby improve the resolution of an answer by determining and using appropriate user interface elements in concert with the teachings of Coi and for at least the purpose of reifying the determined resources and interface elements thereof against an answer list and thereby improving, optimizing, etc. the operations of the LLM; one of ordinary skill in the art would have expected only predictable results therefrom.
Coi in view of Kun does not explicitly discuss the manner in which the LLM utilizes the internal representations such as using an embedding with integrated metadata.
In a related field of endeavor Ho teaches a system and method for determining a database schema using an encoder (Ho: Abstract, etc.; Fig 7, etc.) wherein the schema incorporates the user query and metadata based thereon (Ho: Abstract; ¶ 32, 37, 44; Fig 7) to configure a generative model to determine a graph, tree, data structure model using embeddings therefor and thereby output a best response to the user based on the input query, metadata, etc. (Ho: ¶ 37, 125, 126, etc.). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the Ho taught schema to improve the internal representations of the Coi in view of Kun system and method for at least the purpose of improving the response of the system to a user query, optimizing compute for doing so, better directing an answer to the user query, etc.; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 2
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 1, wherein: the internal representations of the user interface elements comprise respective vector embeddings calculated based on the metadata incorporated into the internal representations of the user interface elements (Coi: ¶ 75-78: action corresponding to, matching, etc. the user query generative of action data structure to cause an application upon a user device to perform the corresponding action wherein user interface element metadata comprise an internal representation incorporated into the action data structure); (Kun: ¶ 3, 13, 29, 42, 48, etc.; Fig 2A, etc.: system operates based on an LLM chatbot framework); (Ho: ¶ 133-136, 146-148; Fig 7: search conducted by generating vectorized word embeddings for determining relevant responses based on internal representations such as using a pre-trained language model such as BERT); and finding a matching user interface element comprises:
computing a vector embedding of the natural language query; and for the vector embedding of the natural language query, finding a matching vector embedding out of the respective vector embeddings, wherein the matching vector embedding is associated with the terminus user interface element (Coi: ¶ 75-78, etc.); (Kun: ¶ 3, 13, 29, 42, 48, etc.; Fig 2A, etc.) (Ho: ¶ 133-136, 146-148; Fig 7: search conducted by generating vectorized word embeddings for determining relevant responses based on internal representations such as using a pre-trained language model such as BERT). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 3
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 2, wherein: finding the matching vector embedding comprises finding a top N matching embeddings; and the large language model comprises a completion large language model that chooses the terminus user interface element out of the top N matching embeddings. Examiner takes official notice that conducing a search over a vectorized space to determine and reify a “top N,” matching embeddings was well-known in the art before the effective filing date of the instant invention and would have comprised an obvious inclusion such as for selecting particular terminus locations, elements, and/or databases upon which to resolve an answer by operation of an appropriate user interface element of the Coi in view of Kun in view of Ho system and method; one of ordinary skill in the art would have expected only predictable results therefrom. The claim is thus considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 4
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 2, wherein: the respective vector embeddings are pre-calculated before the natural language query is received (Ho: ¶ 133-136l; Fig 7, etc.: such as by accessing a pre-trained language model to generate a received query, determine a search over a pre-computed vector space in response thereto, etc.). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 5
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 1, wherein: navigating to a given page out of the plurality of pages of the application on which the terminus user interface element appears comprises: finding the given page in a graph representation of the plurality of pages of the application, wherein the graph representation stores a path of the given page; and navigating to the path of the given page (Coi: Abstract; ¶ 29, 30, 59, 60, etc.; system builds an intent manifest with mappings to particular pages, URL’s, links, etc.; navigates appropriately thereto, operates user interface elements thereof in accord therewith; see claim 1 supra); (Ho: ¶ 133-136, 146-148, etc.; Fig 7, etc.: such as by accessing a pre-trained language model to generate a received query, determine a search over a pre-computed vector space in response thereto, etc. iterate over a knowledge graph and provide graph like data structures in response to the query). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 6
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 5, further comprising: with a large language model (Kun: ¶ 3, 13, 29, 42, 48, etc.; Fig 2A, etc.: system operates based on an LLM chatbot framework), (Ho: ¶ 133-136, 146-148; Fig 7: such as using a pre-trained language model such as BERT) classifying tags appearing in the plurality of pages as being of different element types (Coi: ¶ 56-58, 81, 116, 117: system determines various classes including entity types, element types, content types, formatting, etc. types such as for matching of relevant keywords, tags, labels, etc. based on determination, parsing, etc. of tags, labels, etc.); (Ho: ¶ 6, 44, 95, 105: chatbot interfaces well-known to operate over tagged parts of speech, label tokens, input types, etc. to thereby classify intent based on keywords, intents, topics, etc. determine and respond appropriately to types of tasks based thereon); (Ho: ¶ 22, : natural language labels bear semantic meaning for querying disparate data using language models and responding based on context based thereon); wherein the different element types comprise input, label, and button (Coi: ¶ 56-58, 81, 83, 116, 117: structured access of resolved resources based additionally upon input form field and button or other widget data). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 7
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 5, wherein: the graph representation comprises edges indicating how to navigate between the plurality of pages (Coi: Abstract; ¶ 29, 30, 59, 60, etc.; system builds an intent manifest with mappings to particular pages, URL’s, links, etc.; navigates appropriately thereto, operates user interface elements thereof in accord therewith the pages are considered to correspond to nodes and the actions direct by the system are considered to correspond to edges; see claim 1 supra); (Ho: Abstract; ¶ 133-136, 146-148, etc.; Fig 7, etc.: relations between elements are considered to correspond to edges). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 8
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 5, wherein: the path comprises a Uniform Resource Locator of a page (Coi: Abstract; ¶ 29, 30, 59, 60, etc.; system builds an intent manifest with mappings to particular pages, URL’s, links, etc.; navigates appropriately thereto, operates user interface elements thereof in accord therewith; see claim 1 supra). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 9
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 5, wherein: the path comprises a starting page, and one or more user interface actions to navigate from the starting page to the given page (Coi: Abstract; ¶ 29, 30, 59, 60, etc. system operates to generate an action API, data structure etc. which effectively delineates a path necessary to compete an action, sequence of actions, etc.); and browser automation applies the one or more user interface actions to navigate to the given page (Coi: Abstract; ¶ 29, 30, 59, 60, etc. system operates to generate an action API, data structure etc. which effectively delineates a path necessary to compete an action, sequence of actions, etc. by navigation of a remote resource); (Kun: ¶ 43-49, 58, etc.; Fig 2A, 5, etc.: system effectively executes one or more steps comprising a sequence of actions leading to a desired result); (Ho: Abstract; ¶ 124 133-136, 146-148, etc.; Fig 7, etc.: system generates a logical form of a user input which executes semantic operations relevant thereto to navigate over a series of steps corresponding to the logical form of the user query). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 10
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 1, wherein: presenting the answer data as an answer to the natural language query comprises generating, with a large language model, a natural language answer with the answer data (Kun: ¶ 3, 36, etc.: such as by using an LLM embodied as, upon a chatbot). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 11
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 1, wherein the method further comprises:
extracting one or more input value indications from the natural language query
determining parameter input values based on the input value indications; and submitting the one or more parameter input value indications to the application (Coi: ¶ 14, 70: system receives a user voice query, extracts values not limited to a trigger, keyword, etc. and operable to resolve appropriate resources and query structures therefor); (Kun: Abstract; Fig 2A: a chatbot, LLM, etc. receives a user query operates thereon based on parameters thereof and returns response from database applications); (Ho: Abstract; ¶ 32, 37, 44, 125, 126, 133-136, 146-148; Fig 7: receipt of user query generates database query based on parameters, metadata associated with the query and suitable to generate a query structure for resolved knowledge graph resources and return a graph like data structure in response). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 12
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 11, wherein: determining parameter input values based on the input value indications comprises: with a large language model, choosing from among a list of candidate parameter input values based on the one or more input value indications (Coi: ¶ 14, 70: system receives a user voice query, extracts values not limited to a trigger, keyword, etc. and operable to resolve appropriate resources and query structures therefor); (Kun: Abstract; Fig 2A: a chatbot, LLM, etc. receives a user query operates thereon based on parameters thereof and returns response from database applications); (Ho: Abstract; ¶ 32, 37, 44, 125, 126, 133-136, 146-148; Fig 7: receipt of user query generates database query based on parameters, metadata associated with the query and suitable to generate a query structure for resolved knowledge graph resources and return a graph like data structure in response). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 13
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 12, wherein: the list of candidate parameter input values are fetched with an API call (Coi: ¶ 77, 78, 80, 82: action API generates action data corresponding to the trigger, keyword, etc. to cause performance of an action corresponding thereto); (Kun: Abstract; ¶ 29: LLM operates in concert with a user API). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 14
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 12, wherein: the list of candidate parameter input values are fetched from a list provided by the application (Coi: ¶ 77, 78, 80, 82: action API generates action data corresponding to the trigger, keyword, etc. to cause performance of an action corresponding thereto such as in concert with an intent manifest comprising a list of available links from which to fetch data along prescribed parameters); (Kun: Abstract; ¶ 29, 30, 33, etc.: LLM operates in concert with a user API in a manner enhanced by returning queried lists of parameters, traits, etc. ). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 15
Coi in view of Kun in view of Ho teaches or suggests:
The method of claim 12, wherein: the list of candidate parameter input values are prefetched before receiving the natural language query (Ho: ¶ 11, 125, etc. such as by operating in concert with a data structure associated with a pre trained language model). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 16
Coi teaches:
A computing system comprising: at least one hardware processor; at least one memory coupled to the at least one hardware processor (Coi: ¶ 172, etc.; Fig 1, 4, etc.);
a representation of a plurality of user interface pages of an application wherein the graph representation comprises nodes for the user interface pages and edges indicating how to navigate between the user interface pages (Coi: Abstract; ¶ 29, 30, 59, 60, etc.; system builds an intent manifest with mappings to particular pages, URL’s, links, etc.; navigates appropriately thereto, operates user interface elements thereof in accord therewith the pages are considered to correspond to nodes and the actions direct by the system are considered to correspond to edges; see claim 1 supra)
internal representations of user interface elements appearing in the user interface pages of the application, wherein the internal representations incorporate user interface element metadata (Coi: ¶ 75-78: action corresponding to, matching, etc. the user query generative of action data structure to cause an application upon a user device to perform the corresponding action wherein user interface element metadata comprise an internal representation incorporated into the action data structure);
wherein the system operates within an HTML context (Coi: ¶ 29, 103, etc.); and one or more non-transitory computer-readable media having stored therein computer-executable instructions that, when executed by the computing system, cause the computing system to perform (Coi: ¶ 172, etc.; Fig 1, 4, etc.):
receiving a natural language query (Coi: ¶ 14, 70: system receives a user voice query, identifies a trigger, keyword, etc.);
identifying an internal representation out of the internal representations of user interface elements as matching the natural language query, wherein the internal representation represents a terminus user interface element and incorporates user interface element metadata (Coi: ¶ 75-78: action corresponding to, matching, etc. the user query generative of action data structure to cause an application upon a user device to perform the corresponding action wherein user interface element metadata comprise an internal representation incorporated into the action data structure);
navigating to a terminus page out of the plurality of user interface pages of the application on which the terminus user interface element appears (Coi: ¶ 3, 9-11, 26, 30, 81, 107, 113, 117: the action API determines a type of action corresponding to a keyword, such as a navigation action, such as of an electronic resource and performs corresponding actions upon the electronic resource such as input of data to the electronic resource and resolution of responses thereto);
from the terminus page, extracting a value for the terminus user interface element; and presenting the value as an answer to the natural language query (Cio: Abstract; ¶ 3, 9-14: such as by executing actions by inputting relevant parameters such by populating fields or otherwise resolving a response based on user intent).
Coi does not explicitly teach the system operable of resolve a graph representation of a plurality of user interface pages of an application, wherein the graph representation comprises nodes for the user interface pages and edges indicating how to navigate between the user interface pages; such as using a large language model trained with HTML context.
In a related field of endeavor Kun teaches a system and method for receiving a natural language query associated with a user (Kun: Abstract; Fig 2A); determining a terminus user interface element out of a plurality of user interface elements of an application having a plurality of pages (Kun: Abstract, ¶ 13, 29, 42, etc.; Fig 2A, etc.: generative NL model forms, resolves, and augments a user query based on user ID, contextual data which is communicated to a recipient such as by a terminus user interface element on the client side of the system operable therefore), wherein determining the terminus user interface element comprises, with a large language model (Kun: ¶ 3, etc.: system operates based on an LLM chatbot framework); and presenting answer data to a user (Kun: ¶ 48, etc.: system provides natural language responses to the user query) wherein the model is considered trained with respect to HTML functionality as the model operates upon web page data, using a browser etc. (Kun: ¶ 22, 25, etc.). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize an LLM trained to operate with respect to an HTML implementation or context to generate an augmented query as taught or suggested by Kun and thereby improve the resolution of an answer by determining and using appropriate user interface elements in concert with the teachings of Coi and for at least the purpose of reifying the determined resources and interface elements thereof against an answer list and thereby improving, optimizing, etc. the operations of the LLM; one of ordinary skill in the art would have expected only predictable results therefrom.
Coi in view of Kun does not explicitly discuss the manner in which the LLM utilizes the internal representations such as using an embedding with integrated metadata to resolve a graph representation of a plurality of user interface pages of an application, wherein the graph representation comprises nodes for the user interface pages and edges indicating how to navigate between the user interface pages.
In a related field of endeavor Ho teaches a pretrained generative system and method (Ho: ¶ 126, 133, 152; Fig 7: sch as BERT type models for determining a database schema using an encoder (Ho: Abstract, etc.; Fig 7, etc.) wherein the schema incorporates the user query and metadata based thereon (Ho: Abstract; ¶ 32, 37, 44; Fig 7) to configure a generative model to determine, resolve, etc. a graph, tree, data structure, representation, model using embeddings therefor such as upon a knowledge graph and thereby output a best response to the user based on the input query, metadata, etc. (Ho: ¶ 4, 37, 125, 126, etc.); such that the system queries a knowledge graph and returns graph like structures, representations, etc. (Ho: Abstract; ¶ 32, 37, 44, 133-136, 146-148, etc.; Fig 7, etc.: relations between elements are considered to correspond to edges) wherein the graph representation comprises nodes for the user interface pages and edges indicating how to navigate between the user interface pages (Ho: Abstract; ¶ 32, 37, 44, 133-136, 146-148, etc.; Fig 7, etc.: elements upon a knowledge graph are considered to comprise nodes and relations between elements are considered to correspond to edges). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the Ho taught schema to improve the internal representations of the Coi in view of Kun system and method for at least the purpose of improving the response of the system to a user query, optimizing compute for doing so, better directing an answer to the user query, etc.; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 17
Coi in view of Kun in view of Ho teaches or suggests:
The system of claim 16, further comprising: a stored list of possible input values (Coi: ¶ 3, 9-14, 56-58, 70, 81, 116, 117: system receives a user voice query, extracts values not limited to a trigger, keyword, etc. and operable to resolve appropriate resources and query structures therefor; determining valid inputs such as available within an API or schema comprise resolving a list of valid parameters, values thereof); (Kun: Abstract; ¶ 13, 29, 42, etc.; Fig 2A: a chatbot, LLM, etc. receives a user query operates thereon based on parameters thereof and returns response from database applications; determining valid inputs such as available within an API or schema comprise resolving a list of valid parameters, values thereof); (Ho: Abstract; ¶ 32, 37, 44, 125, 126, 133-136, 146-148; Fig 7: receipt of user query generates database query based on parameters, metadata associated with the query and suitable to generate a query structure for resolved knowledge graph resources and return a graph like data structure in response; determining valid inputs such as available within an API or schema comprise resolving a list of valid parameters, values thereof); and an additional large language model configured to choose one of the possible input values based on an indication of an input value extracted from the natural language query (Ho: ¶ 125: a first model is a pre trained language model and a second is a relation aware transformer both of which are pliant to implementation on, with, etc. an LLM). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 18
Coi in view of Kun in view of Ho teaches or suggests:
The system of claim 16, wherein the one or more non-transitory computer-readable media have stored therein computer-executable instructions that, when executed by the computing system, cause the computing system to perform: with the large language model or another large language model, accepting the natural language query and the value as input and, based on a prompt to present an answer to the natural language query, outputting the value in a natural language format that answers the natural language query (Coi: ¶ 3, 9-14, 56-58, 70, 81, 116, 117); (Kun: Abstract; ¶ 13, 29, 42, etc.; Fig 2A); (Ho: Abstract; ¶ 32, 37, 44, 125, 126, 133-136, 146-148; Fig 7). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 19
Coi in view of Kun in view of Ho teaches or suggests:
The system of claim 16, further comprising: a stored indication of an API from which candidate parameter input values can be fetched (Coi: Abstract; ¶ 28-30, etc.); (Kun: ¶ 17, 24-26, etc.: query tool, large language models, etc. operative in concert with an API); (Ho: ¶ 4, etc.: Chatbot operative with an API). The claim is considered obvious over Coi as modified by Kun and Ho as addressed in the base claim as it would have been obvious to apply the further teaching of Coi, Kun, and/or Ho to the modified device of Coi, Kun, and Ho; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 20
Coi teaches:
One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed by a computing system, cause the computing system to perform operations (Coi: ¶ 172, etc.; Fig 1, 4, etc.) comprising:
receiving a natural language query (Coi: ¶ 14, 70: system receives a user voice query, identifies a trigger, keyword, etc.);
determining a terminus user interface element out of a plurality of user interface elements of an application having a plurality of pages (Coi: ¶ 75-78: action corresponding to, matching, etc. the user query generative of action data structure to cause an application upon a user device to perform the corresponding action wherein user interface element metadata comprise an internal representation incorporated into the action data structure);
wherein determining the terminus user interface element comprises finding a matching user interface element as the terminus user interface element via internal representations of the user interface elements, wherein user interface element metadata is incorporated into the internal representations of the user interface elements such as by using a model (Coi: ¶ 75-78: action corresponding to, matching, etc. the user query generative of action data structure to cause an application upon a user device to perform the corresponding action wherein user interface element metadata comprise an internal representation incorporated into the action data structure),
wherein the model performs semantic matching that goes beyond exact matches (Coi: such as in concert with an exact match, phrase match, broad match or other types of matching);
choosing a parameter input value from a plurality of candidate parameter input values based on an indication of an input value in the natural language query (Cio: Abstract; ¶ 3, 9-14, 75-78: such as by executing actions by determining, inputting, etc. of relevant parameters such by populating fields or otherwise resolving a response based on user intent such as determined by or resolved from the user query);
navigating to a given page out of the plurality of pages of the application on which the terminus user interface element appears (Coi: ¶ 3, 9-11, 26, 30, 81, 107, 113, 117: the action API determines a type of action corresponding to a keyword, such as a navigation action, such as of an electronic resource and performs corresponding actions upon the electronic resource such as input of data to the electronic resource and resolution of responses thereto);
from the given page, extracting determined parameter data values responsive to the query based on a state of the determined resource for the terminus user interface element (Cio: Abstract; ¶ 3, 9-14: such as by executing actions by determining, inputting, etc. of relevant parameters such by populating fields or otherwise resolving a response based on user intent);
presenting the answer data as an answer to the natural language query (Coi: ¶ 35, 67, 148: in this way a user voice input to the system resolves an answer based thereon).
Coi does not explicitly discuss determining the terminus user interface element comprises, with a large language model, from the given page, extracting answer data using, based on, etc. the large language model; choosing a parameter input value with a large language model from a plurality of candidate parameter input values based on an indication of an input value in the natural language query; and presenting the answer data as an answer to the natural language query. wherein the large language model performs semantic matching that goes beyond exact matches; with a large language model.
In a related field of endeavor Kun teaches a system and method for receiving a natural language query associated with a user (Kun: Abstract; Fig 2A); determining a terminus user interface element out of a plurality of user interface elements of an application having a plurality of pages (Kun: Abstract, ¶ 13, 29, 42, etc.; Fig 2A, etc.: generative NL model forms, resolves, and augments a user query based on user ID, contextual data which is communicated to a recipient such as by a terminus user interface element on the client side of the system operable therefore), choosing a parameter input value with a large language model from a plurality of candidate parameter input values based on an indication of an input value in the natural language query (Kun: 13, 24-26. 29, 42, 48; such as by utilizing the model in concert with an API); wherein determining the terminus user interface element comprises, with a large language model (Kun: ¶ 3, etc.: system operates based on an LLM chatbot framework) determining and presenting answer data to a user (Kun: ¶ 48, etc.: system provides natural language responses to the user query). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize an LLM to generate an augmented query as taught or suggested by Kun and thereby improve the resolution of an answer by determining and using appropriate user interface elements in concert with the teachings of Coi and for at least the purpose of reifying the determined resources and interface elements thereof against an answer list and thereby improving, optimizing, etc. the operations of the LLM; one of ordinary skill in the art would have expected only predictable results therefrom.
Coi in view of Kun does not explicitly discuss the manner in which the LLM utilizes the internal representations such as using an embedding with integrated metadata.
In a related field of endeavor Ho teaches a system and method for determining a database schema using an encoder (Ho: Abstract, etc.; Fig 7, etc.) wherein the schema incorporates the user query and metadata based thereon (Ho: Abstract; ¶ 32, 37, 44; Fig 7) to configure a generative model to determine a graph, tree, data structure model using embeddings therefor and thereby output a best response to the user based on the input query, metadata, etc. (Ho: ¶ 37, 125, 126, etc.). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the Ho taught schema to improve the internal representations of the Coi in view of Kun system and method for at least the purpose of improving the response of the system to a user query, optimizing compute for doing so, better directing an answer to the user query, etc.; one of ordinary skill in the art would have expected only predictable results therefrom.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL C MCCORD whose telephone number is (571)270-3701. The examiner can normally be reached 730-630 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CAROLYN EDWARDS can be reached at (571) 270-7136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL C MCCORD/Primary Examiner, Art Unit 2692
/CAROLYN R EDWARDS/Supervisory Patent Examiner, Art Unit 2692