Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of US Provisional Application No. 63/525,422 filed July 7, 2023 and US Provisional Application No. 63/552,791 filed February 13, 2024. Claims 1-6, 8-19, 21-22 have been afforded the benefit of the July 7, 2023 filing date.
Response to Arguments Regarding Claim Rejections - 35 USC § 102
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because
the new ground of rejection does not rely on any reference applied in the prior rejection of record for
any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim [1, 2, 9, 10, 15, 16] are rejected 35 U.S.C. 103 as being unpatentable over
Tunstall-Pedoe (US11977854 B2) in view of Zhang (CN115270988A).
Regarding claim 1, Tunstall-Pedoe discloses A method for generating a logical inference having enhanced consistency for at least a portion of content data of a language model, comprising:
selecting at least a portion of the content data of the language model;
[Column 7, lines 34-35 “(iv) inputting the natural language question, and the processor's answer to the question to the LLM”];
[Column 11, lines 23-32 “This disclosure includes a computer implemented method for the automated analysis or use of data, which comprises the steps of: (a) storing in a memory a structured, machine-readable representation of data that conforms to a machine-readable language (‘machine representation’); the machine representation including representations of user speech or text input to a human/machine interface; (b) automatically processing the machine representations to analyze the user speech or text input”]; [Column 35, lines 6-13 “Sources/methods for learning in examples of the present invention include: (a) learning from conversation or other natural language provided by users: by translating natural language provided by users in spoken or written form into UL and storing it, the concepts, ideas and knowledge represented in the stored UL is learned and can be utilized”]; [Column 81, lines 21-29 “LLMs can also extract assertions from large blocks of text with a suitable prompt. For example, the prompt below can be given to an LLM with suitable text to follow: Please create assertions that are assumed true according to the following text. The assertions should be in full sentences up to 6 words. Make as many assertions as possible. Each assertion starts on a new line without numbers”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”].
[Column 6, lines 25-26 “An advantage is that a trained LLM with training improved by the use of the training file is provided.”]
generating reasoning statements in the form of natural language relevant to the selected portion of the content data
[Column 11, lines 62-67, Column 12, lines 1-15- “According to a further aspect of the disclosure, there is provided a computer system including a processor and a memory, the processor configured to answer a question, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language, and wherein the processor is configured to answer the question using the reasoning steps, the computation units and the semantic nodes, and to store an answer to the question in the memory”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [Column 6 lines 27-38 “According to a thirteenth aspect of the invention, there is provided a computer-implemented method of generating a training file for a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in the training file. An advantage is that an improved LLM training file is generated”]; [Column 7 lines 30-41 “(iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM. An advantage is that an improved answer to the question may be provided by the LLM output”]; [Column 7 lines 42-67, Column 8 lines 1-11 “According to a seventeenth aspect of the invention, there is provided a computer-implemented method of improving output from an LLM, including the steps of (i) receiving a first natural language question; (ii) Inputting or providing the received first natural language question to a large language model (LLM); (iii) The large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested; (v) constructing a question which asks if the output is true, in which the question is represented in the processing language; (vi) inputting the question to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an answer to the question in the memory. An advantage is that the LLM output is checked for accuracy”]; [Column 8 lines 32-52 “ (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language mode”].
training the language model using the generated reasoning statements
[Column 4 lines 48-54 “According to a fourth aspect of the invention, there is provided a method of interacting with a LLM, including the step of training the LLM on the output from a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language. An advantage is that an improved LLM may be provided”]; [Column 6 lines 13-24 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [ Column 6 lines 39-54 “According to a fourteenth aspect of the invention, there is provided a computer-implemented method of re-training a large language model (LLM), the LLM having been previously trained using a training file, the method including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a re-training file; (iii) combining the training file and the re-training file, to generate a combined training file; (iv) using the combined training file to re-train the large language model (LLM); (v) storing weights characterizing the re-trained LLM”]; [Column 6 line 55 “An advantage is that an LLM with an improved training is provided”]. [Column 8 lines 12-52 “According to an eighteenth aspect of the invention, there is provided a computer-implemented method of fact checking output from a large language model (LLM), including the steps of (i) receiving a text input; (ii) inputting or providing the received text input to a large language model (LLM); (iii) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, wherein translating the output includes extracting the assertions in text generated by the LLM; (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language model”].
Specifically, this reference teaches extracting assertions from processed content, which corresponds to the claimed selecting and extracting of relevant information. The reference further teaches constructing one or more questions regarding the extracted assertions and generating corresponding answers using reasoning steps, semantic nodes, and computation units which corresponds to the claimed generating of reasoning statements. The reference also teaches storing the generated answers in a training file and using the training file to train a large language model, which corresponds to the claimed training the model based on generated question-answer pairs (reasoning statements).
However, Tunstall-Pedoe does not teach
upon receiving a prompt for information directed to the language model, determining at least one vector representation for at least a portion of the content data contained in the prompt; projecting the at least one vector representation into the common embedding space in which the vector representations for the generated reasoning statements are embedded; determining nearest neighbor content data for the at least one vector representation for the portion of the content data contained in the prompt in the common embedding space; and including the nearest neighbor content data in the prompt for information directed to the language model to generate a logical inference having enhanced consistency.
But Zhang teaches by determining respective vector representations for the generated reasoning statements and embedding the respective vector representations in a common embedding space
[Page 2, lines 22-25 “The knowledge base construction and update unit is used to build a knowledge base for retrieval. There are multiple instance phrases stored in the knowledge base, and each instance phrase is stored in the form of a key-value pair, wherein the key stores the embedding vector of the instance word, and the value stores the label truth value of the instance phrase;” where the embedding vector corresponds to the vector representations.];
[Page 4, lines 13-16 “There, specifically, each instance phrase is stored in the form of a key-value pair, where the key stores the embedding vector of the instance word, and the value stores the label truth value of the instance phrase. The embedding vector of the example phrase is obtained by learning the example text based on the prompt template through the pre-training language model”];
[Page 4, lines 18-21 “As shown in Figure 2, during each round of training, the first embedding vector of the masked word in the input instance text and its corresponding label true value form a new instance phrases, which are updated asynchronously into the knowledge base” where the shared vector knowledge space in which example vectors are stored and compared corresponds to the common embedding space.];
upon receiving a prompt for information directed to the language model, determining at least one vector representation for at least a portion of the content data contained in the prompt;
[Page 4, lines 24-27 “As shown in Figure 2, the classification model constructed by the embodiment includes a pre-trained language model, which is used to represent the knowledge of the input instance text, so as to extract the embedding vector of the mask position, specifically, the input instance text requires.. After serialization and transformation of the prompt template, the form of the prompt template is: [CLS]Instance text[MASK][SEP],..”];
[Page 4, lines 34-35 “In the embodiment, a pre-trained language model is used to extract the first embedding vector of the masked word in the input instance text, and the first embedding vector is used as the first query vector,”];
[Page 7, lines 36-37 “(1) Construct a prompt template to convert the input instance text. After the prompt template conversion, the input becomes “[CLS] This movie has no meaning! [MASK][SEP]”.” where prompt formatted input instance text corresponds to the claimed prompt, the extracted first embedding vector corresponds to the claimed vector representation for content contained in the prompt];
projecting the at least one vector representation into the common embedding space in which the vector representations for the generated reasoning statements are embedded;
[Page 4, lines 34-37 “In the embodiment, a pre-trained language model is used to extract the first embedding vector of the masked word in the input instance text, and the first embedding vector is used as the first query vector, and for each label category, the KNN (nearest neighbor node algorithm) is used to search Query the m instance phrases closest to the first query vector from the knowledge base as the first adjacent instance phrases,”];
[Page 5, lines 28-30 “The knowledge base construction and update unit is used to build a knowledge base for retrieval. There are multiple instance phrases stored in the knowledge base, and each instance phrase is stored in the form of a key-value pair, wherein the key stores the embedding vector of the instance word” Zhang uses prompt derived vector to search against the stored instance phrase vectors. This places the query vector and stored vectors in the same comparable embedding space.].
determining nearest neighbor content data for the at least one vector representation for the portion of the content data contained in the prompt in the common embedding space;
[Page 7, lines 41-43 “(3) Use the hidden state of the input instance text [MASK] in the last layer of the language model as a query vector to retrieve the nearest neighbor instance phrase from the knowledge base, and calculate the category correlation probability PKNN(yi|qt) based on the instance phrase,”];
[Page 6, lines 29-30 “The non-parametric method KNN is used to retrieve instance phrases adjacent to the input instance text from the knowledge base,”];
[Page 4, lines 36-37 “…the KNN (nearest neighbor node algorithm) is used to search Query the m instance phrases closest to the first query vector from the knowledge base as the first adjacent instance phrases,”].
and including the nearest neighbor content data in the prompt for information directed to the language model to generate a logical inference having enhanced consistency.
[Page 4, lines 37-39 “…, these first adjacent instance phrases are used as additional example input, and the aggregation result obtained by aggregating with the first query vector is used as pre-training The input data of the language model,”];
[Page 4, lines 46-48 “The obtained aggregation results, which are used as input data combined with instance phrases from the knowledge base, are used as context enhancement information to guide the classification model training and correct the classification model predictions,”];
[Page 7, lines 38-40 “(2) Use the pre-trained language model to obtain the input instance text [MASK] position in the embedding vector, retrieve the neural example from the knowledge base, and input the instance text at the [MASK] position in the embedding vector for splicing and aggregation before inputting it into the pre-training language model” where retrieving neighboring examples, combining them with the prompt derived input and then supplying the combined information to the pretrained language model as context input which is then used to improve the model’s prediction];
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine teachings of Tunstall-Pedoe into the teachings of Zhang’s vector embedding and similarity retrieval technique so that the generated reasoning statements are represented in a common embedding space and the most relevant statement is retrieved and included with a prompt. Such a modification would improved retrieval of logically relevant training information and thereby enhance the consistency and accuracy of the language model’s inference.
Regarding claim 2, Tunstall-Pedoe discloses the method of claim 1, wherein the reasoning statements comprise at least one of logically related statements or chain of thought reasoning statements identifying properties of the selected portion of the content data – [ Column 8 lines 12-52 “According to an eighteenth aspect of the invention, there is provided a computer-implemented method of fact checking output from a large language model (LLM), including the steps of (i) receiving a text input; (ii) inputting or providing the received text input to a large language model (LLM); (iii) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, wherein translating the output includes extracting the assertions in text generated by the LLM; (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language;
(vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language model”]; [Column 22, lines 39-45 “Reasoning is where UL is generated from other UL. A reasoning passage is a bit of UL that represents how new UL can be generated from other UL—for example a logical consequence or giving meaning to other nodes or a combination of nodes. e.g. the English “if something originates from France then it is French” translates to a reasoning passage in UL”];
[Column 22, lines 46-52 “Reasoning steps are represented as passages which represent the semantics of the step. Note that in a preferred example reasoning passages are represented in UL like anything else. There is no special syntax or content that extends or changes UL to support reasoning. For example: (ConsequenceOf (IsA X (Cheese Hard)) ((Not HasAttribute) X Creamy))”].
Specifically, this reference teaches constructing questions from extracted assertions and answering those questions using reasoning steps, thereby generating logically related question-answer pairs (reasoning statements) based on the extracted statement.
Regarding claim 9, Tunstall-Pedoe does teach
an apparatus for generating a logical inference having enhanced consistency for at least a portion of content data a language model
comprising a processor and a memory accessible to the processor, the memory having stored therein at least one of programs or instructions executable by the processor to configure the apparatus to – [Column 7, lines 14-41 “ii) using a computer system including a processor and a memory, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the natural language question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM. An advantage is that an improved answer to the question may be provided by the LLM output.
select at least a portion of the content data of the language model –[ Column 11, lines 23-32 “This disclosure includes a computer implemented method for the automated analysis or use of data, which comprises the steps of: (a) storing in a memory a structured, machine-readable representation of data that conforms to a machine-readable language (‘machine representation’); the machine representation including representations of user speech or text input to a human/machine interface; (b) automatically processing the machine representations to analyze the user speech or text input”]; [Column 35, lines 6-13 “Sources/methods for learning in examples of the present invention include: (a) learning from conversation or other natural language provided by users: by translating natural language provided by users in spoken or written form into UL and storing it, the concepts, ideas and knowledge represented in the stored UL is learned and can be utilized”]; [Column 81 lines 22-29 “LLMs can also extract assertions from large blocks of text with a suitable prompt. For example, the prompt below can be given to an LLM with suitable text to follow: Please create assertions that are assumed true according to the following text. The assertions should be in full sentences up to 6 words. Make as many assertions as possible. Each assertion starts on a new line without numbers”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”].
generate reasoning statements in the form of natural language relevant to the selected portion of the content data – [Column 11 Lines 62-67, Column 12 lines 1-15 “According to a further aspect of the disclosure, there is provided a computer system including a processor and a memory, the processor configured to answer a question, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language, and wherein the processor is configured to answer the question using the reasoning steps, the computation units and the semantic nodes, and to store an answer to the question in the memory”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [Column 6 lines 27-38 “According to a thirteenth aspect of the invention, there is provided a computer-implemented method of generating a training file for a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in the training file. An advantage is that an improved LLM training file is generated”]; [Column 7 lines 31-41 “(iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM. An advantage is that an improved answer to the question may be provided by the LLM output”]; [Column 7 lines 41-67, Column 8 lines 1-11 “According to a seventeenth aspect of the invention, there is provided a computer-implemented method of improving output from an LLM, including the steps of (i) receiving a first natural language question; (ii) Inputting or providing the received first natural language question to a large language model (LLM); (iii) The large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested; (v) constructing a question which asks if the output is true, in which the question is represented in the processing language; (vi) inputting the question to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an answer to the question in the memory. An advantage is that the LLM output is checked for accuracy”]; [Column 8 lines 33-51 “(v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language mode”].
train the language model using the generated reasoning statements by training the LLM on the output from a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language. An advantage is that an improved LLM may be provided”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [Column 6 lines 39-54 “According to a fourteenth aspect of the invention, there is provided a computer-implemented method of re-training a large language model (LLM), the LLM having been previously trained using a training file, the method including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a re-training file; (iii) combining the training file and the re-training file, to generate a combined training file; (iv) using the combined training file to re-train the large language model (LLM); (v) storing weights characterizing the re-trained LLM”]; [Column 6 line 55 “An advantage is that an LLM with an improved training is provided”]. [Column 8 lines 12-51 “According to an eighteenth aspect of the invention, there is provided a computer-implemented method of fact checking output from a large language model (LLM), including the steps of (i) receiving a text input; (ii) inputting or providing the received text input to a large language model (LLM); (iii) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, wherein translating the output includes extracting the assertions in text generated by the LLM; (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language model”].
However, Tunstall-Pedoe does not teach determine at least one vector representation for at least a portion of the content data contained in the prompt; project the at least one vector representation into the common embedding space in which the vector representations for the generated reasoning statements are embedded; determine nearest neighbor content data for the at least one vector representation for the portion of the content data contained in the prompt in the common embedding space; and include the nearest neighbor content data in the prompt for information directed to the language model to generate a logical inference having enhanced consistency.
But Zhang teaches by determining respective vector representations for the generated reasoning statements and embedding the respective vector representations in a common embedding space
[Page 2, lines 22-25 “The knowledge base construction and update unit is used to build a knowledge base for retrieval. There are multiple instance phrases stored in the knowledge base, and each instance phrase is stored in the form of a key-value pair, wherein the key stores the embedding vector of the instance word, and the value stores the label truth value of the instance phrase;” where the embedding vector corresponds to the vector representations.];
[Page 4, lines 13-16 “There, specifically, each instance phrase is stored in the form of a key-value pair, where the key stores the embedding vector of the instance word, and the value stores the label truth value of the instance phrase. The embedding vector of the example phrase is obtained by learning the example text based on the prompt template through the pre-training language model”];
[Page 4, lines 18-21 “As shown in Figure 2, during each round of training, the first embedding vector of the masked word in the input instance text and its corresponding label true value form a new instance phrases, which are updated asynchronously into the knowledge base” where the shared vector knowledge space in which example vectors are stored and compared corresponds to the common embedding space.];
determine at least one vector representation for at least a portion of the content data contained in the prompt;
[Page 4, lines 24-27 “As shown in Figure 2, the classification model constructed by the embodiment includes a pre-trained language model, which is used to represent the knowledge of the input instance text, so as to extract the embedding vector of the mask position, specifically, the input instance text requires.. After serialization and transformation of the prompt template, the form of the prompt template is: [CLS]Instance text[MASK][SEP],..”];
[Page 4, lines 34-35 “In the embodiment, a pre-trained language model is used to extract the first embedding vector of the masked word in the input instance text, and the first embedding vector is used as the first query vector,”];
[Page 7, lines 36-37 “(1) Construct a prompt template to convert the input instance text. After the prompt template conversion, the input becomes “[CLS] This movie has no meaning! [MASK][SEP]”.” where prompt formatted input instance text corresponds to the claimed prompt, the extracted first embedding vector corresponds to the claimed vector representation for content contained in the prompt];
project the at least one vector representation into the common embedding space in which the vector representations for the generated reasoning statements are embedded;
[Page 4, lines 34-37 “In the embodiment, a pre-trained language model is used to extract the first embedding vector of the masked word in the input instance text, and the first embedding vector is used as the first query vector, and for each label category, the KNN (nearest neighbor node algorithm) is used to search Query the m instance phrases closest to the first query vector from the knowledge base as the first adjacent instance phrases,”];
[Page 5, lines 28-30 “The knowledge base construction and update unit is used to build a knowledge base for retrieval. There are multiple instance phrases stored in the knowledge base, and each instance phrase is stored in the form of a key-value pair, wherein the key stores the embedding vector of the instance word” Zhang uses prompt derived vector to search against the stored instance phrase vectors. This places the query vector and stored vectors in the same comparable embedding space.].
determine nearest neighbor content data for the at least one vector representation for the portion of the content data contained in the prompt in the common embedding space;
[Page 7, lines 41-43 “(3) Use the hidden state of the input instance text [MASK] in the last layer of the language model as a query vector to retrieve the nearest neighbor instance phrase from the knowledge base, and calculate the category correlation probability PKNN(yi|qt) based on the instance phrase,”];
[Page 6, lines 29-30 “The non-parametric method KNN is used to retrieve instance phrases adjacent to the input instance text from the knowledge base,”];
[Page 4, lines 36-37 “…the KNN (nearest neighbor node algorithm) is used to search Query the m instance phrases closest to the first query vector from the knowledge base as the first adjacent instance phrases,”].
and include the nearest neighbor content data in the prompt for information directed to the language model to generate a logical inference having enhanced consistency.
[Page 4, lines 37-39 “…, these first adjacent instance phrases are used as additional example input, and the aggregation result obtained by aggregating with the first query vector is used as pre-training The input data of the language model,”];
[Page 4, lines 46-48 “The obtained aggregation results, which are used as input data combined with instance phrases from the knowledge base, are used as context enhancement information to guide the classification model training and correct the classification model predictions,”];
[Page 7, lines 38-40 “(2) Use the pre-trained language model to obtain the input instance text [MASK] position in the embedding vector, retrieve the neural example from the knowledge base, and input the instance text at the [MASK] position in the embedding vector for splicing and aggregation before inputting it into the pre-training language model” where retrieving neighboring examples, combining them with the prompt derived input and then supplying the combined information to the pretrained language model as context input which is then used to improve the model’s prediction];
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine teachings of Tunstall-Pedoe into the teachings of Zhang’s vector embedding and similarity retrieval technique so that the generated reasoning statements are represented in a common embedding space and the most relevant statement is retrieved and included with a prompt. Such a modification would improved retrieval of logically relevant training information and thereby enhance the consistency and accuracy of the language model’s inference.
Regarding Claim 10, Tunstall-Pedoe does teach the apparatus of claim 9, wherein the reasoning statements comprise at least one of logically related statements or chain of thought reasoning statements identifying properties of the selected portion of the content data. Claim 10 is rejected for the same reasons as claim 2.
Regarding Claim 15, Tunstall-Pedoe does teach a non-transitory computer readable medium having stored thereon at least one program, the at least one program including instructions which, when executed by a processor, cause the processor to perform a method for generating a logical inference having enhanced consistency for at least a portion of content data of a language model, comprising:– [Column 4 lines 16-23 “According to a first aspect of the invention, there is provided a method of interacting with a LLM, including the step of a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, to provide new context data for the LLM, in order to improve the output, such as continuation text output, generated by the LLM in response to a prompt”]
selecting at least a portion of the content data of the language model –[ Column 11 lines 23-32 “This disclosure includes a computer implemented method for the automated analysis or use of data, which comprises the steps of: (a) storing in a memory a structured, machine-readable representation of data that conforms to a machine-readable language (‘machine representation’); the machine representation including representations of user speech or text input to a human/machine interface; (b) automatically processing the machine representations to analyze the user speech or text input”]; [Column 35, lines 6-13 “Sources/methods for learning in examples of the present invention include: (a) learning from conversation or other natural language provided by users: by translating natural language provided by users in spoken or written form into UL and storing it, the concepts, ideas and knowledge represented in the stored UL is learned and can be utilized”]; [Column 81, lines 21-29 “LLMs can also extract assertions from large blocks of text with a suitable prompt. For example, the prompt below can be given to an LLM with suitable text to follow: Please create assertions that are assumed true according to the following text. The assertions should be in full sentences up to 6 words. Make as many assertions as possible. Each assertion starts on a new line without numbers”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”].
generating reasoning statements in the form of natural language relevant to the selected portion of the content data – [Column 11, lines 62-67, Column 12, lines 1-15 “According to a further aspect of the disclosure, there is provided a computer system including a processor and a memory, the processor configured to answer a question, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language, and wherein the processor is configured to answer the question using the reasoning steps, the computation units and the semantic nodes, and to store an answer to the question in the memory”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [Column 6 lines 27-38 “According to a thirteenth aspect of the invention, there is provided a computer-implemented method of generating a training file for a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in the training file. An advantage is that an improved LLM training file is generated”]; [Column 7 lines 30-41 “(iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM. An advantage is that an improved answer to the question may be provided by the LLM output”]; [Column 7 lines 42-67, Column 8 lines 1-11 “According to a seventeenth aspect of the invention, there is provided a computer-implemented method of improving output from an LLM, including the steps of (i) receiving a first natural language question; (ii) Inputting or providing the received first natural language question to a large language model (LLM); (iii) The large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested; (v) constructing a question which asks if the output is true, in which the question is represented in the processing language; (vi) inputting the question to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an answer to the question in the memory. An advantage is that the LLM output is checked for accuracy”]; [Column 8 lines 32-52 “ (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language mode”].
training the language model using the generated reasoning statements by training the LLM on the output from a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language. An advantage is that an improved LLM may be provided”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [Column 6 lines 39-54 “According to a fourteenth aspect of the invention, there is provided a computer-implemented method of re-training a large language model (LLM), the LLM having been previously trained using a training file, the method including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a re-training file; (iii) combining the training file and the re-training file, to generate a combined training file; (iv) using the combined training file to re-train the large language model (LLM); (v) storing weights characterizing the re-trained LLM”]; [Column 6 lines 55 “An advantage is that an LLM with an improved training is provided”]. [Column 8 lines 12-52 “According to an eighteenth aspect of the invention, there is provided a computer-implemented method of fact checking output from a large language model (LLM), including the steps of (i) receiving a text input; (ii) inputting or providing the received text input to a large language model (LLM); (iii) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, wherein translating the output includes extracting the assertions in text generated by the LLM; (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language model”].
However, Tunstall-Pedoe does not teach training the language model using the generated reasoning statements by determining respective vector representations for the generated reasoning statements and embedding the respective vector representations in a common embedding space;
upon receiving a prompt for information directed to the language model, determining at least one vector representation for at least a portion of the content data contained in the prompt; projecting the at least one vector representation into the common embedding space in which the vector representations for the generated reasoning statements are embedded; determining nearest neighbor content data for the at least one vector representation for the portion of the content data contained in the prompt in the common embedding space; and including the nearest neighbor content data in the prompt for information directed to the language model to generate a logical inference having enhanced consistency.
But Zhang teaches by determining respective vector representations for the generated reasoning statements and embedding the respective vector representations in a common embedding space
[Page 2, lines 22-25 “The knowledge base construction and update unit is used to build a knowledge base for retrieval. There are multiple instance phrases stored in the knowledge base, and each instance phrase is stored in the form of a key-value pair, wherein the key stores the embedding vector of the instance word, and the value stores the label truth value of the instance phrase;” where the embedding vector corresponds to the vector representations.];
[Page 4, lines 13-16 “There, specifically, each instance phrase is stored in the form of a key-value pair, where the key stores the embedding vector of the instance word, and the value stores the label truth value of the instance phrase. The embedding vector of the example phrase is obtained by learning the example text based on the prompt template through the pre-training language model”];
[Page 4, lines 18-21 “As shown in Figure 2, during each round of training, the first embedding vector of the masked word in the input instance text and its corresponding label true value form a new instance phrases, which are updated asynchronously into the knowledge base” where the shared vector knowledge space in which example vectors are stored and compared corresponds to the common embedding space.];
upon receiving a prompt for information directed to the language model, determining at least one vector representation for at least a portion of the content data contained in the prompt;
[Page 4, lines 24-27 “As shown in Figure 2, the classification model constructed by the embodiment includes a pre-trained language model, which is used to represent the knowledge of the input instance text, so as to extract the embedding vector of the mask position, specifically, the input instance text requires.. After serialization and transformation of the prompt template, the form of the prompt template is: [CLS]Instance text[MASK][SEP],..”];
[Page 4, lines 34-35 “In the embodiment, a pre-trained language model is used to extract the first embedding vector of the masked word in the input instance text, and the first embedding vector is used as the first query vector,”];
[Page 7, lines 36-37 “(1) Construct a prompt template to convert the input instance text. After the prompt template conversion, the input becomes “[CLS] This movie has no meaning! [MASK][SEP]”.” where prompt formatted input instance text corresponds to the claimed prompt, the extracted first embedding vector corresponds to the claimed vector representation for content contained in the prompt];
projecting the at least one vector representation into the common embedding space in which the vector representations for the generated reasoning statements are embedded;
[Page 4, lines 34-37 “In the embodiment, a pre-trained language model is used to extract the first embedding vector of the masked word in the input instance text, and the first embedding vector is used as the first query vector, and for each label category, the KNN (nearest neighbor node algorithm) is used to search Query the m instance phrases closest to the first query vector from the knowledge base as the first adjacent instance phrases,”];
[Page 5, lines 28-30 “The knowledge base construction and update unit is used to build a knowledge base for retrieval. There are multiple instance phrases stored in the knowledge base, and each instance phrase is stored in the form of a key-value pair, wherein the key stores the embedding vector of the instance word” Zhang uses prompt derived vector to search against the stored instance phrase vectors. This places the query vector and stored vectors in the same comparable embedding space.].
determining nearest neighbor content data for the at least one vector representation for the portion of the content data contained in the prompt in the common embedding space;
[Page 7, lines 41-43 “(3) Use the hidden state of the input instance text [MASK] in the last layer of the language model as a query vector to retrieve the nearest neighbor instance phrase from the knowledge base, and calculate the category correlation probability PKNN(yi|qt) based on the instance phrase,”];
[Page 6, lines 29-30 “The non-parametric method KNN is used to retrieve instance phrases adjacent to the input instance text from the knowledge base,”];
[Page 4, lines 36-37 “…the KNN (nearest neighbor node algorithm) is used to search Query the m instance phrases closest to the first query vector from the knowledge base as the first adjacent instance phrases,”].
and including the nearest neighbor content data in the prompt for information directed to the language model to generate a logical inference having enhanced consistency.
[Page 4, lines 37-39 “…, these first adjacent instance phrases are used as additional example input, and the aggregation result obtained by aggregating with the first query vector is used as pre-training The input data of the language model,”];
[Page 4, lines 46-48 “The obtained aggregation results, which are used as input data combined with instance phrases from the knowledge base, are used as context enhancement information to guide the classification model training and correct the classification model predictions,”];
[Page 7, lines 38-40 “(2) Use the pre-trained language model to obtain the input instance text [MASK] position in the embedding vector, retrieve the neural example from the knowledge base, and input the instance text at the [MASK] position in the embedding vector for splicing and aggregation before inputting it into the pre-training language model” where retrieving neighboring examples, combining them with the prompt derived input and then supplying the combined information to the pretrained language model as context input which is then used to improve the model’s prediction];
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine teachings of Tunstall-Pedoe into the teachings of Zhang’s vector embedding and similarity retrieval technique so that the generated reasoning statements are represented in a common embedding space and the most relevant statement is retrieved and included with a prompt. Such a modification would improved retrieval of logically relevant training information and thereby enhance the consistency and accuracy of the language model’s inference.
Regarding claim 16, Tunstall-Pedoe does teach the non-transitory computer readable medium of claim 15, wherein the reasoning statements comprise at least one of logically related statements or chain of thought reasoning statements identifying properties of the selected portion of the content data. Claim 16 is rejected for the same reasons as claim 2.
Claim [3, 5, 11, 13, 17, 18] are rejected under 35 U.S.C. 103 as being unpatentable over Tunstall-Pedoe (US11977854 B2) in view of Zhang (CN115270988A) and in further view of Heck (US 20090162824 A1).
Regarding claim 3, Tunstall-Pedoe teaches the method of claim 1, wherein the reasoning statements are generated [by a natural language processing computer] reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language, and wherein the processor is configured to answer the question using the reasoning steps, the computation units and the semantic nodes, and to store an answer to the question in the memory”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [Column 6 lines 27-38 “According to a thirteenth aspect of the invention, there is provided a computer-implemented method of generating a training file for a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in the training file. An advantage is that an improved LLM training file is generated”]; [Column 7 lines 30-41 “(iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM. An advantage is that an improved answer to the question may be provided by the LLM output”]; [Column 7 lines 42-67, Column 8 lines 1-11 “According to a seventeenth aspect of the invention, there is provided a computer-implemented method of improving output from an LLM, including the steps of (i) receiving a first natural language question; (ii) Inputting or providing the received first natural language question to a large language model (LLM); (iii) The large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested; (v) constructing a question which asks if the output is true, in which the question is represented in the processing language; (vi) inputting the question to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an answer to the question in the memory. An advantage is that the LLM output is checked for accuracy”]; [Column 8 lines 32-52 “(v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language mode”].
However, Tunstall-Pedoe in view of Zhang does not teach reasoning statement being generated by at least one of a human or a machine learning model.
But Heck teaches reasoning statements generated by a human –[Page 2, 0022 “FIG. 1 illustrates an exemplary QA robot system 100 for receiving questions, generating answers to those questions, and displaying the answers to the users. Moreover, system 100 also collects feedback on its answers and uses that feedback to refine its ability to answer subsequent questions. System 100, in one embodiment, includes question analyzer 110, QA answer component 120, confidence engine 130, decision maker 140, adjudicator 150, and feedback analyzer 160. In other embodiments, system 100 may include a different set of tools and components. Each of the components of system 100 is discussed below, but first a few commonly used terms are discussed”]; [Page 2, 0023 “ A "question" as used herein is a query submitted by a user to a QA robot. In one embodiment, the question can be in a natural language format, (e.g., the format a person would typically ask a question). Example questions include "what is your name?", "why is the sky blue?", "how many teeth does a shark have?", etc. In other embodiments, a question can be a keyword string, like a search query (e.g., "movies+`James Dean`"). Questions can include requests for a wide variety of data. Some of the types of data a question may request include: (1) informational data, (2) subjective information, (3) localized information, (4) timely information, and (5) search engine data. In other embodiments, questions may request other types of data”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine teachings of Tunstall-Pedoe in view of Zhang into the teachings of Heck because incorporating the human generated question-answer pairs would provide an additional source of accurate or validated question-answer pairs for the training process, thereby improving the quality of the training data and enhancing the performance and reliability of the trained language model.
Regarding claim 5, Tunstall-Pedoe in view of Zhang do not teach the method of claim 1, further comprising: receiving at least one prompt originating from a human intended for the language model; generating an inference in response to the at least one prompt using the language model; receiving information from the human indicating if the generated inference is within a threshold of a target inference of a response to the at least one prompt; and if the generated inference is not within the threshold, providing training data to the language model to train the language model to generate an inference that is within the threshold of the target inference.
However, Heck does teach receiving at least one prompt originating from a human intended for the language model – [Page 2 0022 “FIG. 1 illustrates an exemplary QA robot system 100 for receiving questions, generating answers to those questions, and displaying the answers to the users. Moreover, system 100 also collects feedback on its answers and uses that feedback to refine its ability to answer subsequent questions. System 100, in one embodiment, includes question analyzer 110, QA answer component 120, confidence engine 130, decision maker 140, adjudicator 150, and feedback analyzer 160. In other embodiments, system 100 may include a different set of tools and components. Each of the components of system 100 is discussed below, but first a few commonly used terms are discussed”]; [0023 “A "question" as used herein is a query submitted by a user to a QA robot. In one embodiment, the question can be in a natural language format, (e.g., the format a person would typically ask a question). Example questions include "what is your name?", "why is the sky blue?", "how many teeth does a shark have?", etc. In other embodiments, a question can be a keyword string, like a search query (e.g., "movies+`James Dean`"). Questions can include requests for a wide variety of data. Some of the types of data a question may request include: (1) informational data, (2) subjective information, (3) localized information, (4) timely information, and (5) search engine data. In other embodiments, questions may request other types of data”].
generating an inference in response to the at least one prompt using the language model - [Page 3 0030 "Answers" as used herein refers to the information that is presented to a user in response to a question. Answers can consist of the types of information described above. Answers are derived by a QA robot in a variety of ways”]; [Page 3 0031 “One way to teach the QA robot how to answer questions is to boot it into an initial training mode. According to one embodiment, the QA robot can then be populated with test questions and answers, archived questions and answers from a social network, and information from other sources. The QA robot uses those sources of information to learn. For example, a social network may already have archives of questions and answers that can be fed to the QA robot. In one embodiment, the QA robot stores the questions and their associated answers directly into its knowledgebase and retrieves that information when similar questions are subsequently asked. In one embodiment, this training may be supervised by people to ensure that the answers to a question are correct and that answers are being stored and indexed properly”]. [Page 3 0032 “Another approach QA robot may use to learn how to generate answers to questions is to observe users (particularly expert users) as they respond to questions on a social network. For example, suppose a user posts the question "where can I buy good Indian food in Portland, Oreg.?" Users familiar with the area may respond to the question listing some of their favorite Indian restaurants. The QA robot captures the question and the posted answers. The captured information is analyzed to determine how often a particular restaurant is listed among the answers. If a restaurant is listed several times by several different users, the QA robot captures that information and can deduce that that particular restaurant may be a good answer to the question”].
receiving information from the human indicating if the generated inference is within a threshold of a target inference of a response to the at least one prompt - [Page 3 0033 “Moreover, if the user who posted the question later returns and indicates (e.g., by giving a thumb up or down to the answer, rating the answer on a scale of 1-10, etc.) that the suggestion was wonderful, then the QA robot gains even more confidence in the answer so that the next time a user asks for good Indian food in Portland the QA robot can retrieve and display the listed restaurant. As an additional benefit, the QA robot can give credibility points to the users that answered the question so that their answers (both previous and subsequent ones) are given more weight in later analyses of questions”].
and if the generated inference is not within the threshold, providing training data to the language model to train the language model to generate an inference that is within the threshold of the target inference - [Page 3 0031 “One way to teach the QA robot how to answer questions is to boot it into an initial training mode. According to one embodiment, the QA robot can then be populated with test questions and answers, archived questions and answers from a social network, and information from other sources. The QA robot uses those sources of information to learn. For example, a social network may already have archives of questions and answers that can be fed to the QA robot. In one embodiment, the QA robot stores the questions and their associated answers directly into its knowledgebase and retrieves that information when similar questions are subsequently asked. In one embodiment, this training may be supervised by people to ensure that the answers to a question are correct and that answers are being stored and indexed properly”]; [Page 3 0034 “Similarly, if the user who posted the question returns and indicates he hated the suggestions, then the QA robot takes that into consideration in determining how to answer subsequent questions. For example, the QA robot may take credibility points away from the users that recommended the restaurant (and give less weight to their other answers). Especially, if other users chime in later saying they did not like the restaurant. Once enough users express dissatisfaction with the restaurant, the QA robot may add the restaurant to a list of restaurants to avoid. Then if a subsequent user asks what Indian restaurants to avoid, the QA robot has at least one answer available. In this way, over time, the QA robot can learn how to respond to question”]; [ Page 6 0074 “Moreover, the threshold may change as system 100 gains a wider body of knowledge. Initially, system 100 may want to post high precision answers on a very select number of questions while it learns the best format to answer questions and builds its knowledgebases. Then as system 100 learns and adapts, the threshold may change, particularly, as system 100 identifies what a good answer is and what users expect in an answer”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine teachings of Tunstall-Pedoe in view of Zhang into the teachings of Heck because such a combination would leverage known human correction techniques to improve machine learning training data in the threshold based framework, resulting in predictable improvements in model accuracy.
Regarding Claim 11, Tunstall-Pedoe in view of Zhang in view of Heck does teach the apparatus of claim 9, wherein the reasoning statements are generated by at least one of a human or a machine learning model. Claim 11 is rejected for the same reasons as claim 3.
Regarding claim 13, Tunstall-Pedoe in view of Zhang in view of Heck teach the apparatus of claim 9, wherein the apparatus is further configured to: receive at least one prompt originating from a human and intended for the language model; generate an inference in response to the at least one prompt using the language model; receive information from the human indicating if the generated inference is within a threshold of a target inference of a response to the at least one prompt; and if the generated inference is not within the threshold, provide training data to the language model to train the language model to generate an inference that is within the threshold of the target inference. Claim 13 is rejected for the same reasons as claim 5.
Regarding claim 17, Tunstall-Pedoe in view of Zhang in view of Heck does teach the non-transitory computer readable medium of claim 15, wherein the reasoning statements are generated by at least one of a human or a machine learning model. Claim 17 is rejected for the same reasons as claim 3.
Regrading claim 18, Tunstall-Pedoe in view of Zhang in view of Heck does teach the non-transitory computer readable medium of claim 15, further comprising: receiving at least one prompt originating from a human and intended for the language model; generating an inference in response to the at least one prompt using the language model; receiving information from the human indicating if the generated inference is within a threshold of a target inference of a response to the at least one prompt; and if the generated inference is not within the threshold, providing training data to the language model to train the language model to generate an inference that is within the threshold of the target inference.
Claim 18 is rejected for the same reasons as claim 5.
Claim [8 ] are rejected under 35 U.S.C. 103 as being unpatentable over Tunstall-Pedoe (US11977854 B2) in view of Heck (US 20090162824 A1) and in further view of Boguraev (US 10614725 B2).
Regarding claim 8, Tunstall-Pedoe teaches
A method for training a language model for enhanced consistency for at least a portion of content data of the language model, comprising:
Tunstall-Pedoe teaches training a language model for enhanced consistency for at least a portion of content data of the language model, – [Column 11, lines 62-67, Column 12, lines 1-15 “According to a further aspect of the disclosure, there is provided a computer system including a processor and a memory, the processor configured to answer a question, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language, and wherein the processor is configured to answer the question using the reasoning steps, the computation units and the semantic nodes, and to store an answer to the question in the memory”]; [Column 6 lines 13-23 “According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM”]; [Column 6 lines 27-38 “According to a thirteenth aspect of the invention, there is provided a computer-implemented method of generating a training file for a large language model (LLM), including the steps of (i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in the training file. An advantage is that an improved LLM training file is generated”]; [Column 7 lines 30-41 “(iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM. An advantage is that an improved answer to the question may be provided by the LLM output”]; [Column 7 lines 42-67, Column 8 lines 1-11 “According to a seventeenth aspect of the invention, there is provided a computer-implemented method of improving output from an LLM, including the steps of (i) receiving a first natural language question; (ii) Inputting or providing the received first natural language question to a large language model (LLM); (iii) The large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested; (v) constructing a question which asks if the output is true, in which the question is represented in the processing language; (vi) inputting the question to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an answer to the question in the memory. An advantage is that the LLM output is checked for accuracy”]; [Column 8 lines 32-52 “ (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language mode”].
generating a first query for information directed to at least the portion of the content data;
[Column 8, lines 30-31 “translating the output includes extracting the assertions in text generated by the LLM;”];
[Column 8, lines 32-35 “constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language;” where the question asking whether an extracted assertion is true corresponds to a query for information directed to the portion of content containing that assertion];
determining, using the language model, a logical inference in response to the first query for information directed to at least the portion of the content data
[Column 7, lines 30-38 “(iii) the processor answering the question using the reasoning
steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM;”];
[Column 7, lines 40-41 “An advantage is that an improved answer to the question 40
may be provided by the LLM output.”].
However, Tunstall-Pedoe does not teach
determining, using the language model, respective logical inferences in response to the additional natural language queries for information; wherein the language model is trained to be self-consistent about a knowledge of at least the portion of the content data based on the logical inferences determined for the first query for information and the additional queries for information directed to at least the portion of the content data
such that a logical inference of the trained language model in response to a subsequently received query for information is increased as compared with the logical inference of the language model in response to a same or similar query for information before the training of the language model.
Heck teaches
determining, using the language model, respective logical inferences in response to the additional natural language queries for information; wherein the language model is trained to be self-consistent about a knowledge of at least the portion of the content data based on the logical inferences determined for the first query for information and the additional queries for information directed to at least the portion of the content data
[0023 “A “question” as used herein is a query submitted by a user to a QA robot. In one embodiment, the question can be in a natural language format,..”];
[0020 “In one embodiment, a QA robot includes software tools to analyze questions submitted by users, to find the most “correct” answer to the questions, and to display the answers to the users.”];
[0031 “According to one embodiment, the QA robot can then be populated with test questions and answers, archived questions and answers from a social network, and information from other sources. The QA robot uses those sources of information to learn.”];
[0031 “In one embodiment, the QA robot stores the questions and their associated answers directly into its knowledgebase and retrieves that information when similar questions are subsequently asked”];
[0016 “In this way, a QA robot can use good answers as templates to respond to similar questions. By observing human interaction, the QA robot can enlarge its database of answers, refine which questions it chooses to answer, and refine the way it answers the questions.”];
such that a logical inference of the trained language model in response to a subsequently received query for information is increased as compared with the logical inference of the language model in response to a same or similar query for information before the training of the language model
[0031 “In one embodiment, the QA robot stores the questions and their associated answers directly into its knowledgebase and retrieves that information when similar questions are subsequently asked.”];
[0022 “Moreover, system 100 also collects feedback on its answers and uses that feedback to refine its ability to answer subsequent questions”];
[0064 “ Then when the same or very similar question is posed later to system 100, the QA expert simply regurgitates the best answer it has stored. “];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine teachings of Tunstall-Pedoe into the teaching of Heck because it would allow the system not only to determine whether an answer meets a required threshold but also to use the evaluation outcome to guide further training or refinement when the threshold is not satisfied. Such a combination would predictability improve the system by enabling a more robust answer validation system that uses threshold comparisons. This would improve the future responses.
Tunstall-Pedoe in view of Heck do not teach
generating reasoning statements in the form of additional natural language queries for information logically related to the first query for information;
However, Boguraev teaches
generating reasoning statements in the form of additional natural language queries for information logically related to the first query for information;
[Column 2, lines 19-22 “ A first question to be answered by a Question Answering (QA) system is provided to a processor. The processor creates a collection of candidate answers to the
first question. “];
[Column 2 lines 28-30 “The processor generates a plurality of hypotheses concerning
the missing information. The evidence analysis module produces a secondary question based on each hypothesis”];
[Column 1, lines 57-59“The missing information comprises any information that improves confidence scores
for candidate answers to a question.”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine teachings of Tunstall-Pedoe in view of Heck with the teachings of Boguraev to generate additional queries logically related to a first query and determine corresponding answers, thereby expanding the training information and improving the model’s consistency and accuracy for subsequent same or similar queries.
Claim [ 4, 6, 12, 14, 19 ] are rejected under 35 U.S.C. 103 as being unpatentable over
Tunstall-Pedoe (US11977854 B2) ) in view of Zhang (CN115270988A) and in view of Heck (US 20090162824 A1) and in further view of Hebenthal (US 20200320411 A1).
Regarding claim 4, Tunstall-Pedoe in view of Zhang and in view of Heck teach the method of claim 1, further comprising: responding to a prompt for information using the trained language model; themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the natural language question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM. An advantage is that an improved answer to the question may be provided by the LLM output”].
However, Tunstall-Pedoe in view of Zhang and in view of Heck do not teach verifying if the response to the prompt is within a threshold of a target response to the prompt.
But Hebenthal teaches verifying if the response to the prompt is within a threshold of a target response to the prompt - [Page 2, 0019 “In certain embodiments, inaccuracies in predicted results are improved upon by collecting additional truth samples and retraining the ML system without the need of creating new prediction algorithms. For example, truth samples include predicted results that were verified and/or corrected by external evaluation entities (e.g., by human experts). In some examples, the systems and methods are capable of spotting inaccuracies in the predicted results. For example, in response to spotting inaccuracies in the predicted results, inaccuracies are decreased by dynamically collecting additional truth samples from the external evaluation entities. As an example, in response to receiving additional truth samples, the ML system is automatically retrained after a predetermined truth interval”] [Threshold comparison: 0038 “Referring to FIG. 2, in certain embodiments, the result evaluation component 208 is configured, in response to the one or more accuracy parameters being smaller than a predetermined minimum threshold, increase the truth counter 226. For example, the result evaluation component 208 is configured to increase the truth counter 226 for each question associated with textual data of the processed documents 214. As an example, in response to identifying an accuracy degradation event, the result evaluation component 208 is configured to store the truth counter 226 in the accuracy store 206. In some examples, the result evaluation component 208 is configured to determine based at least in part on the quality control parameter whether an evaluation of the questions by the external evaluation entities 222 is required until the accuracy degradation event is cleared. For example, the result evaluation component 208 is configured to clear the accuracy degradation event if the one or more accuracy parameters are equal to or larger than the predetermined minimum threshold. In certain examples, the result processing component 204 is configured to increase the quality control parameter”]; [Page 6 0043 “Generation of new models (retraining): In response to a truth counter of at least one question being larger than a first predetermined truth threshold, one or more second models are generated at process 416, and a second accuracy score associated with the one or more second models is determined at process 418. At process 420, in response to the second accuracy score being larger than the first accuracy score associated with the one or more first models, the one or more first models with the one or more second models are replaced at the machine learning system”]; [0049 “ In response to the one or more accuracy parameters being smaller than a predetermined minimum threshold, an accuracy degradation event is identified, and the quality control parameter is increased. In response to a truth counter of at least one question being larger than a first predetermined truth threshold, one or more second models are generated, and a second accuracy score associated with the one or more second models is determined”]; [Page 7 0052” In response to the second accuracy score being smaller than or equal to the first accuracy score, the one or more second models is generated, and the second accuracy score associated with the one or more second models is determined. In other examples, the method further includes determining based at least in part on a quality control parameter whether an evaluation of the questions by one or more external entities is required until the accuracy degradation event is cleared”];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine teachings of Tunstall-Pedoe in view of Zhang and in view of Heck into the teaching of Hebenthal because it would improve the accuracy of machine generated answers. This modification merely applies a known threshold based reliability determination technique to the machine learning system in order to ensure the responses are generated when they meet a predetermined threshold, yielding predictable improvements in response accuracy and reliability.
Regarding claim 6, Tunstall-Pedoe in view of Zhang and in view of Heck does teach the method of claim 5, wherein the receiving at least one prompt; the generating an inference, the receiving information,
Heck teaches the receiving at least one prompt – [Page 2 0022 “FIG. 1 illustrates an exemplary QA robot system 100 for receiving questions, generating answers to those questions, and displaying the answers to the users. Moreover, system 100 also collects feedback on its answers and uses that feedback to refine its ability to answer subsequent questions. System 100, in one embodiment, includes question analyzer 110, QA answer component 120, confidence engine 130, decision maker 140, adjudicator 150, and feedback analyzer 160. In other embodiments, system 100 may include a different set of tools and components. Each of the components of system 100 is discussed below, but first a few commonly used terms are discussed”]; [Page 2 0023 “A "question" as used herein is a query submitted by a user to a QA robot. In one embodiment, the question can be in a natural language format, (e.g., the format a person would typically ask a question). Example questions include "what is your name?", "why is the sky blue?", "how many teeth does a shark have?", etc. In other embodiments, a question can be a keyword string, like a search query (e.g., "movies+`James Dean`"). Questions can include requests for a wide variety of data. Some of the types of data a question may request include: (1) informational data, (2) subjective information, (3) localized information, (4) timely information, and (5) search engine data. In other embodiments, questions may request other types of data”].
Heck teaches the generating an inference - [Page 3 0030 "Answers" as used herein refers to the information that is presented to a user in response to a question. Answers can consist of the types of information described above. Answers are derived by a QA robot in a variety of ways”]; [0031 “One way to teach the QA robot how to answer questions is to boot it into an initial training mode. According to one embodiment, the QA robot can then be populated with test questions and answers, archived questions and answers from a social network, and information from other sources. The QA robot uses those sources of information to learn. For example, a social network may already have archives of questions and answers that can be fed to the QA robot. In one embodiment, the QA robot stores the questions and their associated answers directly into its knowledgebase and retrieves that information when similar questions are subsequently asked. In one embodiment, this training may be supervised by people to ensure that the answers to a question are correct and that answers are being stored and indexed properly”]. [Page 3 0032 “Another approach QA robot may use to learn how to generate answers to questions is to observe users (particularly expert users) as they respond to questions on a social network. For example, suppose a user posts the question "where can I buy good Indian food in Portland, Oreg.?" Users familiar with the area may respond to the question listing some of their favorite Indian restaurants. The QA robot captures the question and the posted answers. The captured information is analyzed to determine how often a particular restaurant is listed among the answers. If a restaurant is listed several times by several different users, the QA robot captures that information and can deduce that that particular restaurant may be a good answer to the question”].
Heck teaches the receiving information - [Page 3 0033 “Moreover, if the user who posted the question later returns and indicates (e.g., by giving a thumb up or down to the answer, rating the answer on a scale of 1-10, etc.) that the suggestion was wonderful, then the QA robot gains even more confidence in the answer so that the next time a user asks for good Indian food in Portland the QA robot can retrieve and display the listed restaurant. As an additional benefit, the QA robot can give credibility points to the users that answered the question so that their answers (both previous and subsequent ones) are given more weight in later analyses of questions”].
See the rejection of claim 5 for rationale to modify Tunstall-Pedoe in view of Zhang in view of Heck.
However, Tunstall-Pedoe in view of Zhang in view of Heck does not teach the providing training data are repeated until an inference is generated by the language model that is within the threshold of the target inference.
But Hebenthal does teach the providing training data are repeated until an inference is generated by the language model that is within the threshold of the target inference - [Page 2, 0019 “In certain embodiments, inaccuracies in predicted results are improved upon by collecting additional truth samples and retraining the ML system without the need of creating new prediction algorithms. For example, truth samples include predicted results that were verified and/or corrected by external evaluation entities (e.g., by human experts). In some examples, the systems and methods are capable of spotting inaccuracies in the predicted results. For example, in response to spotting inaccuracies in the predicted results, inaccuracies are decreased by dynamically collecting additional truth samples from the external evaluation entities. As an example, in response to receiving additional truth samples, the ML system is automatically retrained after a predetermined truth interval”] [Threshold comparison: 0038 “Referring to FIG. 2, in certain embodiments, the result evaluation component 208 is configured, in response to the one or more accuracy parameters being smaller than a predetermined minimum threshold, increase the truth counter 226. For example, the result evaluation component 208 is configured to increase the truth counter 226 for each question associated with textual data of the processed documents 214. As an example, in response to identifying an accuracy degradation event, the result evaluation component 208 is configured to store the truth counter 226 in the accuracy store 206. In some examples, the result evaluation component 208 is configured to determine based at least in part on the quality control parameter whether an evaluation of the questions by the external evaluation entities 222 is required until the accuracy degradation event is cleared. For example, the result evaluation component 208 is configured to clear the accuracy degradation event if the one or more accuracy parameters are equal to or larger than the predetermined minimum threshold. In certain examples, the result processing component 204 is configured to increase the quality control parameter”]; [Page 6 0043 “Generation of new models (retraining): In response to a truth counter of at least one question being larger than a first predetermined truth threshold, one or more second models are generated at process 416, and a second accuracy score associated with the one or more second models is determined at process 418. At process 420, in response to the second accuracy score being larger than the first accuracy score associated with the one or more first models, the one or more first models with the one or more second models are replaced at the machine learning system”]; [0049 “ In response to the one or more accuracy parameters being smaller than a predetermined minimum threshold, an accuracy degradation event is identified, and the quality control parameter is increased. In response to a truth counter of at least one question being larger than a first predetermined truth threshold, one or more second models are generated, and a second accuracy score associated with the one or more second models is determined”]; [Page 7 0052” In response to the second accuracy score being smaller than or equal to the first accuracy score, the one or more second models is generated, and the second accuracy score associated with the one or more second models is determined. In other examples, the method further includes determining based at least in part on a quality control parameter whether an evaluation of the questions by one or more external entities is required until the accuracy degradation event is cleared”];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine teachings of Tunstall-Pedoe in view of Zhang in view of Heck into the teaching of Hebenthal because it would improve the accuracy of machine generated answers. Hebenthal teaches retraining or generating new machine learning models when performance metrics fall below a predetermined accuracy threshold, thereby improving the performance of machine learning model and ensuring the subsequent generated answers meet the required accuracy threshold.
Regarding claim 12, Tunstall-Pedoe in view of Zhang in view of Heck does teach the apparatus of claim 9, wherein the apparatus is further configured to: respond to a prompt for information using the trained language model; and verify if the response to the prompt is within a threshold of a target response to the prompt. Claim 12 is rejected for the same reasons as claim 4.
Regrading claim 14, Tunstall-Pedoe in view of Zhang in view of Heck and in further view of Hebenthal teach the apparatus of claim 13, wherein the receiving at least one prompt, the generating an inference, the receiving information, and the providing training data are repeated until an inference is generated by the language model that is within the threshold of the target inference. Claim 14 is rejected for the same reasons as claim 6.
Regarding claim 19, Tunstall-Pedoe in view of Zhang in view of Heck and in further view of Hebenthal does teach the non-transitory computer readable medium of claim 18, wherein the receiving at least one prompt, the generating an inference, the receiving information, and the providing training data are repeated until an inference is generated by the language model that is within the threshold of the target inference. Claim 19 is rejected for the same reasons as claim 6.
Claim [21 ] are rejected under 35 U.S.C. 103 as being unpatentable over Tunstall-Pedoe (US11977854 B2) in view of Heck (US 20090162824 A1) and in further view of Boguraev (US 10614725 B2) and in further view of Shindo et al (Shindo, Hikaru, et al. "Learning differentiable logic programs for abstract visual reasoning." Machine Learning 113.11 (2024): 8533-8584. Note: this paper was first published as an arXiv preprint on July 3, 2023)
Regarding claim 21, Tunstall-Pedoe in view of Heck and in further view of Boguraev do not teach the method of claim 8, wherein the language model generates a logic program by assigning a probability for the first query for information and the additional queries for information in a differentiable fashion.
However, Shindo teaches
the language model generates a logic program by assigning a probability for the first query for information and the additional queries for information in a differentiable fashion.
[Section 3.5 “Fig. 4 shows an overview of learning of NEUMANN. It learns logic programs in two steps: (1) NEUMANN generates promising clauses by iterating scoring and sampling of clauses.”];
[3.5.2 Weight Optimization “NEUMANN composes differentiable logic programs that consist of multiple clauses as follows:..”]
[3.5.2 Weight Optimization “Algorithm 3 shows the entire learning process of NEUMANN. (Line 1-3) An initial reasoning graph is built. (Line 5-10) Clauses C are scored by computing gradients. Useful clauses in C get negatively large gradients, and thus they are scored high at line 10. (Line 13-21) Sample clauses to be refined to generate new clauses according to the scores using the Gumbel-max trick.”];
[3.5.1 Clause Generation “NEUMANN generates candidate clauses by iteratively (1) scoring clauses using gradients and (2) performing differentiable sampling on the scores and refining them.”];
[Page 46 Task 2 (T2) “ NEUMANN assigns probabilities over query atoms according to the input, i.e. it gives 1.0 for a query atom given input and 0.0 for other query atoms.”];
[Page 46 Task 2 (T2). “Queries about different operations are given randomly, so the model needs to handle different types of queries in the prediction.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine teachings of Tunstall-Pedoe in view of Zhang in view of Heck with Shindo because Shindo differentiable logic program technique to assign probabilities to the first and additional queries (supplied by Tunstall-Pedoe in view of Zhang in view of Heck) and optimize the resulting logic program would thereby improve the accuracy and consistency of the model’s logical inferences.
Claim [22 ] are rejected under 35 U.S.C. 103 as being unpatentable over Tunstall-Pedoe (US11977854 B2) in view of Heck (US 20090162824 A1) and in further view of Boguraev (US 10614725 B2) and in further view of Shindo et al (Shindo, Hikaru, et al. "Learning differentiable logic programs for abstract visual reasoning." Machine Learning 113.11 (2024): 8533-8584.) and in further view of Huang(Huang, Jiani, et al. "Scallop: From probabilistic deductive databases to scalable differentiable reasoning." Advances in Neural Information Processing Systems 34 (2021): 25134-25145.)
Regarding claim 22, Tunstall-Pedoe in view of Heck and in further view of Boguraev and in further view of Shindo do teach the method of claim 21, wherein the probability of the logic program is maximized by adjusting the probability that the language model assigns to each logically related query for information in a differentiable fashion
Shindo teaches wherein the probability of the logic program is maximized by adjusting the probability that the language model assigns to each logically related query for information in a differentiable fashion
[3.5.2 Weight Optimization “NEUMANN composes differentiable logic programs that consist of multiple clauses as follows:..”];
[3.5.1 Clause Generation “NEUMANN generates candidate clauses by iteratively (1) scoring clauses using gradients and (2) performing differentiable sampling on the scores and refining them.”];
[3.5 Learning Logic Programs by NEUMANN “(2) After the iteration of clause generation steps, NEUMANN assigns randomly-initialized clause weights and optimizes them to minimize the classification loss.”];
[Page 46 Task 2 (T2) “ NEUMANN assigns probabilities over query atoms according to the input, i.e. it gives 1.0 for a query atom given input and 0.0 for other query atoms.”];
[Page 46 Task 2 (T2). “Queries about different operations are given randomly, so the model needs to handle different types of queries in the prediction.” where minimizing loss through gradient decent adjusts the program weights so that probability of the correct query result increases and therefore the selected logic program is increased or maximized ].
However, Tunstall-Pedoe in view of Heck and in further view of Boguraev and in further view of Shindo do not teach using a probabilistic data space
But Scallop teaches using a probabilistic data space
[Page 4, 3. Background “F and J form a probabilistic database. By combining the F, J with Rand Q,weobtain a probabilistic Datalog program P.” where the probabilistic data space corresponds to the Scallop’s probabilistic database. ];
[Page 5, Probability Calculation. “Given the set of proofs Sq for a query result q, the success probability Pr(q) is simply the likelihood of Sq, denoted Pr(Sq), which can be computed using Weighted Model Counting “];
[Page 5, 4 Framework “2. Learning (Section 4.2): Given a neural symbolic reasoning dataset D and a loss function L, learn a perception model Mθ which, for each (x,y) ∈ D, transforms x into a probabilistic database captured by Datalog program Px θ. We aim to minimize the following objective: J(θ) = 1 |D| (x,y)∈D LExec(Px θ),y .”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine teachings of Tunstall-Pedoe in view of Heck and in further view of Boguraev and in further view of Shindo with Scallop to generate and optimize a differentiable logic program using probabilities assigned to query results within a probabilistic database, would thereby improve the accuracy and consistency of the language model’s logical inferences.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHEZA ABDUL AZIZ whose telephone number is (571)272-9610. The examiner can normally be reached Monday-Friday 7:30am-5pm Alternate Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL C WASHBURN/Supervisory Patent Examiner, Art Unit 2657
Prosecution Insights