DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 03/06/2024 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is/are being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-23 are rejected under 35 U.S.C.101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-15 are directed to a process. Claims 16-23 are directed to a machine or an article of manufacture.
With respect to claim(s) 1, 16, and 22:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
(Claims 1 and 22) constructing/construct a first token sequence based on the set of in-context prompt/completion pairs; (Mental process – A person can mentally construct (think of) a token sequence based on prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
(Claim 16) construct a first token sequence based on the set of in-context prompt/completion pairs, wherein each token sequence in the set of token sequences can be fitted into the context window; (Mental process – A person can mentally construct (think of) a token sequence based on prompt/completion pairs and mentally fit a token sequence in a context window – see MPEP § 2106.04(a)(2)(III))
(Claims 1 and 22) fitting the first token sequence into the context window; (Mental process – A person can mentally fit a token sequence in a context window – see MPEP § 2106.04(a)(2)(III))
generate a next token in accordance with the target task. (Mental process – A person can mentally generate a next token (e.g., word) according to a task – see MPEP § 2106.04(a)(2)(III))
If claim limitations, under their broadest reasonable interpretation, cover performance of the limitations as a mental process, but for the recitation of generic computer components, then the claim limitations fall within the mathematical or mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
(Claim 1) A method of performing in-context training for a machine learning (ML) model (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 16) An apparatus for performing in-context training for a machine learning (ML) model, the apparatus comprising: one or more processors; a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the apparatus to; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 22) A system for performing in-context training for a machine learning (ML) model, the system comprising: one or more processors; a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the system to: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
receiving/receive an ML model comprising a context window of a predetermined size; (Mere data gathering – Adding insignificant extra-solution activity of mere data gathering to the judicial exception – see § MPEP2106.05(g).)
receiving/receive a set of in-context prompt/completion pairs prepared for a target task; (Mere data gathering – Adding insignificant extra-solution activity of mere data gathering to the judicial exception – see § MPEP2106.05(g).)
performing/perform a first in-context training pass using the first token sequence to train the ML model to (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
(Claim 1) A method of performing in-context training for a machine learning (ML) model (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 16) An apparatus for performing in-context training for a machine learning (ML) model, the apparatus comprising: one or more processors; a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the apparatus to; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 22) A system for performing in-context training for a machine learning (ML) model, the system comprising: one or more processors; a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the system to: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
receiving/receive an ML model comprising a context window of a predetermined size; (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC)- see MPEP § 2106.05(d)(ll)(i) - Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information).)
receiving/receive a set of in-context prompt/completion pairs prepared for a target task; (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC)- see MPEP § 2106.05(d)(ll)(i) - Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information).)
performing/perform a first in-context training pass using the first token sequence to train the ML model to (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim(s) 2 and 17:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
(Claim 2) wherein constructing the first token sequence based on the set of in-context prompt/completion pairs includes: (Mental process – A person can mentally construct (think of) a token sequence based on prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
(Claim 17) construct the first token sequence (Mental process – A person can mentally construct (think of) a token sequence – see MPEP § 2106.04(a)(2)(III))
selecting a first subset of the set of in-context prompt/completion pairs, wherein the combined size of the selected first subset of the in-context prompt/completion pairs is less than or equal to the predetermined size of the context window; (Mental process – A person can mentally select a subset of prompt completion pairs that combined are smaller than a predetermined size – see MPEP § 2106.04(a)(2)(III))
concatenating the selected first subset of the in-context prompt/completion pairs to form the first token sequence (Mental process – A person can mentally concatenate prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
(Claim 17) wherein the set of token sequences includes a first token sequence, and wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(claim 17) wherein the first token sequence is used to perform a first in-context training pass in the sequence of in-context training passes. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
(Claim 17) wherein the set of token sequences includes a first token sequence, and wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(claim 17) wherein the first token sequence is used to perform a first in-context training pass in the sequence of in-context training passes. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 3 and 18:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
wherein the first subset of the in-context prompt/completion pairs is selected such that the combined size of the selected first subset of the in-context prompt/completion pairs is as close to the predetermined size as possible without exceeding the predetermined size to maximize the usage of the context window and to include as many different prompt/completion pairs as possible. (Mental process – A person can mentally select prompt completions pairs such that the pairs do not exceed a predetermined size when combined – see MPEP § 2106.04(a)(2)(III))
Additionally, the claim(s) do not recite any new additional elements that would amount to an integration of the abstract idea into a practical application (individually or in combination) or significantly more than the judicial exception.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 4:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs by the first in-context training; (Mental process – A person can mentally determine if there are unselected prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
in response to determining that there are unselected prompt/completion pairs, constructing a second token sequence based on the set of in-context prompt/completion pairs; (Mental process – A person can mentally construct a token sequence after determining there are unselected pairs – see MPEP § 2106.04(a)(2)(III))
fitting the second token sequence into the context window; (Mental process – A person can mentally fit a token sequence in a context window – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein after completing the first in-context training pass using the first token sequence (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
performing a second in-context training pass using the second token sequence to further train the ML model to perform the target task. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein after completing the first in-context training pass using the first token sequence (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
performing a second in-context training pass using the second token sequence to further train the ML model to perform the target task. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 5:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
wherein constructing the second token sequence based on the set of in-context prompt/completion pairs includes: (Mental process – A person can mentally construct (think of) a token sequence based on prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
selecting a second subset of the set of in-context prompt/completion pairs, wherein: (Mental process – A person can mentally select a subset of prompt completion pairs – see MPEP § 2106.04(a)(2)(III))
concatenating the selected second subset of the in-context prompt/completion pairs to form the second token sequence. (Mental process – A person can mentally concatenate prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
the second subset of in-context prompt/completion pairs does not include any in-context prompt/completion pair in the first subset of in-context prompt/completion pairs; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
the combined size of the selected second subset of the in-context prompt/completion pairs is as close to the predetermined size as possible to maximize the usage of the context window and to include as many different and unused prompt/completion pairs as possible; and (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
the second subset of in-context prompt/completion pairs does not include any in-context prompt/completion pair in the first subset of in-context prompt/completion pairs; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
the combined size of the selected second subset of the in-context prompt/completion pairs is as close to the predetermined size as possible to maximize the usage of the context window and to include as many different and unused prompt/completion pairs as possible; and (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 6:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs from the first in-context training pass and the second in-context training pass; and (Mental process – A person can mentally determine if there are unselected prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
in response to determining that there are unselected prompt/completion pairs, constructing a third token sequence from the unselected prompt/completion pairs; (Mental process – A person can mentally construct a token sequence after determining there are unselected pairs – see MPEP § 2106.04(a)(2)(III))
fitting the third token sequence into the context window; (Mental process – A person can mentally fit a token sequence in a context window – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein after completing the second in-context training pass using the second token sequence, the method further comprises: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
performing a third in-context training pass using the third token sequence to further train the ML model to perform the target task. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein after completing the second in-context training pass using the second token sequence, the method further comprises: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
performing a third in-context training pass using the third token sequence to further train the ML model to perform the target task. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 7:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs from all of the previous in-context training passes; (Mental process – A person can mentally determine if there are unselected prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
if so, constructing one or more additional token sequences from the unselected prompt/completion pairs until the set of in-context prompt/completion pairs are fully exhausted; (Mental process – A person can mentally construct a token sequence after determining there are unselected pairs – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
performing one or more additional in-context training passes using the one or more additional token sequences to further train the ML model; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
otherwise, terminating the in-context training for the ML model, wherein there is no duplicated prompt/completion pair used by any two in-context training passes in the set of in-context training passes, thereby improving a model training efficiency based on the set of in-context prompt/completion pairs. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
performing one or more additional in-context training passes using the one or more additional token sequences to further train the ML model; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
otherwise, terminating the in-context training for the ML model, wherein there is no duplicated prompt/completion pair used by any two in-context training passes in the set of in-context training passes, thereby improving a model training efficiency based on the set of in-context prompt/completion pairs. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 8:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
wherein a training time associated with training the ML model is proportional to a first number of in-context prompt/completion pairs in the set of in-context prompt/completion pairs divided by an average number of selected prompt/completion pairs of a set of constructed token sequences associated with the set of in-context training passes. (Mathematical concepts – This limitation describes a mathematical relationship – see MPEP § 2106.04(a)(2)(I))
Additionally, the claim(s) do not recite any new additional elements that would amount to an integration of the abstract idea into a practical application (individually or in combination) or significantly more than the judicial exception.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 9:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
backpropagating from the second prompt/completion pair that immediately follows the first prompt/completion pair while using the first prompt/completion pair as an associated context, thereby effectively performing a one-shot training on the second prompt/completion pair. (Mathematical concepts – Backpropagation is a specific method that involves mathematical calculations – see MPEP § 2106.04(a)(2)(I))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein performing the first in-context training pass using the first token sequence includes: initially using the first prompt/completion pair in the first token sequence to perform a zero-shot training without involving other prompt/completion pairs in the first token sequence; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein performing the first in-context training pass using the first token sequence includes: initially using the first prompt/completion pair in the first token sequence to perform a zero-shot training without involving other prompt/completion pairs in the first token sequence; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 10:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
determining if there is at least a third prompt/completion pair following the second prompt/completion pair in the first token sequence; (Mental process – A person can mentally determine if there are prompt/completion pairs in a token sequence – see MPEP § 2106.04(a)(2)(III))
if so, backpropagating from the third prompt/completion pair immediately following the second prompt/completion pair while using the first and second prompt/completion pairs as the associated context, thereby effectively performing a two-shot training on the third prompt/completion pair; (Mathematical concepts – Backpropagation is a specific method that involves mathematical calculations – see MPEP § 2106.04(a)(2)(I))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein after performing the one-shot training, the method further comprises: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
otherwise, terminating the first in-context training pass based on the first token sequence. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein after performing the one-shot training, the method further comprises: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
otherwise, terminating the first in-context training pass based on the first token sequence. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 11:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
determining if there is at least a fourth prompt/completion pair following the third prompt/completion pair in the first token sequence; (Mental process – A person can mentally determine if there are prompt/completion pairs in a token sequence – see MPEP § 2106.04(a)(2)(III))
if so, backpropagating from the fourth prompt/completion pair immediately following the third prompt/completion pair while using the first, second, and third prompt/completion pairs as the associated context, thereby effectively performing a three-shot training on the fourth prompt/completion pair; (Mathematical concepts – Backpropagation is a specific method that involves mathematical calculations – see MPEP § 2106.04(a)(2)(I))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein after performing the two-shot training, the method further comprises: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
otherwise, terminating the first in-context training pass based on the first token sequence. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein after performing the two-shot training, the method further comprises: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
otherwise, terminating the first in-context training pass based on the first token sequence. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 12 and 20:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
performing a zero-shot training by backpropagating on the first completion token in the first prompt/completion pair without involving other prompt/completion pairs in the first token sequence; (Mathematical concepts – Backpropagation is a specific method that involves mathematical calculations – see MPEP § 2106.04(a)(2)(I))
sequentially performing N-1 backward passes, wherein each backward pass in the sequence of N-1 backward passes is a (M-1)-shot training on the Mth completion token in the Mth prompt/completion pair in the first token sequence while using the preceding M-1 prompt/completion pairs as context, wherein M = 2, ..., N. (Mathematical concepts – Backward passes are part of the backpropagation process, which involves mathematical calculations – see MPEP § 2106.04(a)(2)(I))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein the first token sequence is composed of a sequence of N concatenated prompt/completion pairs, (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 12) and wherein performing the first in- context training pass using the first token sequence includes: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 20) and wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to perform the first in-context training pass by: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein the first token sequence is composed of a sequence of N concatenated prompt/completion pairs, (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 12) and wherein performing the first in- context training pass using the first token sequence includes: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 20) and wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to perform the first in-context training pass by: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 13:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein the ML model includes a transformer model; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
and wherein performing the first in-context training pass using the first token sequence includes training the transformer model using the first token sequence. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein the ML model includes a transformer model; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
and wherein performing the first in-context training pass using the first token sequence includes training the transformer model using the first token sequence. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 14:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein the target task is to respond to queries of a target topic, and wherein the set of in-context prompt/completion pairs is a set of query/answer examples of the same target topic. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein the target task is to respond to queries of a target topic, and wherein the set of in-context prompt/completion pairs is a set of query/answer examples of the same target topic. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 15:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein each prompt/completion pair in the set of in- context prompt/completion pairs has the same format as the other prompt/completion pairs in the set of in-context prompt/completion pairs. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein each prompt/completion pair in the set of in- context prompt/completion pairs has the same format as the other prompt/completion pairs in the set of in-context prompt/completion pairs. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 19:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
determine if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs by the first in-context training pass; (Mental process – A person can mentally determine if there are unselected prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
in response to determining that there are unselected prompt/completion pairs, select a second subset of the set of in-context prompt/completion pairs, (Mental process – A person can mentally select prompt/completion pairs after determining there are unselected pairs – see MPEP § 2106.04(a)(2)(III))
concatenate the selected second subset of the in-context prompt/completion pairs to form the second token sequence (Mental process – A person can mentally concatenate prompt/completion pairs – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein after constructing the first token sequence, the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
wherein the combined size of the selected second subset of the in-context prompt/completion pairs is equal to or substantially equal to the predetermined size of the context window; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
wherein the second token sequence is used to perform a second in-context training pass in the sequence of in-context training passes. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein after constructing the first token sequence, the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
wherein the combined size of the selected second subset of the in-context prompt/completion pairs is equal to or substantially equal to the predetermined size of the context window; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
wherein the second token sequence is used to perform a second in-context training pass in the sequence of in-context training passes. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 21:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
wherein a training time associated with training the ML model is proportional to a first number of in-context prompt/completion pairs in the set of in-context prompt/completion pairs divided by an average number of selected prompt/completion pairs associated with the set of token sequences. (Mathematical concepts – This limitation describes a mathematical relationship – see MPEP § 2106.04(a)(2)(I))
Additionally, the claim(s) do not recite any new additional elements that would amount to an integration of the abstract idea into a practical application (individually or in combination) or significantly more than the judicial exception.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 23:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein the ML model includes a transformer-based language model; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
and wherein performing the first in-context training pass using the first token sequence includes training the transformer-based language model. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein the ML model includes a transformer-based language model; (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
and wherein performing the first in-context training pass using the first token sequence includes training the transformer-based language model. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 4, 6, 9-17, 20, and 22-23 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by XU ("On the Tool Manipulation Capability of Open-source Large Language Models"), hereafter XU.
Regarding Claim 1:
XU teaches:
A method of performing in-context training for a machine learning (ML) model, comprising: (XU [page 1, Abstract] teaches: "By analyzing common tool manipulation failures, we first demonstrate that open-source LLMs may require training with usage examples, in-context demonstration and generation style regulation to resolve failures.”)
receiving an ML model comprising a context window of a predetermined size; (XU [page 18, section B. 1 Models] teaches: "The models have a sequence length of 4096 (i.e., comprising a context window of a predetermined size) to push beyond the context window limitations of the existing open-source language models.")
receiving a set of in-context prompt/completion pairs prepared for a target task; (XU [page 2, Figure 1] teaches: “In a single-step scenario, an action generator directly generates API calls to accomplish the goal.” XU [page 22, section C.3 Training details] teaches: "In each sample, all the goal-action pairs (i.e., in-context prompt/completion pairs) are from the same task (i.e., prepared for a target task)." XU [page 22, Figure 6] teaches: “We use all-shot loss for model alignment. We concatenate several examples into a single training sample and backpropagate through the loss on the blue actions in every example.” Examiner’s note: The pairs were obtained (i.e., receiving) in order to train the models. Additionally, a set of in-context prompt/completion pairs can be interpreted as the total set of training samples used for model alignment, which are prepared using the task data shown in XU [page 21, Table 10].)
constructing a first token sequence based on the set of in-context prompt/completion pairs; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples (i.e., first token sequence), we concatenate (i.e., constructing) API documents and multiple pairs of goal and API calls (i.e., based on the set of in-context prompt/completion pairs) as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "In each sample, all the goal-action pairs are from the same task.")
fitting the first token sequence into the context window; (XU [page 22, section C.3 Training details] teaches: "We use a max sequence length of 2048 (i.e., the first token sequence) without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task." Examiner's note: XU [page 18, section B. 1 Models] teaches a context window of length of 4096 tokens and XU [page 22, section C.3 Training details] teaches a concatenated sequence of length of 2048 tokens. Therefore, the concatenated sequence fits in the context window when it is inputted for processing (i.e., fitting).)
performing a first in-context training pass using the first token sequence to train the ML model to generate a next token in accordance with the target task. (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 15, section A.1 OpenWeather] teaches: "The model’s generation (i.e., generate the next token) will be considered successful if the output matches the expected result (i.e., in accordance with the target task) precisely." Examiner's note: Under BRI, performing a first in-context training pass can be interpreted as processing one concatenated training sample of token length 2048 (i.e., using the first token sequence to train the ML model) within the initial batch of training samples.)
Regarding Claim 2:
XU teaches the elements of claim 1 as outlined above. XU further teaches:
wherein constructing the first token sequence based on the set of in-context prompt/completion pairs includes: selecting a first subset of the set of in-context prompt/completion pairs, […] (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6." XU [page 22, Figure 6] teaches: "We concatenate several examples (i.e., a first subset) into a single training sample and backpropagate through the loss on the blue actions in every example." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. […] In each sample, all the goal-action pairs are from the same task." Examiner's note: Under BRI, selecting a first subset of the set of in-context prompt/completion pairs can be interpreted as choosing the several examples from the dataset to be concatenated as one input sequence for training the LLM.)
[…] wherein the combined size of the selected first subset of the in-context prompt/completion pairs is less than or equal to the predetermined size of the context window; (XU [page 22, section C.3 Training details] teaches: "We use a max sequence length of 2048 (i.e., the first token sequence) without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task." Examiner's note: XU [page 18, section B. 1 Models] teaches a context window of length of 4096 tokens and XU [page 22, section C.3 Training details] teaches a concatenated sequence of length of 2048 tokens. Therefore, the size of the concatenated sequence (i.e., combined size of the selected first subset of the in-context prompt/completion pairs) is less than the predetermined size of the context window.)
concatenating the selected first subset of the in-context prompt/completion pairs to form the first token sequence. (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls (i.e., concatenating the selected first subset of the in-context prompt/completion pairs) as one input sequence (i.e., to form the first token sequence) to the LLMs. We use an all-shot loss formulation illustrated in Figure 6." XU [page 22, Figure 6] teaches: "We concatenate several examples into a single training sample and backpropagate through the loss on the blue actions in every example." XU [page 22, section C.3 Training details] teaches: "In each sample, all the goal-action pairs are from the same task.")
Regarding Claim 4:
XU teaches the elements of claim 2 as outlined above. XU further teaches:
wherein after completing the first in-context training pass using the first token sequence, the method further comprises: determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs by the first in-context training; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs by the first in-context training can be interpreted as continuing to process the next concatenated training sample from the dataset after completing the first concatenated training sample until the dataset has been processed. Additionally, after completing the first in-context training pass using the first token sequence can be interpreted as processing the first concatenated training sample.)
in response to determining that there are unselected prompt/completion pairs, constructing a second token sequence based on the set of in-context prompt/completion pairs; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples (i.e., constructing a second token sequence), we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” Examiner's note: XU [page 22, section C.3 Training details] teaches finetuning the models using a batch of 16 training samples from the dataset for a total of 8 epochs. Under BRI, in response to determining that there are unselected prompt/completion pairs can be interpreted as having remaining goal-action pairs from the dataset for processing during an epoch. Additionally, constructing a second token sequence based on the set of in-context prompt/completion pairs can be interpreted as processing a second concatenated training sample of API documents and multiple pairs of goal and API calls as one input sequence to the LLM.)
fitting the second token sequence into the context window; (XU [page 22, section C.3 Training details] teaches: "We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task." Examiner's note: XU [page 18, section B. 1 Models] teaches a context window of length of 4096 tokens and XU [page 22, section C.3 Training details] teaches a concatenated sequence of length of 2048 tokens. Therefore, the concatenated sequence fits in the context window when it is inputted for processing (i.e., fitting). Additionally, the second token sequence can be interpreted as a second concatenated training sample of API documents and multiple pairs of goal and API calls as one input sequence to the LLM.)
performing a second in-context training pass using the second token sequence to further train the ML model to perform the target task. (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 15, section A.1 OpenWeather] teaches: "The model’s generation will be considered successful if the output matches the expected result (i.e., to perform the target task) precisely." Examiner's note: Under BRI, performing a second in-context training pass can be interpreted as processing a second concatenated training sample of token length 2048 (i.e., using the second token sequence to further train the ML model) within the initial batch of training samples.)
Regarding Claim 6:
XU teaches the elements of claim 4 as outlined above. XU further teaches:
wherein after completing the second in-context training pass using the second token sequence, the method further comprises: determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs from the first in-context training pass and the second in-context training pass; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs from the first in-context training pass and the second in-context training pass can be interpreted as continuing to process the next concatenated training sample from the dataset after completing the first and second training samples.)
in response to determining that there are unselected prompt/completion pairs, constructing a third token sequence from the unselected prompt/completion pairs; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples (i.e., constructing a third token sequence), we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” Examiner's note: XU [page 22, section C.3 Training details] teaches finetuning the models using a batch of 16 training samples from the dataset for a total of 8 epochs. Under BRI, in response to determining that there are unselected prompt/completion pairs can be interpreted as having remaining goal-action pairs from the dataset for processing during an epoch. Additionally, constructing a third token sequence from the unselected prompt/completion pairs can be interpreted as constructing a third concatenated training sample from the dataset for processing, wherein the third concatenated training sample concatenates API documents and multiple pairs of goal and API calls as one input sequence to the LLM.)
fitting the third token sequence into the context window; (XU [page 22, section C.3 Training details] teaches: "We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task." Examiner's note: XU [page 18, section B. 1 Models] teaches a context window of length of 4096 tokens and XU [page 22, section C.3 Training details] teaches a concatenated sequence of length of 2048 tokens. Therefore, the concatenated sequence fits in the context window when it is inputted for processing (i.e., fitting). Additionally, the third token sequence can be interpreted as a third concatenated training sample of API documents and multiple pairs of goal and API calls as one input sequence to the LLM.)
performing a third in-context training pass using the third token sequence to further train the ML model to perform the target task. (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 15, section A.1 OpenWeather] teaches: "The model’s generation will be considered successful if the output matches the expected result (i.e., to perform the target task) precisely." Examiner's note: Under BRI, performing a third in-context training pass can be interpreted as processing the third concatenated training sample (i.e., using the third token sequence to further train the ML model) within the initial batch of training samples.)
Regarding Claim 9:
XU teaches the elements of claim 1 as outlined above. XU further teaches:
wherein performing the first in-context training pass using the first token sequence includes: (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 15, section A.1 OpenWeather] teaches: "The model’s generation will be considered successful if the output matches the expected result precisely." Examiner's note: Under BRI, performing a first in-context training pass can be interpreted as processing one concatenated training sample of token length 2048 (i.e., using the first token sequence) within the initial batch of training samples.)
initially using the first prompt/completion pair in the first token sequence to perform a zero-shot training without involving other prompt/completion pairs in the first token sequence; (XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, Figure 6] teaches a training example for all-shot model alignment using N goal-API pairs. Additionally, XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, initially using the first prompt/completion pair in the first token sequence can be interpreted as processing the first goal-action pair (Goal 1 and API calls 1, as shown in XU’s Figure 6) in the first concatenated training sample. Additionally, to perform a zero-shot training backpropagation can be interpreted as the LLM model generating the next word autoregressively via API call 1, where the next prediction is conditioned on
x
0
being goal 1
p
x
1
|
x
0
, and where the prediction uses no previous goal-action pairs (i.e., without involving their prompt/completion pairs in the first token sequence). The API calls are “blue actions”, where backpropagation occurs, and thus the loss for the generated next prediction
p
x
1
|
x
0
is backpropagated for model alignment.)
backpropagating from the second prompt/completion pair that immediately follows the first prompt/completion pair while using the first prompt/completion pair as an associated context, thereby effectively performing a one-shot training on the second prompt/completion pair. (XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." Examiner’s note: Under BRI, backpropagating from the second prompt/completion pair can be interpreted as predicting
p
x
3
x
0
,
x
1
,
x
2
, where
x
0
is Goal 1,
x
1
is the LLM generated next prediction,
x
2
is Goal 2, and
x
3
is the next prediction being predicted by the LLM immediately following the first goal-action pair (i.e., pair that immediately follows the first prompt/completion pair) while using
x
0
and
x
1
as context (i.e., while using the first prompt/completion pair as an associated context), and thus performing one-shot training for second goal-action pair
x
2
and
x
3
. After predicting
x
3
by the LLM, the loss is backpropagated at the second “blue action” (i.e., backpropagating from the second prompt/completion pair).)
Regarding Claim 10:
XU teaches the elements of claim 9 as outlined above. XU further teaches:
wherein after performing the one-shot training, the method further comprises: determining if there is at least a third prompt/completion pair following the second prompt/completion pair in the first token sequence; and if so, backpropagating from the third prompt/completion pair immediately following the second prompt/completion pair while using the first and second prompt/completion pairs as the associated context, thereby effectively performing a two-shot training on the third prompt/completion pair; otherwise, terminating the first in-context training pass based on the first token sequence. (XU [page 3, Large language model] teaches generating the next word
x
N
+
1
. When
N
=
4
, the next word prediction would be:
p
x
5
x
0
,
x
1
⏟
p
a
i
r
1
,
x
2
,
x
3
⏟
p
a
i
r
2
,
x
4
where after predicting
x
5
for the third goal-action pair (i.e., two-shot training), the loss would be backpropagated at the third blue action. In the case where
N
=
2
,
x
2
and
x
3
would be the last pair and there would be no more predictions to be made (i.e., terminating the first in-context training pass based on the first token sequence.).)
Regarding Claim 11:
XU teaches the elements of claim 10 as outlined above. XU further teaches:
wherein after performing the two-shot training, the method further comprises: determining if there is at least a fourth prompt/completion pair following the third prompt/completion pair in the first token sequence; and if so, backpropagating from the fourth prompt/completion pair immediately following the third prompt/completion pair while using the first, second, and third prompt/completion pairs as the associated context, thereby effectively performing a three-shot training on the fourth prompt/completion pair; otherwise, terminating the first in-context training pass based on the first token sequence. (XU [page 3, Large language model] teaches generating the next prediction
x
N
+
1
. When
N
=
6
, the next prediction prediction would be:
p
x
7
x
0
,
x
1
⏟
p
a
i
r
1
,
x
2
,
x
3
⏟
p
a
i
r
2
,
x
4
,
x
5
⏟
p
a
i
r
3
,
x
6
where after predicting
x
7
for the fourth goal-action pair (i.e., three-shot training), the loss would be backpropagated at the fourth blue action. In the case where
N
=
4
,
x
4
and
x
5
would be the last pair and there would be no more predictions to be made (i.e., terminating the first in-context training pass based on the first token sequence.).)
Regarding Claim 12:
XU teaches the elements of claim 1 as outlined above. XU further teaches:
wherein the first token sequence is composed of a sequence of N concatenated prompt/completion pairs, and wherein performing the first in-context training pass using the first token sequence includes: (XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." Examiner’s note: XU’s Figure 6 shows N concatenated goal-API call pairs for all-shot training. Under BRI, performing a first in-context training pass can be interpreted as processing one concatenated training sample of token length 2048 (i.e., using the first token sequence to train the ML model) within the initial batch of training samples.)
performing a zero-shot training backpropagation on the first completion token in the first prompt/completion pair without involving other prompt/completion pairs in the first token sequence; (XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, Figure 6] teaches a training example for all-shot model alignment using N goal-API pairs. Additionally, XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, initially using the first prompt/completion pair in the first token sequence can be interpreted as processing the first goal-action pair (Goal 1 and API calls 1, as shown in XU’s Figure 6) in the first concatenated training sample. Additionally, to perform a zero-shot training backpropagation can be interpreted as the LLM model generating the next prediction autoregressively via API call 1, where the next prediction is conditioned on
x
0
being goal 1
p
x
1
|
x
0
, and where the prediction uses no previous goal-action pairs (i.e., without involving their prompt/completion pairs in the first token sequence). The API calls are “blue actions”, where backpropagation occurs, and thus the loss for the generated next prediction
p
x
1
|
x
0
is backpropagated for model alignment.)
sequentially performing N−1 backward passes, wherein each backward pass in the sequence of N−1 backward passes is a (M−1)-shot training on the Mth completion token in the Mth prompt/completion pair in the first token sequence while using the preceding M−1 prompt/completion pairs as context, wherein M=2, . . . , N. (XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." Examiner’s note: Under BRI, sequentially performing N-1 backward passes can be interpreted as backpropagating the loss at the blue API-call actions for Goal 2 through Goal N of XU’s Figure 6 (i.e., each backward pass in the sequence of N−1 backward passes) for the concatenated sequence (i.e., in the first token sequence). Accordingly, for each
M
=
2
,
…
,
N
, XU predicts the Mth API call (i.e., the Mth completion token of the Mth prompt/completion pair) using the preceding M-1 goal-action pairs and the current Mth goal as context (i.e., using the preceding M-1 prompt/completion pairs as context), thereby performing a (M-1)-shot training, as shown in XU’s Figure 6.)
Regarding Claim 13:
XU teaches the elements of claim 1 as outlined above. XU further teaches:
wherein the ML model includes a transformer model; and (XU [page 17, section B.1 Models] teaches: "As listed in Table 8, we select a set of representative LLMs from both closed-source and open-source community. The closed models are the (Generative Pre-trained Transformer) GPT series from OpenAI, especially the GPT-3[22] and its successors[13]. […] Due to the lack of detailed information about its training, we are motivated to study methods to build models achieving similar capabilities, especially using open-source models. We select the representative and the most advanced open-source models from recent years in our work. They are all decoder-only models, based on transformers[51] architecture (i.e., the ML model includes a transformer language model). Bloomz[52] is the largest open-source LLM built upon the large-scale multilingual pretrained BLOOM[53]. Bloomz is funtuned on xP3[52], a crosslingual task mixture, for crosslingual generalization to unseen tasks and languages. StarCoder[33] is a family of models developed for purely code generation and synthesis with 8K context length. They exhibit superior performance on common code generation benchmarks.")
wherein performing the first in-context training pass using the first token sequence includes training the transformer model using the first token sequence. (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, the first in-context training pass can be interpreted as processing one concatenated training sample of length 2048 within the initial batch of training samples when training of the LLM model.)
Regarding Claim 14:
XU teaches the elements of claim 1 as outlined above. XU further teaches:
wherein the target task is to respond to queries of a target topic, and wherein the set of in-context prompt/completion pairs is a set of query/answer examples of the same target topic. (XU [page 15, section A.1 OpenWeather] teaches: "The model’s generation will be considered successful if the output matches the expected result precisely." XU [page 21, Table 10] teaches various tasks (i.e., target topic) such as OpenWeather, which contain goals related to answering questions about the weather, and The Cat API, where the goal is to respond to a query related to cats.)
Regarding Claim 15:
XU teaches the elements of claim 1 as outlined above. XU further teaches:
wherein each prompt/completion pair in the set of in-context prompt/completion pairs has the same format as the other prompt/completion pairs in the set of in-context prompt/completion pairs. (XU [page 5, section 4.1 Multi-tool model alignment with programmatic data curation] teaches: "We create a handful of templates consisting of goal descriptions and corresponding API calls. These templates contain one or more placeholder pairs. Each of these pairs maps to a key word in the goal and an argument in the corresponding API calls.")
Regarding Claim 16:
The claim recites similar limitations as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale. XU further teaches:
An apparatus for performing in-context training for a machine learning (ML) model, the apparatus comprising: one or more processors; a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the apparatus to; (XU [page 19, section B.2 Evaluation] teaches: “We evaluate all the models on a mixture of GPUs and RDUs[66, 67, 68]. In particular, the 176b-parameter bloomz is evaluated on RDU, while all the other models are evaluated on NVIDIA A100 GPUs with 80GB RAM.” XU [page 22, section C.3 Training details] teaches: “We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM.”)
perform a sequence of in-context training passes using the set of token sequences to train the ML model to generate a next token in accordance with the target task. (XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 21, Table 10] teaches various tasks for training the model (i.e., in accordance with the target task) such as OpenWeather, which contain goals related to answering questions about the weather, and The Cat API, where the goal is to respond to a query related to cats. Examiner’s note: Under BRI, perform a sequence of in-context training passes can be interpreted as backpropagating the loss (i.e., to train the ML model) at the blue API-call actions (i.e., to generate a next token) for Goal 1 through Goal N of XU’s Figure 6 for the concatenated sequence (i.e., using the set of token sequences).)
Regarding Claim 17:
XU teaches the elements of claim 16 as outlined above. Additionally, the claim recites similar limitations as corresponding claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale. XU further teaches:
wherein the set of token sequences includes a first token sequence, and wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to construct the first token sequence by: (XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner’s note: Under BRI, the set of token sequences includes a first token sequence can be interpreted as the dataset used, which includes the goal-action pair sequences that will be used for training.)
wherein the first token sequence is used to perform a first in-context training pass in the sequence of in-context training passes. (XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." Examiner’s note: Under BRI, the first token sequence is used to perform a first in-context training pass can be interpreted as processing the first concatenated training sample of length 2048 within the initial batch of training samples, and in the sequence of in-context training passes can be interpreted as continuing to processes the remaining concatenated sequences in the batch.)
Regarding Claim 20:
XU teaches the elements of claim 17 as outlined above. Additionally, the claim recites similar limitations as corresponding claim 12 and is rejected for similar reasons as claim 12 using similar teachings and rationale.
Regarding Claim 22:
The claim recites similar limitations as corresponding claims 1 and 16 and is rejected for similar reasons as claims 1 and 16 using similar teachings and rationale.
Regarding Claim 23:
XU teaches the elements of claim 22 as outlined above. XU further teaches.
wherein the ML model includes a transformer-based language model; and (XU [page 17, section B.1 Models] teaches: "As listed in Table 8, we select a set of representative LLMs from both closed-source and open-source community. The closed models are the (Generative Pre-trained Transformer) GPT series from OpenAI, especially the GPT-3[22] and its successors[13]. […] Due to the lack of detailed information about its training, we are motivated to study methods to build models achieving similar capabilities, especially using open-source models. We select the representative and the most advanced open-source models from recent years in our work. They are all decoder-only models, based on transformers[51] architecture (i.e., the ML model includes a transformer-based language model). Bloomz[52] is the largest open-source LLM built upon the large-scale multilingual pretrained BLOOM[53]. Bloomz is funtuned on xP3[52], a crosslingual task mixture, for crosslingual generalization to unseen tasks and languages. StarCoder[33] is a family of models developed for purely code generation and synthesis with 8K context length. They exhibit superior performance on common code generation benchmarks.")
wherein performing the first in-context training pass using the first token sequence includes training the transformer-based language model. (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, the first in-context training pass can be interpreted as processing one concatenated training sample of length 2048 within the initial batch of training samples when training of the LLM model.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 5, 7-8, 18-19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over XU in view of BROWN ("Language Models are Few-Shot Learners"), hereafter BROWN.
Regarding Claim 3:
XU teaches the elements of claim 2 as outlined above. XU is not relied upon for teaching, but BROWN teaches:
wherein the first subset of the in-context prompt/completion pairs is selected such that the combined size of the selected first subset of the in-context prompt/completion pairs is as close to the predetermined size as possible without exceeding the predetermined size to maximize the usage of the context window and to include as many different prompt/completion pairs as possible. (BROWN [page 43, section B Details of Model Training] teaches: "During training we always train on sequences of the full
n
c
t
x
=
2048
token context window, packing multiple documents into a single sequence when documents are shorter than 2048, in order to increase computational efficiency.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of XU and BROWN before them, to include BROWN’s training on sequences of the full context window in XU’s LLM tool manipulation method. By applying BROWN’s full context window usage, one could modify the 2048 token length of the concatenated training samples in XU to increase the context window usage to be the size of XU’s context window of 4096 token length. One would have been motivated to make such a combination in order to increase computational efficiency (BROWN [page 43, section B Details of Model Training]).
Regarding Claim 5:
XU teaches the elements of claim 4 as outlined above. XU further teaches:
wherein constructing the second token sequence based on the set of in-context prompt/completion pairs includes: selecting a second subset of the set of in-context prompt/completion pairs, wherein: (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6." XU [page 22, Figure 6] teaches: "We concatenate several examples (i.e., a second subset) into a single training sample (i.e., constructing the second token sequence based on the set of in-context prompt/completion pairs) and backpropagate through the loss on the blue actions in every example." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. […] In each sample, all the goal-action pairs are from the same task." Examiner's note: Under BRI, selecting a second subset of the set of in-context prompt/completion pairs can be interpreted as choosing the several examples as a second concatenated training sample from the dataset to be concatenated as one input sequence for training the LLM.)
concatenating the selected second subset of the in-context prompt/completion pairs to form the second token sequence. (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls (i.e., concatenating the selected second subset of the in-context prompt/completion pairs) as one input sequence (i.e., to form the second token sequence) to the LLMs. We use an all-shot loss formulation illustrated in Figure 6." XU [page 22, Figure 6] teaches: "We concatenate several examples into a single training sample and backpropagate through the loss on the blue actions in every example." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner’s note: Under BRI, the second token sequence can be interpreted as a second concatenated training sample when training the LLM during the first epoch.)
XU is not relied upon for teaching, but BROWN teaches: the second subset of in-context prompt/completion pairs does not include any in-context prompt/completion pair in the first subset of in-context prompt/completion pairs; (BROWN [page 43, section B Details of Model Training] teaches: "Data are sampled without replacement during training (until an epoch boundary is reached) to minimize overfitting.")
the combined size of the selected second subset of the in-context prompt/completion pairs is as close to the predetermined size as possible to maximize the usage of the context window and to include as many different and unused prompt/completion pairs as possible; (BROWN [page 43, section B Details of Model Training] teaches: "During training we always train on sequences of the full
n
c
t
x
=
2048
token context window, packing multiple documents into a single sequence when documents are shorter than 2048, in order to increase computational efficiency.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of XU and BROWN before them, to include BROWN’s training on sequences of the full context window using data sampled without replacement in XU’s LLM tool manipulation method. By applying BROWN’s full context window usage, one could modify the 2048 token length of the concatenated training samples in XU to increase the context window usage to be the size of XU’s context window of 4096 token length. One would have been motivated to make such a combination in order to increase computational efficiency and minimize overfitting (BROWN [page 43, section B Details of Model Training]).
Regarding Claim 7:
XU teaches the elements of claim 6 as outlined above. XU further teaches:
determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs from all of the previous in-context training passes; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, determining if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs from all of the previous in-context training passes can be interpreted as continuing to process the next concatenated training sample from the dataset after completing the previous concatenated training samples.)
if so, constructing one or more additional token sequences from the unselected prompt/completion pairs until the set of in-context prompt/completion pairs are fully exhausted; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples (i.e., constructing one or more additional token sequences), we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” Examiner's note: XU [page 22, section C.3 Training details] teaches finetuning the models using a batch of 16 training samples from the dataset for a total of 8 epochs. Under BRI, constructing one or more additional token sequences from the unselected prompt/completion pairs can be interpreted as constructing a an additional concatenated training sample from the dataset for processing, wherein the additional concatenated training sample concatenates API documents and multiple pairs of goal and API calls as one input sequence to the LLM.)
performing one or more additional in-context training passes using the one or more additional token sequences to further train the ML model; (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 15, section A.1 OpenWeather] teaches: "The model’s generation will be considered successful if the output matches the expected result precisely." Examiner's note: Under BRI, performing one or more additional in-context training passes can be interpreted as processing additional concatenated training samples during training, each including a token sequence (i.e., using the one or more additional token sequences to further train the ML model) of token length of 2048 for processing.)
otherwise, terminating the in-context training for the ML model, […] (XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs." Examiner's note: Under BRI, terminating the in-context training for the ML model can be interpreted as completing one epoch of model alignment training.)
XU is not relied upon for teaching, but BROWN teaches: wherein there is no duplicated prompt/completion pair used by any two in-context training passes in the set of in-context training passes, thereby improving a model training efficiency based on the set of in-context prompt/completion pairs. (BROWN [page 43, section B Details of Model Training] teaches: "Data are sampled without replacement during training (until an epoch boundary is reached) to minimize overfitting.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of XU and BROWN before them, to include BROWN’s data sampling without replacement during training in XU’s LLM tool manipulation method. By applying BROWN’s data sampling without replacement during training, one could concatenate examples without repeating goal-actions pairs for training the LLM model. One would have been motivated to make such a combination in order to minimize overfitting (BROWN [page 43, section B Details of Model Training]).
Regarding Claim 8:
XU in view of BROWN teaches the elements of claim 7 as outlined above. XU further teaches:
wherein a training time associated with training the ML model is proportional to a first number of in-context prompt/completion pairs in the set of in-context prompt/completion pairs divided by an average number of selected prompt/completion pairs of a set of constructed token sequences associated with the set of in-context training passes. (XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner’s note: XU [page 21, Table 10] teaches tasks and their corresponding number of training samples: OpenWeather (1800), The Cat API (1800), Home Search (1800), Trip Booking (1800), Google Sheets (118), VirtualHome (512), WebShop (1800), and Tabletop (74). Adding all of the training samples yields 9,704 total training samples. Each training sample shown in XU’s Table 10 corresponds to a single goal-action pair (i.e., a prompt completion pair), so the dataset contains 9,704 prompt/completion pairs. XU concatenates several pairs into each input sequence for all-shot training, as shown in XU [page 22, Figure 6]. One of ordinary skill in the art would recognize that the total training time is proportional to the number of constructed token sequences processed, which is equal to the total number of prompt/completion pairs (9,704) divided by the average number of pairs concatenated per token sequence. Therefore, under BRI, processing XU’s goal-action pairs as a set of constructed token sequences teaches this limitation.)
Regarding Claim 18:
XU teaches the elements of claim 17 as outlined above. Additionally, the claim recites similar limitations as corresponding claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding Claim 19:
XU teaches the elements of claim 17 as outlined above. XU further teaches:
wherein after constructing the first token sequence, the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to: (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 19, section B.2 Evaluation] teaches: “We evaluate all the models on a mixture of GPUs and RDUs[66, 67, 68]. In particular, the 176b-parameter bloomz is evaluated on RDU, while all the other models are evaluated on NVIDIA A100 GPUs with 80GB RAM.” Examiner's note: Under BRI, after constructing the first token sequence can be interpreted as constructing the first concatenated training sample.)
determine if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs by the first in-context training pass; (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner's note: Under BRI, determine if there are unselected prompt/completion pairs in the set of in-context prompt/completion pairs by the first in-context training can be interpreted as continuing to process the next concatenated training sample from the dataset after completing the first concatenated training sample until the dataset has been processed.)
in response to determining that there are unselected prompt/completion pairs, select a second subset of the set of in-context prompt/completion pairs, […] (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples (i.e., constructing a second token sequence), we concatenate API documents and multiple pairs of goal and API calls as one input sequence to the LLMs. We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. We use a max sequence length of 2048 without packing and mix the data from all the tasks into a single dataset with random shuffling. In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 3, Large language model] teaches: “Large language model Autoregressive language models encode probabilities of the next word
x
N
+
1
given
x
0
,
x
1
,
…
,
x
N
as the context sequence [21]. By sampling from this conditional probability
p
x
N
+
1
x
0
,
x
1
,
…
,
x
N
)
iteratively, it generates language continuations from given contexts.” Examiner's note: XU [page 22, section C.3 Training details] teaches finetuning the models using a batch of 16 training samples from the dataset for a total of 8 epochs. Under BRI, in response to determining that there are unselected prompt/completion pairs can be interpreted as having remaining goal-action pairs from the dataset for processing during an epoch. Additionally, select a second subset of the set of in-context prompt/completion pairs can be interpreted as choosing the several examples to construct a second training sample from the dataset to be concatenated as one input sequence for training the LLM for a subsequent in-context training pass.)
concatenate the selected second subset of the in-context prompt/completion pairs to form the second token sequence, (XU [page 20, section C.2 All-shot loss] teaches: "To construct the training samples, we concatenate API documents and multiple pairs of goal and API calls (i.e., concatenate the selected second subset of the in-context prompt/completion pairs) as one input sequence (i.e., to form the second token sequence) to the LLMs. We use an all-shot loss formulation illustrated in Figure 6." XU [page 22, Figure 6] teaches: "We concatenate several examples into a single training sample and backpropagate through the loss on the blue actions in every example." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." Examiner’s note: Under BRI, the second token sequence can be interpreted as a second concatenated training sample when training the LLM during the first epoch.)
wherein the second token sequence is used to perform a second in-context training pass in the sequence of in-context training passes. (XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." XU [page 22, section C.3 Training details] teaches: "We finetune each model on the same dataset created with the method described in Section C.1 for 8 epochs. [...] In each sample, all the goal-action pairs are from the same task. [...] We use a batch size of 16 and a constant learning rate of 1e − 5 for each model and train on an internal cluster of 4 A100 GPU’s, each with 80GB RAM." XU [page 15, section A.1 OpenWeather] teaches: "The model’s generation will be considered successful if the output matches the expected result (i.e., to perform the target task) precisely." XU [page 20, section C.2 All-shot loss] teaches: "We use an all-shot loss formulation illustrated in Figure 6 which learns to generate the API calls for every goal in a sequence. We use this loss formulation because it empirically delivers better success rate, especially when using in-context demonstrations, than the conventional loss which only backpropagates the loss associated with the API calls for the last goal." Examiner's note: Under BRI, perform a second in-context training pass can be interpreted as processing a second concatenated training sample of token length 2048 (i.e., the second token sequence is used to perform a second in-context training pass in the sequence of in-context training passes) within the initial batch of training samples. Additionally, under BRI, a second in-context training pass in the sequence of in-context training passes can be interpreted as processing Goal 1 through Goal N of XU’s Figure 6 for a second concatenated training sample.)
XU is not relied upon for teaching, but BROWN teaches: […] wherein the combined size of the selected second subset of the in-context prompt/completion pairs is equal to or substantially equal to the predetermined size of the context window; (BROWN [page 43, section B Details of Model Training] teaches: "During training we always train on sequences of the full
n
c
t
x
=
2048
token context window, packing multiple documents into a single sequence when documents are shorter than 2048, in order to increase computational efficiency.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of XU and BROWN before them, to include BROWN’s training on sequences of the full context window in XU’s LLM tool manipulation method. By applying BROWN’s full context window usage, one could modify the 2048 token length of the concatenated training samples in XU to increase the context window usage to be the size of XU’s context window of 4096 token length. One would have been motivated to make such a combination in order to increase computational efficiency (BROWN [page 43, section B Details of Model Training]).
Regarding Claim 21:
XU teaches the elements of claim 17 as outlined above. Additionally, the claim recites similar limitations as corresponding claim 8 and is rejected for similar reasons as claim 8 using similar teachings and rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Alvaro S Laham Bauzo whose telephone number is (571)272-5650. The examiner can normally be reached Mon-Fri 7:30 AM - 11:00 AM | 1:00 PM - 5:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached on (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S.L./Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights