DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Drawings
The applicant’s submitted drawings appear to be acceptable for examination purposes. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the drawings.
Information Disclosure Statement
The information disclosure statement filed 16 April 2024 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered.
Claim Objections
Claim 20 is objected to because of the following informalities: “agent to produces an intermediate result” appears as though it should be “agent to produce an intermediate result” or “agent that produces an intermediate result” or similar. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12, 13, 17, 18, and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 12 recites the limitation “the updating attending to relations among…” The intended scope of the claim is not clear because it is not clear what is meant by “attending to relations” among the listed tasks/responses. For the purposes of examination, the examiner assumes that the updating includes determining some relation between them. Additionally, claim 12 recites the limitations the first candidate student-generated task, and the second candidate-generated task in lines 6-7. There is insufficient antecedent basis for this limitation in the claim (Examiner’s Note: it appears as though they should both be student-generated responses).
Claim 13 recites the limitation “a machine-trained model having parameters produced by the method of claim 1.” The intended scope of the claim is not clear because the method of claim 1 does not only produce a set of parameters, and thus it is not clear what portions of the method of claim 1 are included. For the purposes of examination, the examiner has assumed that the machine-trained model has the updated parameters of the student model and includes all of the prior steps in producing them.
As per claim 17, see the rejection of claim 12 above.
Claim 18 depends upon claim 17, and thus includes the aforementioned limitation(s).
Claim 20 recites the limitation “A computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform operations comprising.” The intended scope of the claim is not clear because it is not clear whether the claim is drawn to the computer-readable storage medium or the processing system, or some combination of both. For the purposes of examination, the examiner has assumed that it is intended to be a system including a computer-readable storage medium storing instructions and a processing system executing the instructions.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) mental processes and/or mathematical concepts. This judicial exception is not integrated into a practical application and does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, as described below.
Step 1 for all claims:
Under the first part of the analysis, claims 1-13 recite a method, claims 14-19 recite a device, and claim 20 recites a manufacture. Accordingly, these claims fall within the four statutory categories of invention and the analysis proceeds to Step 2A, prongs 1 and 2, and Step 2B, as described below.
As per claim 1:
Under step 2A, prong 1, the claim recites an abstract idea including the following mental process elements:
generating plural tasks based on the content – a data scientist looks at content and creates a plurality of tasks based on the content.
transforming the plural tasks into plural teacher-generated responses – the data scientist creates teacher responses based upon the tasks.
transforming the plural tasks into student-generated responses – the data scientist creates student responses based upon the tasks.
If a claim, under the broadest reasonable interpretation covers a mathematical relationship between variables or numbers, a numerical formula or equation, or a mathematical calculation, it will be considered as falling within the “mathematical concepts” grouping of abstract ideas. If a claim, under the broadest reasonable interpretation covers concepts that can be performed in the human mind, or by a human using a pen and paper, including observation, evaluation, judgment, or opinion, it will be considered as falling within the “mental processes” grouping of abstract ideas. Additionally, performing mathematical calculations using a formula that could be practically performed in the human mind may be considered to fall within both the mathematical concepts grouping and the mental process grouping. See MPEP § 2106.04(a)(2).
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
A method for training a student model comprising: – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
obtaining a source item that contains content – this is recited at a high level of generality and amounts to insignificant extra-solution activity as data gathering/storage that is limited to a particular type of data, generally linking the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(g) and (h), and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
using a group of example-generating agents – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using a teacher model having a first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using the student model, the student model having a second size that is less than the first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the student model producing the student generated-responses independently of information that describes at least part of processes by which the teacher model has produced the plural teacher-generated responses – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
updating parameters of the student model based on the student-generated responses and corresponding teacher-generated responses – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
A method for training a student model comprising: – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
obtaining a source item that contains content – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data). The courts have also found limitations directed to obtaining and storing information electronically, recited at a high level of generality, to be well-understood, routine, and conventional. See MPEP § 2106.05(d)(II) “receiving or transmitting data over a network,” "electronic record keeping,” and "storing and retrieving information in memory.”
using a group of example-generating agents – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using a teacher model having a first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using the student model, the student model having a second size that is less than the first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the student model producing the student generated-responses independently of information that describes at least part of processes by which the teacher model has produced the plural teacher-generated responses – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
updating parameters of the student model based on the student-generated responses and corresponding teacher-generated responses – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 2:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein at least some of the tasks produced by the group of example-generating agents include questions derived from the content of the source item – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein at least some of the tasks produced by the group of example-generating agents include questions derived from the content of the source item – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 3:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein the group of example-generating agents are organized in a graph, the graph specifying input-output connections among the group of example-generating agents – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein the group of example-generating agents are organized in a graph, the graph specifying input-output connections among the group of example-generating agents – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 4:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein at least one task is produced using a first example-generating agent that produces an intermediate result, and a second example-generating agent that performs further processing on the intermediate result – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein at least one task is produced using a first example-generating agent that produces an intermediate result, and a second example-generating agent that performs further processing on the intermediate result – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 5:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein the group of example-generating agents performs operations that include one or more of:
generating questions based on the source item; and/or
rewriting the source item in a specified style; and/or
generating code based on the source item; and/or
summarizing the source item; and/or
specifying an order of statements in the source item; and/or
adding detail to the source item – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein the group of example-generating agents performs operations that include one or more of:
generating questions based on the source item; and/or
rewriting the source item in a specified style; and/or
generating code based on the source item; and/or
summarizing the source item; and/or
specifying an order of statements in the source item; and/or
adding detail to the source item – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 6:
The claim recites the following additional mental process elements:
wherein, for a particular task, … produces an original response based on the particular task – the data scientist produces an original response for a specified task.
and wherein … refines the original response to produce a refined response having enhanced accuracy compared to the original response – after consideration, the data scientist refines their original response to be more accurate.
and wherein … produces one or more student responses for the particular task independently of information that describes how the teacher model refined the original response – the data scientist creates separate student responses for the particular task.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 7:
The claim recites the following additional mental process elements:
wherein, for a particular task, … produces an original response based on a first prompt that provides a first instruction – the data scientist produces an original response for a specified task based upon a first prompt and instruction.
and wherein … produces a refined response based on a second prompt that provides a second instruction that is different than the first instruction – after consideration, the data scientist refines their original response based upon an additional prompt and instruction.
and wherein … produces one or more student responses for the particular task independently of information that describes at least some aspects of the first instruction and/or the second instruction – the data scientist separately produces student responses.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 8:
The claim recites the following additional mental process elements:
choosing the first instruction based on an assessment of a class associated with the particular task – the data scientist looks at a class associated with the particular task and chooses the first instruction.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
The claim does not include any additional elements, under step 2A prong two, or step 2B, except those listed above in prior claim(s). Accordingly, at step 2A, prong two, the claim as a whole does not integrate the judicial exception into a practical application. See MPEP § 2106.04(d). Furthermore, at step 2B, the claim elements both individually and in combination do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 9:
The claim recites the following additional mental process elements:
producing one or more candidate student-generated responses to the particular task – the data scientist produces candidate student responses for a particular task.
assessing suitability of each of the one or more candidate student-generated responses …to provide preference information – the data scientist assesses the suitability of each candidate response and provides their preferences.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein the transforming of the tasks by the student model includes, for a particular task: – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
using the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the updating of the parameters of the student model being based on the one or more student-generated responses, a teacher-generated response to the particular task, and the preference information – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the parameters of the student model being associated with a policy applied by the student model – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the producing, assessing, and updating being repeated one or more times to successively improve performance of the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
in each repetition of the updating, the parameters of a next version of the student model being based, in part, on parameters of a current version of the student model – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein the transforming of the tasks by the student model includes, for a particular task: – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
using the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the updating of the parameters of the student model being based on the one or more student-generated responses, a teacher-generated response to the particular task, and the preference information – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the parameters of the student model being associated with a policy applied by the student model – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the producing, assessing, and updating being repeated one or more times to successively improve performance of the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
in each repetition of the updating, the parameters of a next version of the student model being based, in part, on parameters of a current version of the student model – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 10:
The claim recites the following additional mental process and/or mathematical concept elements:
wherein the method performs optimizations based on the preference information to find a stable policy for application by the student model, among competing policies, that satisfies a Nash equilibrium, the Nash equilibrium being a state in which there is no incentive to move from the stable policy to a competing policy – the data scientist finds and selects a stable policy, among competing policies, that satisfies the Nash equilibrium. Alternatively/additionally – comparing incentives of competing policies is a mathematical calculation.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
The claim does not include any additional elements, under step 2A prong two, or step 2B, except those listed above in prior claim(s). Accordingly, at step 2A, prong two, the claim as a whole does not integrate the judicial exception into a practical application. See MPEP § 2106.04(d). Furthermore, at step 2B, the claim elements both individually and in combination do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 11:
The claim recites the following additional mathematical concept elements:
wherein the updating uses a loss function that bypasses an operation of explicitly generating a reward function and an operation of performing reward maximization – the loss function is a mathematical formula (see, e.g., paras [0044]-[0046] of the specification as filed).
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
The claim does not include any additional elements, under step 2A prong two, or step 2B, except those listed above in prior claim(s). Accordingly, at step 2A, prong two, the claim as a whole does not integrate the judicial exception into a practical application. See MPEP § 2106.04(d). Furthermore, at step 2B, the claim elements both individually and in combination do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 12:
The claim recites the following additional mental process and/or mathematical concept elements:
generating a subset for the particular task that includes a first candidate student-generated response that is determined to be a suitable response to the particular task and a second candidate student-generated response that is determined to be a less suitable response to the task compared to the first candidate student-generated response – the data scientist creates a subset of candidate responses for a particular task based upon their determinations of suitability for each response. Alternatively/additionally – comparing suitability scores is a mathematical calculation.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the updating attending to relations among the particular task, the first candidate student-generated task, and the second candidate-generated task as specified in the subset – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the updating attending to relations among the particular task, the first candidate student-generated task, and the second candidate-generated task as specified in the subset – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 13:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
a machine-trained model having parameters produced by the method of claim 1 – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
a machine-trained model having parameters produced by the method of claim 1 – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 14:
Under step 2A, prong 1, the claim recites an abstract idea including the following mental process and/or mathematical concept elements:
providing a task that is based on the content provided by a source item – a data scientist looks at content and creates a task based on the content.
transforming the task into a teacher-generated response – the data scientist creates teacher responses based upon the task.
producing one or more candidate student-generated responses to the task – the data scientist creates candidate student responses based upon the task.
assessing suitability of each of the one or more candidate student-generated responses …to provide preference information – the data scientist assesses the suitability of each candidate response and provides their preferences.
the updating using a loss function that bypasses an operation of explicitly generating a reward function and an operation of reward maximization – the loss function is a mathematical formula
If a claim, under the broadest reasonable interpretation covers a mathematical relationship between variables or numbers, a numerical formula or equation, or a mathematical calculation, it will be considered as falling within the “mathematical concepts” grouping of abstract ideas. If a claim, under the broadest reasonable interpretation covers concepts that can be performed in the human mind, or by a human using a pen and paper, including observation, evaluation, judgment, or opinion, it will be considered as falling within the “mental processes” grouping of abstract ideas. Additionally, performing mathematical calculations using a formula that could be practically performed in the human mind may be considered to fall within both the mathematical concepts grouping and the mental process grouping. See MPEP § 2106.04(a)(2).
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
A computing system for training a student model, comprising: – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
an instruction data store for storing computer-readable instructions – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
and a processing system for executing the computer-readable instructions in the data store to perform operations including: – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
obtaining a source item that contains content – this is recited at a high level of generality and amounts to insignificant extra-solution activity as data gathering/storage that is limited to a particular type of data, generally linking the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(g) and (h), and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
using a teacher model having a first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using a student model that has a second size that is less than the first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
and updating parameters of the student model based on the one or more student-generated responses, the teacher-generated response, and the preference information, the parameters of the student model being associated with a policy applied by the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the producing, assessing, and updating being repeated one or more times to successively improve performance of the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
in each repetition of the updating, the parameters of a next version of the student model being based, in part, on parameters of a current version of the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
A computing system for training a student model, comprising: – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
an instruction data store for storing computer-readable instructions – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
and a processing system for executing the computer-readable instructions in the data store to perform operations including: – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
obtaining a source item that contains content – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data). The courts have also found limitations directed to obtaining and storing information electronically, recited at a high level of generality, to be well-understood, routine, and conventional. See MPEP § 2106.05(d)(II) “receiving or transmitting data over a network,” "electronic record keeping,” and "storing and retrieving information in memory.”
using a teacher model having a first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using a student model that has a second size that is less than the first size – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
and updating parameters of the student model based on the one or more student-generated responses, the teacher-generated response, and the preference information, the parameters of the student model being associated with a policy applied by the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
the producing, assessing, and updating being repeated one or more times to successively improve performance of the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
in each repetition of the updating, the parameters of a next version of the student model being based, in part, on parameters of a current version of the student model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 15, see the rejection of claim 10 above.
As per claim 16:
The claim recites the following additional mental process elements:
to assign one or more points to a particular candidate student-generated response based on an extent to which the particular candidate student-generated response has one or more specified characteristics – the data scientist assigns scores to each candidate student response based on their specified characteristics.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein the assessing the suitability is performed by prompting the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein the assessing the suitability is performed by prompting the teacher model – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 17, see the rejection of claim 12 above.
As per claim 18:
The claim recites the following additional mental process and/or mathematical concept elements:
wherein the operations further comprise choosing the first candidate student-generated response in response to determining that the first candidate student-generated response has a first score that satisfies a prescribed test of suitability, and choosing the second candidate student-generated response upon determining that the second student-generated response has a second score that is worse than the first score by at least a prescribed amount – the data scientist chooses a candidate student response based upon their scores. Alternatively/additionally – comparing scores and calculating the difference between scores are mathematical calculations.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
The claim does not include any additional elements, under step 2A prong two, or step 2B, except those listed above in prior claim(s). Accordingly, at step 2A, prong two, the claim as a whole does not integrate the judicial exception into a practical application. See MPEP § 2106.04(d). Furthermore, at step 2B, the claim elements both individually and in combination do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 19:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein the providing the task includes using one or more example-generating agents of a group of example-generating responses to produce the task based on the source item – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein the providing the task includes using one or more example-generating agents of a group of example-generating responses to produce the task based on the source item – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 20:
Under step 2A, prong 1, the claim recites an abstract idea including the following mental process concept elements:
generating plural tasks based on the content – a data scientist looks at content and creates tasks based on the content.
If a claim, under the broadest reasonable interpretation covers a mathematical relationship between variables or numbers, a numerical formula or equation, or a mathematical calculation, it will be considered as falling within the “mathematical concepts” grouping of abstract ideas. If a claim, under the broadest reasonable interpretation covers concepts that can be performed in the human mind, or by a human using a pen and paper, including observation, evaluation, judgment, or opinion, it will be considered as falling within the “mental processes” grouping of abstract ideas. Additionally, performing mathematical calculations using a formula that could be practically performed in the human mind may be considered to fall within both the mathematical concepts grouping and the mental process grouping. See MPEP § 2106.04(a)(2).
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
A computer-readable storage medium for storing computer-readable instructions – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
a processing system executing the computer-readable instructions to perform operations comprising: – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
obtaining a source item that contains content – this is recited at a high level of generality and amounts to insignificant extra-solution activity as data gathering/storage that is limited to a particular type of data, generally linking the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(g) and (h), and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
using different combinations of plural example-generating agents provided by a group of example-generating agents that perform different respective functions – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
and storing the plural tasks in a data store – this is recited at a high level of generality and amounts to insignificant extra-solution activity as data gathering/storage that is limited to a particular type of data, generally linking the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(g) and (h), and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the generating of a particular task of the plural tasks including:
using a first example-generating agent to produces an intermediate result based on the source item – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using a second example-generating agent to perform further processing on the intermediate result – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
A computer-readable storage medium for storing computer-readable instructions – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
a processing system executing the computer-readable instructions to perform operations comprising: – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
obtaining a source item that contains content – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data). The courts have also found limitations directed to obtaining and storing information electronically, recited at a high level of generality, to be well-understood, routine, and conventional. See MPEP § 2106.05(d)(II) “receiving or transmitting data over a network,” "electronic record keeping,” and "storing and retrieving information in memory.”
using different combinations of plural example-generating agents provided by a group of example-generating agents that perform different respective functions – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
and storing the plural tasks in a data store – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data). The courts have also found limitations directed to obtaining and storing information electronically, recited at a high level of generality, to be well-understood, routine, and conventional. See MPEP § 2106.05(d)(II) “receiving or transmitting data over a network,” "electronic record keeping,” and "storing and retrieving information in memory.”
the generating of a particular task of the plural tasks including:
using a first example-generating agent to produces an intermediate result based on the source item – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
using a second example-generating agent to perform further processing on the intermediate result – this amounts to no more than a recitation of the words "apply it" (or an equivalent) including mere instructions to implement an abstract idea or other exception on a computer, and/or at most generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(f) and (h).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Baker (US 2020/0311572).
As per claim 20, Baker teaches a computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform operations [the method/system may be implemented as program instructions stored in one or more memories and executed by one or more connected processors (para. 0046, etc.)] comprising:
obtaining a source item that contains content [data is obtained from one or more data sources (paras. 0025, 0117; figs. 1, 12B; etc.); where the data source data can include various types of data such as speech (content) (paras. 0066-68, etc.)];
generating plural tasks based on the content using different combinations of plural example-generating agents provided by a group of example-generating agents that perform different respective functions [the system can include multiple reference systems (fig. 4, etc.) and/or each reference system can include multiple models (figs. 12A-D, etc.), where reference system(s) generate tasks/training data for the student system (paras. 0114-115, etc.) from the data source(s) (paras. 0025, 0117, etc.) and can include different combinations/mixes of machine learning model types (paras. 0049-51, etc.); where the different models and model types are different combinations of plural example-generating agents]; and
storing the plural tasks in a data store [the data used by the learning modules can be stored in different storage for different users (para. 0055, etc.)],
the generating of a particular task of the plural tasks including:
using a first example-generating agent to produces an intermediate result based on the source item; and
using a second example-generating agent to perform further processing on the intermediate result [each reference system can include multiple models, such as data generators, classifiers, weighted selection, noise generation, etc., (figs. 12A-D; etc.); where the noise/data generator is an agent that produces an intermediate result that is sent to the classifiers (additional/second agents) which further process the outputs to produce classification outputs (additional intermediate results) which are sent to the weighted random selection/LES control (additional/third agent) to process those results and produce the labeled output tasks/training data].
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 4, 5, 9, 11-14, 16, 17, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (Direct Language Model Alignment from Online AI Feedback, Feb 2024, pgs. 1-18 – cited in an IDS) in view of Baker (US 2020/0311572).
As per claim 1, Guo teaches a method for training a student model comprising:
obtaining a source item that contains content [data prompts are provided to the models from a prompt distribution (pg. 2, fig. 1; pgs. 4-5, section 4.1, first paragraph; etc.) which are source items containing content (text, etc.)];
generating plural tasks based on the content [prompts (tasks) may be sampled from a prompt distribution (etc.), where task-based prompt datasets (plural tasks) may be gathered and provided to the models (pg. 2, section 2, “Pairwise preference collection”; pgs. 4-5, section 4.1, first paragraph; etc.)];
transforming the plural tasks into plural teacher-generated responses using a teacher model [the LLM annotator (teacher model) generates labels/annotations (teacher-generated responses) for the responses from the (student) LLM being aligned, based upon the prompt 𝑥 (the plural tasks) (pg. 2, fig. 1; etc.); which is transforming the plural tasks (prompts) into plural teacher-generated responses (annotations/labels) using the teacher model] having a first size [the LLM annotator (teacher) can be of a larger size than the student LLM (pg. 7, section 4.5; pgs. 8-9, section 5, “Self-annotating models”; etc.)];
transforming the plural tasks into student-generated responses using the student model [the LLM being aligned (student model) receives the prompts (plural tasks) and produces multiple responses (pg. 2, fig. 1; etc.); which is transforming the plural tasks (prompts) into student-generated responses using the student model], the student model having a second size that is less than the first size [the LLM annotator (teacher) can be of a larger size than the student LLM (pg. 7, section 4.5; pgs. 8-9, section 5, “Self-annotating models”; etc.)];
the student model producing the student generated-responses independently of information that describes at least part of processes by which the teacher model has produced the plural teacher-generated responses [the student model is independent of the teacher model (pg. 2, fig. 1; etc.) where the teacher model specification describing the process by which the teacher model produces responses is separate/independent from the student]; and
updating parameters of the student model based on the student-generated responses and corresponding teacher-generated responses [the model parameters and policies of the (student) LLM being aligned are updated from responses of the student model annotated with the responses of the teacher model (pg. 2, fig. 1; pgs. 2-3, section 2, “Direct alignment from preference (DAP) methods”; pg. 4, Algorithm 1; etc.)].
While Guo teaches utilizing multiple prompts from different tasks (see above), it has not been relied upon for teaching generating plural tasks based on the content using a group of example-generating agents.
Baker teaches generating plural tasks based on the content using a group of example-generating agents [reference system(s) generate tasks/training data for the student system (paras. 0114-115, etc.) from the data source(s) (paras. 0025, 0117, etc.); where the reference systems generating tasks from the data content are the group of example-generating agents].
Guo and Baker are analogous art, as they are within the same field of endeavor, namely improving/optimizing student models using teacher models.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize the reference systems to generate the plural tasks for the student and teacher models, as taught by Baker, in the generating task-based prompts for the student and teacher models in the system/method taught by Guo.
Baker provides motivation as [using reference systems to generate data can provide a specific and well controlled learning task for the system and provide greater flexibility in the learning process (para. 0115, etc.)].
As per claim 2, Guo/Baker teaches wherein at least some of the tasks produced by the group of example-generating agents include questions derived from the content of the source item [the prompts can include questions (Guo: pg. 2, fig. 1; etc.); which are generated by the reference system(s) for the student model(s) (Baker: paras. 0114-115, etc.) from the data source(s) (Baker: paras. 0025, 0117, etc.), including recognizing speech (Baker: para. 0085, etc.); in which case this includes generating question prompts from speech content].
As per claim 4, Guo/Baker teaches wherein at least one task is produced using a first example-generating agent that produces an intermediate result, and a second example-generating agent that performs further processing on the intermediate result [each reference system can include multiple models, such as data generators, classifiers, weighted selection, noise generation, etc., (Baker: figs. 12A-D; etc.); where the noise/data generator (agents) produce an intermediate result that is sent to the classifiers (agents) which further process the outputs to produce classification outputs (additional intermediate results) which are sent to the weighted random selection/LES control (agent) to process those results and produce the labeled output tasks/training data].
As per claim 5, Guo/Baker teaches wherein the group of example-generating agents performs operations that include one or more of:
generating questions based on the source item; and/or
rewriting the source item in a specified style; and/or
generating code based on the source item; and/or
summarizing the source item; and/or
specifying an order of statements in the source item; and/or
adding detail to the source item [the prompts can include questions (Guo: pg. 2, fig. 1; etc.); which are generated by the reference system(s) for the student model(s) (Baker: paras. 0114-115, etc.) from the data source(s) (Baker: paras. 0025, 0117, etc.), including recognizing speech (Baker: para. 0085, etc.); in which case this includes at least generating question prompts from speech content, specifies the order of statements in the speech, and can also include generating summaries of text (Guo: Appendix E; etc.)].
As per claim 9, Guo/Baker teaches wherein the transforming of the tasks by the student model includes, for a particular task:
producing one or more candidate student-generated responses to the particular task [the LLM being aligned (student) generates multiple candidate responses to a particular prompt (task) (Guo: pg. 2, fig. 1; etc.)]; and
assessing suitability of each of the one or more candidate student-generated responses using the teacher model, to provide preference information [the LLM produces preference ranking annotation/labels of the student responses (Guo: pg. 2, fig. 1 and section 2; etc.); where the preference ranking is an assessment of suitability of the candidate student-generated responses using the teacher model that provides preference information],
the updating of the parameters of the student model being based on the one or more student-generated responses, a teacher-generated response to the particular task, and the preference information, the parameters of the student model being associated with a policy applied by the student model [the model parameters and policies of the (student) LLM being aligned are updated from responses of the student model, annotated with the responses of the teacher model (Guo: pg. 2, fig. 1; pgs. 2-3, section 2, “Direct alignment from preference (DAP) methods”; pg. 4, Algorithm 1; etc.) where the parameters of the student model are associated with the policy applied by the student model (Guo: pg. 4, section 3 and Algorithm 1; etc.)],
the producing, assessing, and updating being repeated one or more times to successively improve performance of the student model [producing responses from the student model, assessing the responses by the annotator/teacher, and updating the student model parameters is repeated iteratively (Guo: pg. 4, Algorithm 1; etc.) to improve the performance of the model (Guo: pg. 4, Algorithm 1; pg. 5, section 4.2; etc.)],
in each repetition of the updating, the parameters of a next version of the student model being based, in part, on parameters of a current version of the student model [producing responses from the student model, assessing the responses by the annotator/teacher, and updating the student model parameters is repeated iteratively (Guo: pg. 4, Algorithm 1; etc.), which updates the parameters of the current version of the model to produce the next version, repeatedly].
As per claim 11, Guo/Baker teaches wherein the updating uses a loss function that bypasses an operation of explicitly generating a reward function and an operation of performing reward maximization [the training method (updating) can include applying an error cost (loss) function (Baker: paras. 0030, 0101, 0121-124; etc.), which does not rely on maximization of a reward function].
As per claim 12, Guo/Baker teaches generating a subset for the particular task that includes a first candidate student-generated response that is determined to be a suitable response to the particular task and a second candidate student-generated response that is determined to be a less suitable response to the task compared to the first candidate student-generated response [the LLM being aligned (student) generates multiple (two) candidate responses to a particular prompt (task) and the LLM produces preference ranking annotation/labels of the (two) student responses (Guo: pg. 2, fig. 1 and section 2; etc.) and the learning models can include producing a short list (subset) of answers based upon the answer scores (Baker: para. 0062, etc.); where the preference ranking is an assessment of suitability of the candidate student-generated responses using the teacher model that provides preference information and where more/less suitable are still both suitable responses to the particular prompt/task];
the updating attending to relations among the particular task, the first candidate student-generated task, and the second candidate-generated task as specified in the subset [the model parameters and policies of the (student) LLM being aligned are updated from responses of the student model, annotated with the responses of the teacher model (Guo: pg. 2, fig. 1; pgs. 2-3, section 2, “Direct alignment from preference (DAP) methods”; pg. 4, Algorithm 1; etc.), where the parameters of the student model being updated are relations among the task and each candidate-response].
As per claim 13, Guo/Baker teaches a machine-trained model having parameters produced by the method of claim 1 [the student can be an LLM (Guo: pg. 2, fig. 1; etc.) and the student model(s) can include machine learning models such as DNN having large numbers of parameters (Baker: paras. 0004, 0029, etc.) and multiple student models can include a mix of machine learning model types (Baker: paras. 0050-51, etc.)].
As per claim 14, Guo teaches [performing] operations including:
obtaining a source item that contains content [data prompts are provided to the models from a prompt distribution (pg. 2, fig. 1; pgs. 4-5, section 4.1, first paragraph; etc.) which are source items containing content (text, etc.)];
providing a task that is based on the content provided by a source item [prompts (tasks) may be sampled from a prompt distribution (etc.), where task-based prompt datasets (plural tasks) may be gathered and provided to the models (pg. 2, section 2, “Pairwise preference collection”; pgs. 4-5, section 4.1, first paragraph; etc.)];
transforming the task into a teacher-generated response using a teacher model [the LLM annotator (teacher model) generates labels/annotations (teacher-generated responses) for the responses from the (student) LLM being aligned, based upon the prompt 𝑥 (the plural tasks) (pg. 2, fig. 1; etc.); which is transforming the plural tasks (prompts) into plural teacher-generated responses (annotations/labels) using the teacher model] having a first size [the LLM annotator (teacher) can be of a larger size than the student LLM (pg. 7, section 4.5; pgs. 8-9, section 5, “Self-annotating models”; etc.)];
producing one or more candidate student-generated responses to the task using a student model [the LLM being aligned (student model) receives the prompts (plural tasks) and produces multiple responses (pg. 2, fig. 1; etc.); which is transforming the plural tasks (prompts) into student-generated responses using the student model] that has a second size that is less than the first size [the LLM annotator (teacher) can be of a larger size than the student LLM (pg. 7, section 4.5; pgs. 8-9, section 5, “Self-annotating models”; etc.)];
assessing suitability of each of the one or more candidate student-generated responses using the teacher model, to provide preference information [the LLM being aligned (student) generates multiple (two) candidate responses to a particular prompt (task) and the LLM produces preference ranking annotation/labels of the (two) student responses (pg. 2, fig. 1 and section 2; etc.); where the preference ranking is an assessment of suitability of the candidate student-generated responses using the teacher model that provides preference information and where more/less suitable are still both suitable responses to the particular prompt/task]; and
updating parameters of the student model based on the one or more student-generated responses, the teacher-generated response, and the preference information, the parameters of the student model being associated with a policy applied by the student model [the model parameters and policies of the (student) LLM being aligned are updated from responses of the student model, annotated with the responses of the teacher model (Guo: pg. 2, fig. 1; pgs. 2-3, section 2, “Direct alignment from preference (DAP) methods”; pg. 4, Algorithm 1; etc.) where the parameters of the student model are associated with the policy applied by the student model (Guo: pg. 4, section 3 and Algorithm 1; etc.)],
the producing, assessing, and updating being repeated one or more times to successively improve performance of the student model [producing responses from the student model, assessing the responses by the annotator/teacher, and updating the student model parameters is repeated iteratively (pg. 4, Algorithm 1; etc.) to improve the performance of the model (pg. 4, Algorithm 1; pg. 5, section 4.2; etc.)],
in each repetition of the updating, the parameters of a next version of the student model being based, in part, on parameters of a current version of the student model [producing responses from the student model, assessing the responses by the annotator/teacher, and updating the student model parameters is repeated iteratively (Guo: pg. 4, Algorithm 1; etc.), which updates the parameters of the current version of the model to produce the next version, repeatedly].
While Guo teaches a system for updating a student model utilizing a teacher model (see above) as well as providing instructions to control the models (see, e.g., Guo: abstract), it has not been relied upon for teaching a computing system for training a student model, comprising:
an instruction data store for storing computer-readable instructions; and
a processing system for executing the computer-readable instructions in the data store, to perform [the] operations; [and] the updating using a loss function that bypasses an operation of explicitly generating a reward function and an operation of reward maximization.
Baker teaches a computing system for training a student model, comprising:
an instruction data store for storing computer-readable instructions; and
a processing system for executing the computer-readable instructions in the data store, to perform [the] operations [the method may be implemented as program instructions stored in one or more memories and executed by one or more connected processors (para. 0046, etc.)]; [and]
the updating using a loss function that bypasses an operation of explicitly generating a reward function and an operation of reward maximization [the training method (updating) can include applying an error cost (loss) function (paras. 0030, 0101, 0121-124; etc.), which does not rely on maximization of a reward function].
Guo and Baker are analogous art, as they are within the same field of endeavor, namely improving/optimizing student models using teacher models.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the student and teacher model system as program instructions executed by a computer system, and to utilize a loss function that does not rely upon reward maximization, as taught by Baker, for implementing and updating the student model using a teacher model in the method taught by Guo.
Because both Guo and Baker provide methods of improving/optimizing student models using teacher models it would have been obvious to one of ordinary skill in the art to implement the student and teacher model system as program instructions executed by a computer system, as taught by Baker, for implementing and updating the student model using a teacher model in the method taught by Guo, to achieve the predictable result of providing the storage and compute necessary to carry out the operations of implementing and updating the model. Additionally, Baker provides motivation for utilizing the loss function as [by utilizing a separate loss function, the student model can minimize some error rate or cost while the coach optimizing some other set of hyperparameters and/or optimizing the structure of the student model (para. 0007, etc.) and can select a specific cost/error function based upon observations (para. 0008, etc.) or decide to combine minimizing the cost/error with the structural changes (para. 0044, etc.)]. It has also been held that omission of an element and its function (in this case, the reward maximization) in a combination where the remaining elements perform the same function as before involves only routine skill in the art. In re Karlson, 136 USPQ 184.
As per claim 16, Guo/Baker teaches wherein the assessing the suitability is performed by prompting the teacher model to assign one or more points to a particular candidate student-generated response based on an extent to which the particular candidate student-generated response has one or more specified characteristics [providing the preference labels can include the LLM annotator calculating scores (one or more points) for the candidate responses of the student model (Guo: pg. 4, section 3, “Annotating prompts with text-controllability”); where the scores are generated based upon the characteristics of the candidate responses (see also Guo: Appendix E)].
As per claim 17, see the rejection of claim 12 above.
As per claim 19, Guo/Baker teaches wherein the providing the task includes using one or more example-generating agents of a group of example-generating responses to produce the task based on the source item [the system can include multiple reference systems (Baker: fig. 4, etc.) and/or each reference system can include multiple models (Baker: figs. 12A-D, etc.), where reference system(s) generate tasks/training data for the student system (Baker: paras. 0114-115, etc.) from the data source(s) (Bake: paras. 0025, 0117, etc.); and where the reference systems generating tasks from the data content are the group of example-generating agents].
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo and Baker as applied to claim 1 above, and further in view of Wu et al. (AutoGen: Enabling Next-Gen LLM Applications vis Multi-Agent Conversation, Oct 2023, pgs. 1-43 – cited in an IDS).
As per claim 3, Guo/Baker teaches the method of claim 1, as described above.
While Guo/Baker teaches utilizing a group of example-generating agents (see above), it has not been relied upon for teaching wherein the group of example-generating agents are organized in a graph, the graph specifying input-output connections among the group of example-generating agents.
Wu teaches wherein the group of example-generating agents are organized in a graph, the graph specifying input-output connections among the group of example-generating agents [AutoGen is a framework that provides LLM applications via multiple agents that can converse with each other to accomplish tasks (pg. 1, abstract and fig. 1; etc.) including generating prompts, solutions, and responses, etc. (pg. 2, section 1; pg. 4, section 2.2 and fig. 2; etc.); where the agents are organized in a graph that provides input-output connections between the agents that can be flexibly defined (see, e.g., pg. 1, fig. 1; etc.)].
Guo/Baker and Wu are analogous art, as they are within the same field of endeavor, namely utilizing multiple LLMs or combinations of models to generate data.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize a group of agents organized in a graph for generating data/prompts, as taught by Wu, for the reference systems/example-generating agents used to generate plural tasks/prompts from the source content in the system/method taught by Guo/Baker.
Wu provides motivation as [by combining agents in a customizable, conversable framework that can operate in various modes and employ different combinations of agents, the system provides flexibility for different applications/users (abstract, etc.)].
Claim(s) 6-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo and Baker as applied to claim 1 above, and further in view of Madaan et al. (Self-Refine: Iterative Refinement with Self-Feedback, May 2023, pgs. 1-61 – cited in an IDS).
As per claim 6, Guo/Baker teaches wherein the student model produces one or more student responses for the particular task independently of information that describes how the teacher model refined the original response [the student model is independent of the teacher model (Guo: pg. 2, fig. 1; etc.) where the teacher model specification describing the process by which the teacher model produces responses is separate/independent from the student, including the response refinement below].
While Guo/Baker teaches utilizing a teacher model that can be updated (see above), it has not been relied upon for teaching wherein, for a particular tasks, the teacher model produces an original response based on the particular task, and wherein the teacher model refines the original response to produce a refined response having enhanced accuracy compared to the original response.
Madaan teaches wherein, for a particular task, the teacher model produces an original response based on the particular task, and wherein the teacher model refines the original response to produce a refined response having enhanced accuracy compared to the original response [using Self-Refine, an LLM produces an initial output, then the same LLM provides feedback for its output and uses it to refine itself, iteratively, creating an initial draft and subsequently refining it (pg. 1, abstract and section 1; etc.) providing improved accuracy (pg. 42, fig. 15; etc.)].
Guo/Baker and Madaan are analogous art, as they are within the same field of endeavor, namely utilizing and optimizing LLMs.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the Self-Refine process on an LLM so that it provides an initial response, creates feedback from the response, then produces a refined/improved response, as taught by Madaan, for the teacher model LLM providing responses based on particular tasks in the system/method taught by Guo/Baker.
Madaan provides motivation as [Self-Refine provides improved outputs/accuracy of the LLM without supervised training, additional data, or reinforcement learning (abstract, etc.)].
As per claim 7, Guo/Baker teaches wherein the student model produces one or more student responses for the particular task independently of information that describes at least some aspects of the first instruction and/or the second instruction [the student model is independent of the teacher model (Guo: pg. 2, fig. 1; etc.) where the teacher model specification describing the process by which the teacher model produces responses is separate/independent from the student, including the instructions below].
While Guo/Baker teaches utilizing a teacher model that can be updated and utilize instructions (see above), it has not been relied upon for teaching wherein, for a particular task, the teacher model produces an original response based on a first prompt that provides a first instruction, and wherein the teacher model produces a refined response based on a second prompt that provides a second instruction that is different than the first instruction.
Madaan teaches wherein, for a particular task, the teacher model produces an original response based on a first prompt that provides a first instruction, and wherein the teacher model produces a refined response based on a second prompt that provides a second instruction that is different than the first instruction [using Self-Refine, an LLM produces an initial output, then the same LLM provides feedback for its output and uses it to refine itself, iteratively, creating an initial draft and subsequently refining it (pg. 1, abstract and section 1; etc.) which prompts can include instructions to the LLM (pg. 3, second paragraph; pg. 8, “Does Self-Refine works in an instruction only setup?”; etc.); where the initial prompt is the first instruction, and the feedback is the second instruction different than the first].
Guo/Baker and Madaan are analogous art, as they are within the same field of endeavor, namely utilizing and optimizing LLMs.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the Self-Refine process on an LLM so that it provides an initial response, creates feedback from the response, then produces a refined/improved response, as taught by Madaan, for the teacher model LLM providing responses based on particular tasks in the system/method taught by Guo/Baker.
Madaan provides motivation as [Self-Refine provides improved outputs/accuracy of the LLM without supervised training, additional data, or reinforcement learning (abstract, etc.)].
As per claim 8, Guo/Baker/Madaan teaches choosing the first instruction based on an assessment of a class associated with the particular task [using Self-Refine, an LLM produces an initial output, then the same LLM provides feedback for its output and uses it to refine itself, iteratively, creating an initial draft and subsequently refining it (Madaan: pg. 1, abstract and section 1; etc.) which prompts can include instructions to the LLM (Madaan: pg. 3, second paragraph; pg. 8, “Does Self-Refine works in an instruction only setup?”; etc.); where the initial input is selected based upon a class associated with the input (Baker: figs. 12B-C; etc.)].
Claim(s) 10 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo and Baker as applied to claims 9 and 14 above, and further in view of Munos et al. (Nash Learning from Human Feedback, Dec 2023, pgs. 1-28 – cited in an IDS).
As per claim 10, Guo/Baker teaches the method of claim 9, above.
While Guo/Baker teaches performing optimizations based on the preference information (see above), it has not been relied upon for teaching wherein the method performs optimizations based on the preference information to find a stable policy for application by the student model, among competing policies, that satisfies a Nash equilibrium, the Nash equilibrium being a state in which there is no incentive to move from the stable policy to a competing policy.
Munos teaches wherein the method performs optimizations based on the preference information to find a stable policy for application by the student model, among competing policies, that satisfies a Nash equilibrium, the Nash equilibrium being a state in which there is no incentive to move from the stable policy to a competing policy [fine-tuning of an LLM using feedback can include finding a policy that consistently generates responses preferred over those generated by competing policies, defining the Nash equilibrium of the preference model (pg. 1, abstract, etc.); which means there is no incentive to move from this stable policy to a competing policy].
Guo/Baker and Munos are analogous art, as they are within the same field of endeavor, namely utilizing and optimizing LLMs based on preference information.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to fine-tune an LLM using feedback to achieve a Nash equilibrium policy from preference information, as taught by Munos, in fine-tuning the student LLM policies, using feedback from the teacher LLM, in the system/method taught by Guo/Baker.
Munos provides motivation as [fine-tuning the model to a Nash equilibrium policy using preference information and feedback improves the accuracy of the model outputs while aligning the model to human preferences (pg. 1, abstract, etc.)].
As per claim 15, see the rejection of claim 10 above.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo and Baker as applied to claim 17 above, and further in view of Yang et al. (RLCD: Reinforcement Learning from Contrastive Distillation for LM Alignment, March 2024, pgs. 1-40 – cited in an IDS).
As per claim 18, Guo/Baker teaches wherein the operations further comprise choosing the first candidate student-generated response in response to determining that the first candidate student-generated response has a first score that satisfies a prescribed test of suitability [providing the preference labels can include the LLM annotator calculating scores for the candidate responses of the student model and chooses the preferred response based on the scores (Guo: pg. 4, section 3, “Annotating prompts with text-controllability”; Appendix E; etc.), where having the higher score is satisfying a prescribed test of suitability].
While Guo/Baker teaches preference scoring the responses by the teacher model and choosing a response (see above), it has not been relied upon for teaching choosing the second candidate student-generated response upon determining that the second student-generated response has a second score that is worse than the first score by at least a prescribed amount.
Yang teaches choosing the second candidate student-generated response upon determining that the second student-generated response has a second score that is worse than the first score by at least a prescribed amount [the preference model assigns scores to the candidate scores and is trained to optimize the difference between the two scores to match the preference data (pg. 3, section 3.1.1; pgs. 8-9, sections 5.2-6; etc.) for positive and negative prompts (abstract, etc.); where determining a prescribed amount for the difference in scores is the optimizing the difference between the scores (see also below)].
Guo/Baker and Yang are analogous art, as they are within the same field of endeavor, namely optimizing a student model with a teacher (distillation) including preference scoring/alignment.
It would have been obvious to eon of ordinary skill in the art, before the effective filing date of the claimed invention, to optimize the preference scores of the preference model utilizing both negative and positive prompts, as taught by Yang, to optimize the preference model scoring in the system taught by Guo/Baker.
Yang provides motivation as [using two different prompts causes model outputs to be more differentiated on average, resulting in cleaner preference labels in the absence of human annotations, which then improves the student model (abstract, etc.)]. Additionally, it has been held that discovering an optimum value of a result effective variable (in this case, the score(s) desired for choosing outputs/preference) involves only routine skill in the art. In re Boesch, 617 F.2d 272, 205 USPQ 215 (CCPA 1980).
Conclusion
The following is a summary of the treatment and status of all claims in the application as recommended by M.P.E.P. 707.07(i): claims 1-20 are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Das (US 12,638,543) – discloses a connected multi-agent network for predictions.
Yasunaga et al. (Graph-based, Self-Supervised Program Repair from Diagnostic Feedback, June 2020, pgs. 1-10) – discloses a graph based self-supervised program repair using a program-feedback graph.
The examiner requests, in response to this Office action, that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line number(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections. See 37 CFR 1.111(c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEORGE GIROUX whose telephone number is (571)272-9769. The examiner can normally be reached M-F 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEORGE GIROUX/Primary Examiner, Art Unit 2128
Prosecution Insights