DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites series of steps of obtaining, processing, determining, modifying related to concepts performed in the human mind (including an observation, evaluation, judgment, opinion).
The claim recites obtaining, processing, determining, modifying…. The limitation of obtaining, processing, determining, modifying is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting “a computing system, computing device, processor or computer-readable media” nothing in the claim element precludes the step from practically being performed in the mind.
For example, but for the “by a computing system, computing device, processor or computer-readable media” language, “obtaining, processing, determining, modifying” in the context of this claim encompasses the user manually calculating or processing the amount of use of each vocabulary or data, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components.
For example, but for the “by computing system or computer device” language, “obtaining, processing, determining, modifying” in the context of this claim encompasses the user thinking that the most important vocabulary or data should be selected from other less important vocabulary or data. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements – using a computing system or computer device to perform both the obtaining, processing, determining, modifying steps. The “computing system or computer device” in both steps is recited at a high-level of generality (i.e., as a generic “computing system or computer device” performing a generic computer function of obtaining, processing, determining, modifying based on a determined amount of use) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using “a computing system or computer device” to perform the obtaining, processing, determining, modifying steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Dependent claims are also does not add any additional elements, as such the claims are not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-23 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Dahlem et al. US 2024/0169264.
In regarding to claim 1 Dahlem teaches:
1. A computer-implemented method for performing preference optimization, the method comprising: obtaining, by a computing system comprising one or more computing devices, a training tuple comprising a training sequence comprising a sequence of tokens;
Dahlem, 0068-0069, 0073
processing, by the computing system, at least a portion of the sequence of tokens in the training sequence with a sequence processing model to generate base probabilities respectively for one or more candidate tokens included in a token vocabulary;
Dahlem, 0073-0075, 0099-0100
processing, by the computing system, at least the portion of the sequence of tokens in the training sequence with a reward model to generate conditional reward probabilities respectively for the one or more candidate tokens;
Dahlem, 0073-0076, 0099-0100
determining, by the computing system, a posterior probability for at least an actual next token included in the training sequence based on the base probabilities and the conditional reward probabilities;
Dahlem, 0073-0076, 0099-0100
processing, by the computing system, at least the portion of the sequence of tokens in the training sequence with a posterior prediction model to generate a distilled posterior probability for at least the actual next token in the training sequence;
Dahlem, 0073-0076, 0099-0100
and modifying, by the computing system, one or more values of one or more parameters of the posterior prediction model based on a distillation loss function that generates a loss value based at least in part on the posterior probability for at least the actual next token and the distilled posterior probability for at least the actual next token.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 2 Dahlem teaches:
2. The computer-implemented method of claim 1, wherein the posterior prediction model comprises a posterior prediction layer appended to the sequence processing model.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 3 Dahlem teaches:
3. The computer-implemented method of claim 2, wherein the method further comprises modifying, by the computing system, one or more values of one or more parameters of the sequence processing model based on the distillation loss function.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 4 Dahlem teaches:
4. The computer-implemented method of claim 2, wherein: the sequence processing model comprises a base prediction layer that generates the base probabilities; the sequence processing model comprises a reward prediction layer that generates the conditional reward probabilities; and the method further comprises deploying the posterior prediction layer and the sequence processing model exclusive of the base prediction layer and the reward prediction layer.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 5 Dahlem teaches:
5. The computer-implemented method of claim 1, wherein the posterior prediction model comprises a separate model that is separate from the sequence processing model.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 6 Dahlem teaches:
6. The computer-implemented method of claim 1, wherein the distillation loss function comprises a cross entropy loss or a square loss.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 7 Dahlem teaches:
7. The computer-implemented method of claim 1, wherein the distillation loss function comprises a pairwise distillation loss function.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 8 Dahlem teaches:
8. The computer-implemented method of claim 1, further comprising modifying, by the computing system, one or more values of one or more parameters of the reward model based on a reward loss function that compares the conditional reward probabilities to a preference label included in the training tuple.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 9 Dahlem teaches:
9. The computer-implemented method of claim 8, wherein the reward loss function comprises a pointwise loss function or a pairwise loss function.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 10 Dahlem teaches:
10. The computer-implemented method of claim 8, wherein the modification of the reward model is performed prior to the modification of the posterior prediction model.
Dahlem, 0078-0080, 0084, 0099-0100
In regarding to claim 11 Dahlem teaches:
11. The computer-implemented method of claim 1, wherein determining, by the computing system, the posterior probability for at least the actual next token included in the training sequence based on the base probabilities and the conditional reward probabilities comprises multiplying the base probability for at least the actual next token in the training sequence and the conditional reward probability for at least the actual next token in the training sequence and then dividing by conditional reward probability for at least the actual next token in the training sequence from a prior time step.
Dahlem, 0078-0080, 0084, 0099-0100 and Fig. 5
In regarding to claim 12 Dahlem teaches:
12. The computer-implemented method of claim 1, wherein determining, by the computing system, the posterior probability for at least the actual next token included in the training sequence based on the base probabilities and the conditional reward probabilities comprises adjusting the conditional reward probabilities based on a learned uncertainty score.
Dahlem, 0078-0080, 0084, 0099-0100 and Fig. 5
In regarding to claim 14 Dahlem teaches:
14. The computing system of claim 13, wherein the plurality of different reward models are arranged in a series configuration in which each reward model in the series is conditioned upon the respective preference control variables for all preceding reward models in the series having a positive preference.
Dahlem, 0078-0080, 0084, 0099-0100 and Fig. 5
In regarding to claim 16 Dahlem teaches:
16. The computing system of claim 13, wherein the plurality of different reward models are arranged in a parallel configuration in which each reward model generates the conditional reward probabilities for the corresponding preference control variable independent of the other preference control variables.
Dahlem, 0078-0080, 0084, 0099-0100 and Fig. 5
In regarding to claim 17 Dahlem teaches:
17. The computing system of claim 16, wherein determining, by the computing system, the posterior probability for at least the actual next token included in the training sequence based on the base probabilities and the plurality of sets of conditional reward probabilities comprises: processing, by the computing system, the plurality of sets of conditional reward probabilities with a preference summary layer to generate a combined preference prediction; and multiplying, by the computing system, the combined preference prediction with the base probabilities and then dividing by a combined preference prediction from a prior time step to generate the posterior probability for at least the actual next token in the training sequence.
Dahlem, 0078-0080, 0084, 0099-0100 and Fig. 5
Claims 13, 15, 18-19 and 20 list all similar elements of claims 1, 11, 2-3 and 4, but in system form rather than method form. Therefore, the supporting rationale of the rejection to claims 1, 11, 2-3 and 4 applies equally as well to claims 13, 15, 18-19 and 20.
In regarding to claim 21 Dahlem teaches:
21. The computing system of claim 13, wherein the plurality of different reward models comprise a plurality of different reward prediction layers that are appended to the sequence processing model and share the sequence processing model as a base, and wherein the plurality of different reward prediction layers comprise a plurality of expert models that respectively correspond to a plurality of different tasks.
Dahlem, 0078-0080, 0084, 0099-0100 and Fig. 5
Claim 22 list all similar elements of claim 1, but in a non-transitory computer readable media form rather than method form. Therefore, the supporting rationale of the rejection to claim 1 applies equally as well to claim 22.
In regarding to claim 23 Dahlem teaches:
23. The one or more non-transitory computer-readable media of claim 22, wherein the non-transitory computer-readable media further store a lookahead prediction model or layer that predicts a variance of preference predictions that is used to reduce decoding complexity at inference time.
Dahlem, 0078-0080, 0084, 0099-0100 and Fig. 5
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Rakocz et al. US 2024/0169267.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL T TEKLE whose telephone number is (571)270-1117. The examiner can normally be reached Monday-Friday 8:00-4:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at 571-272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL T TEKLE/Primary Examiner, Art Unit 2481
Prosecution Insights