Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
The following action is in response to the communication(s) received on 04/16/2024.
As of the claims filed 04/16/2024:
Claims 1-20 are pending.
Claims 1, 8, and 15 are independent claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a method, thus a process, one of the four statutory categories of patentable subject matter (Step 1). However, Claim 1 further recites:
selecting... at least some of the plurality of machine experts for answering the question, which is an evaluation or judgement that can be performed in the human mind;
generating... candidate answers to the question, wherein each of the candidate answers is created based on a respective reference from a source, which is an evaluation or judgement that can be performed in the human mind;
performing quality assessment on each of the candidate answers from the at least some machine experts, which is an evaluation or judgement that can be performed in the human mind;
determining, based on a result of quality assessment on the candidate answers, one of the candidate answers as an answer to the question, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1.Under Step 2A Prong 2, the claim recites:
receiving, from a user, a question related to a subject matter, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application;
based on past performances of a plurality of machine experts, which merely specifies the particular field of use or particular technological environment in which the abstract idea is to be performed, which by MPEP 2106.05(h) cannot integrate the abstract idea into a practical application;
by the selected at least some machine experts, as the performance of an abstract idea on a computer is not more than instructions to 'apply it' on a computer, which by MPEP 2106.05(f) cannot integrate an abstract idea into a practical application;
providing the answer to the user in response to the question, which is merely an insignificant extra-solution activity of data output, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data gathering (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)); the activity of data output (MPEP 2106.05(g)) cannot provide significantly more, as receiving or transmitting data over a network is well understood, routine, and conventional (MPEP 2106.05(d)(II)(i), buySAFE, Inc. v. Google, Inc); the particular field of use or particular technological environment (MPEP 2106.05(h)) cannot provide significantly more; implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more. The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 2, dependent on 1, further recites
and identifying the at least some machine experts based on the information characterizing their respective past performances, wherein the information includes a fidelity attribute representing a cumulative level of satisfaction on answers previously gene, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1.Under Step 2A Prong 2, the claim recites:
accessing information characterizing past performance of each of the plurality of machine experts, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data gathering (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)). The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 3, dependent on 2, further recites
the answers previously generated are for previous questions on the subject matter, which is merely a detail of an abstract idea (identifying the at least some machine experts).
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2, the claim recites: no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 4, dependent on 3, further recite. no additional abstract ideas. However: Under Step 2A Prong 2, the claim recites:
the cumulative level of satisfaction is determined based on feedback provided by a plurality of human evaluators on the previously generated answers, which merely specifies the particular field of use or particular technological environment in which the abstract idea is to be performed, which by MPEP 2106.05(h) cannot integrate the abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the particular field of use or particular technological environment (MPEP 2106.05(h)) cannot provide significantly more. The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 5, dependent on 1, further recites
determining a feature vector of the question, which is an evaluation or judgement that can be performed in the human mind;
comparing the feature vector of the question with feature vectors of different references from at least one source to identify the respective reference with a reference feature vector matching the feature vector of the question according to a predetermined criterion, which is an evaluation or judgement that can be performed in the human mind;
and creating the candidate answer based on the respective reference..., which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2, the claim recites:
via a language model previously trained via machine learning, as the performance of an abstract idea on a computer is not more than instructions to 'apply it' on a computer, which by MPEP 2106.05(f) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more. The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 6, dependent on 1, further recites
processing information related to the candidate answer, including the question and a reference relied upon to generate the candidate answer, which is an evaluation or judgement that can be performed in the human mind;
determining relevance between the candidate answer and the question, which is an evaluation or judgement that can be performed in the human mind;
evaluating accuracy of the candidate answer with respect to the question, which is an evaluation or judgement that can be performed in the human mind;
computing a metric indicative of similarity between the candidate answer and the reference, which is an evaluation or judgement that can be performed in the human mind;
determining fidelity of the candidate answer based on the metric, which is an evaluation or judgement that can be performed in the human mind;
and obtaining a quality assessment result of the candidate answer based on the relevance, accuracy, and fidelity of the candidate answer, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2, the claim recites: no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 7, dependent on 1, further recites
incorporating the feedback in a training data set for adapting the plurality of machine experts, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2, the claim recites:
receiving feedback for the answer, obtained based on evaluation directed to the answer, from one or more human evaluators, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application;
wherein evaluation from each of the one or more human evaluators include a ranking of the answer, a cumulative fidelity score of the human evaluator, and optionally an alternative answer in place of the answer with an alternative reference used to support the alternative answer, which merely specifies the particular field of use or particular technological environment in which the abstract idea is to be performed, which by MPEP 2106.05(h) cannot integrate the abstract idea into a practical application;
and adapting the plurality of machine experts via machine learning based on the training data set, as the performance of an abstract idea on a computer is not more than instructions to 'apply it' on a computer, which by MPEP 2106.05(f) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data gathering (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)); the particular field of use or particular technological environment (MPEP 2106.05(h)) cannot provide significantly more; implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more. The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claims 8-14 recite A machine readable and non-transitory medium, thus an article of manufacture, one of the four statutory categories of patentable subject matter. However, Claims 8-14 recite having information recorded thereon, wherein the information, when read by the machine, causes the machine to perform precisely the abstract ideas and additional elements of Claims 1-7, respectively. Therefore, Step 2A Prong 1 analysis remains the same. As for Step 2A Prong 2 and Step 2B: performance on a computer cannot integrate an abstract idea into a practical application (Step 2A Prong 2) nor provide significantly more than the abstract idea itself (Step 2B) (MPEP 2106.05(f)), Claims 8-14 are rejected as subject-matter ineligible for reasons set forth in the rejections of Claims 1-7, respectively.
Claims 15 and 17-20 recite A machine readable and non-transitory medium, thus an article of manufacture, one of the four statutory categories of patentable subject matter. However, Claims 15 and 17-20 recite having information recorded thereon, wherein the information, when read by the machine, causes the machine to perform precisely the abstract ideas and additional elements of Claims 1 and 4-7, respectively. Therefore, Step 2A Prong 1 analysis remains the same. As for Step 2A Prong 2 and Step 2B: performance on a computer cannot integrate an abstract idea into a practical application (Step 2A Prong 2) nor provide significantly more than the abstract idea itself (Step 2B) (MPEP 2106.05(f)), Claims 15 and 17-20 are rejected as subject-matter ineligible for reasons set forth in the rejections of Claims 1 and 4-7, respectively.
Claim 16, dependent on Claim 15, also recites precisely the limitations of Claims 2-3 combined. Thus, Claims 16 are rejected for reasons set forth in Claims 2-3 combined.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6, 8-13, and 15-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Pitis et al., "Boosted Prompt Ensembles for Large Language Models" (hereinafter Pitis).
Regarding Claim 1, Pitis teaches:
A method, comprising: receiving, from a user, a question related to a subject matter; (Pitis [alg.1, “when prompted with… question q”]
PNG
media_image1.png
308
902
media_image1.png
Greyscale
)
selecting, based on past performances of a plurality of machine experts, at least some of the plurality of machine experts for answering the question; (Pitis [p.3 alg.1] Our algorithm progressively grows preds, the set of answers to each question… preds[q].append([LLM(p,q)for in range(m)]))
generating, by the selected at least some machine experts, candidate answers to the question, wherein each of the candidate answers is created based on a respective reference from a source; (Pitis [p.3 alg.1, answers A]; [p.3 right ¶4] we first sample a set of m candidate reasoning paths and answers for each problem in DTrain using the most recent prompt… [p.4 right last ¶] Datasets We consider the following datasets: • AQUA (Algebra QA with Rationales), a dataset of roughly 100,000 algebraic word problems and 254 test questions, which is sometimes referred to as the MATHQA dataset... We randomly sample 200 training problems for our labeled training set.) (Note: AQUA corresponds to a source; each candidate reasoning path uses the training set from AQUA to generate a candidate answer, thus corresponding to being created based on a respective reference from a source)
performing quality assessment on each of the candidate answers from the at least some machine experts; (Pitis [p.2 left ¶2] …self-consistency (SC), which replaces the standard greedy decoding of the LLM output with a stochastic output space ensemble that marginalizes over multiple reasoning paths by sampling with positive temperature (e.g., T =0.7) and choosing the final prediction p∗ with highest agreement:
PNG
media_image2.png
44
233
media_image2.png
Greyscale
This exploits the fact that diverse reasoning paths that lead to the same answer are more likely to be correct. Our work builds on self-consistency by using the agreement among reasoning paths to determine the set of “Hard” problems and, for the test-time version of our algorithm, the set of LLM generated answers that are likely to be correct.) (Note: choosing the prediction with highest level of agreement for the determined “Hard” problems corresponds to performing quality assessment on each of the candidate answers)
determining, based on a result of quality assessment on the candidate answers, one of the candidate answers as an answer to the question; providing the answer to the user in response to the question. (Pitis [p.2 left ¶2] …self-consistency (SC), which replaces the standard greedy decoding of the LLM output with a stochastic output space ensemble that marginalizes over multiple reasoning paths by sampling with positive temperature (e.g., T =0.7) and choosing the final prediction p∗ with highest agreement:
PNG
media_image2.png
44
233
media_image2.png
Greyscale
This exploits the fact that diverse reasoning paths that lead to the same answer are more likely to be correct. [p.3 alg.1] return prompts, preds)) (Note: choosing the prediction with highest level of agreement corresponds to determining one candidate answer as the answer to the question)
Regarding Claim 2, Pitis respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Pitis further teaches:
The method of claim 1, wherein the selecting comprises: accessing information characterizing past performance of each of the plurality of machine experts; (Pitis [p.3 alg.1] Our algorithm progressively grows preds, the set of answers to each question… NEWPROMPT(T, preds, [answers A])) (Note: the accumulated preds corresponds to the list of past performances of each of the plurality of machine experts)
and identifying the at least some machine experts based on the information characterizing their respective past performances, wherein the information includes a fidelity attribute representing a cumulative level of satisfaction on answers previously generated by the machine expert. (Pitis [p.3 right last ¶] Then, motivated by curriculum learning, we form a new prompt by selecting correct reasoning paths from those problems of intermediate difficulty, where the current boosted ensemble only sometimes gets the correct answer. Specifically, we sort the problems where at least one reasoning path led to the correct answer by the number of correct reasoning paths, and select (problem, correct reasoning path) pairs from amongst the hardest problems. Following Fu et al. (2022)’s discovery that longer reasoning paths improve in-context reasoning performance, for each hard problem chosen, we choose from the reasoning paths that led to a correct answer by using a complexity heuristic, measured by the number of sentences in reasoning path (Fu et al., 2022). Concatenating this set of (problem, correct reasoning path) pairs forms a new prompt, which we use for the next iteration of the algorithm, until we have a set of n prompts comprising a boosted ensemble) (Note: the measured number of sentences in reasoning path corresponds to the fidelity attribute)
Regarding Claim 3, Pitis respectively teaches and incorporates the claimed limitations and rejections of Claim 2. Pitis further teaches:
The method of claim 2, wherein the answers previously generated are for previous questions on the subject matter. (Pitis [p.3 alg.1] Our algorithm progressively grows preds, the set of answers to each question… NEWPROMPT(T, preds, [answers A])) (Note: the algorithm grows pred using the initial prompt p, while NEWPROMPT uses the accumulated pred, thus corresponding to the answers previously generated for previous questions)
Regarding Claim 4, Pitis respectively teaches and incorporates the claimed limitations and rejections of Claim 3. Pitis further teaches:
The method of claim 3, wherein the cumulative level of satisfaction is determined based on feedback provided by a plurality of human evaluators on the previously generated answers. (Pitis [p.3 alg.1, C ←[qforqinTif there is “sufficient agreement” for the majority prediction in preds[q]]; [p.4 left ¶3] The definition of sufficient agreement is a hyperparameter. In our experiments, we consider sufficient agreement to be achieved for question q with most common prediction p∗ if Σp∈pred[q]I(p = p∗)/nm is higher than some sufficient agreement hyperparameter ∆) (Note: using a hyperparameter for determining sufficient agreement corresponds to the cumulative level of satisfaction being based on feedback provided by human evaluators)
Regarding Claim 5, Pitis respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Pitis further teaches:
The method of claim 1, wherein the generating a candidate answer comprises: determining a feature vector of the question; comparing the feature vector of the question with feature vectors of different references from at least one source to identify the respective reference with a reference feature vector matching the feature vector of the question according to a predetermined criterion; and creating the candidate answer based on the respective reference via a language model previously trained via machine learning. (Pitis [p.2 fig.1]
PNG
media_image3.png
373
927
media_image3.png
Greyscale
) (Note: each point on the diagram corresponds to a feature vector of the question; determining correct/incorrect answers corresponds to identifying the respective reference feature vector according to the predetermined criteria; generating its own labels corresponds to creating the candidate answer)
Regarding Claim 6, Pitis respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Pitis further teaches:
The method of claim 1, wherein the obtaining a quality assessment of each of the candidate answers comprises: processing information related to the candidate answer, including the question and a reference relied upon to generate the candidate answer; (Pitis [p.3 alg.1] Our algorithm progressively grows preds, the set of answers to each question… NEWPROMPT(T, preds, [answers A]);
[p.3 alg.1, answers A]; [p.3 right ¶4] we first sample a set of m candidate reasoning paths and answers for each problem in DTrain using the most recent prompt… [p.4 right last ¶] Datasets We consider the following datasets: • AQUA (Algebra QA with Rationales), a dataset of roughly 100,000 algebraic word problems and 254 test questions, which is sometimes referred to as the MATHQA dataset... We randomly sample 200 training problems for our labeled training set.
[p.2 fig.1]
PNG
media_image3.png
373
927
media_image3.png
Greyscale
) (Note: AQUA corresponds to a source; each candidate reasoning path uses the training set from AQUA to generate a candidate answer, thus corresponding to being created based on a respective reference from a source; the identified correct/incorrect points require the source for generation, thus corresponding to a reference relied upon for generating the candidate answer)
determining relevance between the candidate answer and the question; (Pitis [p.2 fig.1]
PNG
media_image3.png
373
927
media_image3.png
Greyscale
)(Note: the distance from the new prompt and the candidate answers correspond to the relevance)
evaluating accuracy of the candidate answer with respect to the question; (Pitis [p.3 alg.1]
PNG
media_image4.png
67
438
media_image4.png
Greyscale
) (Note: identifying
computing a metric indicative of similarity between the candidate answer and the reference; (Pitis [p.2 left ¶2] …self-consistency (SC), which replaces the standard greedy decoding of the LLM output with a stochastic output space ensemble that marginalizes over multiple reasoning paths by sampling with positive temperature (e.g., T =0.7) and choosing the final prediction p∗ with highest agreement:
PNG
media_image2.png
44
233
media_image2.png
Greyscale
This exploits the fact that diverse reasoning paths that lead to the same answer are more likely to be correct. [p.3 alg.1] return prompts, preds)) (Note: choosing the prediction with highest level of agreement corresponds to a metric indicative of similarity between the candidate answer and the reference)
determining fidelity of the candidate answer based on the metric; and obtaining a quality assessment result of the candidate answer based on the relevance, accuracy, and fidelity of the candidate answer. (Pitis [p.2 left ¶2] …self-consistency (SC), which replaces the standard greedy decoding of the LLM output with a stochastic output space ensemble that marginalizes over multiple reasoning paths by sampling with positive temperature (e.g., T =0.7) and choosing the final prediction p∗ with highest agreement:
PNG
media_image2.png
44
233
media_image2.png
Greyscale
This exploits the fact that diverse reasoning paths that lead to the same answer are more likely to be correct. [p.3 alg.1] return prompts, preds)) (Note: higher p* corresponds to higher fidelity of the candidate answer)
Independent Claim 8 recites machine readable and non-transitory medium having information recorded thereon, wherein the information, when read by the machine, causes the machine to perform the following steps (Pitis [p.4 right ¶3] Model Our primary experiments are carried out with the code-davinci-002 (“Codex”) model via the OpenAI API) to perform precisely the methods of Claim 1. Thus, Claim 8 is rejected for reasons set forth in Claim 1.
Claims 9-13, dependent on Claim 8, also recite precisely the methods of Claims 2-6, respectively. Thus, Claims 9-13 are rejected for reasons set forth in Claims 2-6, respectively.
Independent Claim 15 recites A system, comprising: an artificial intelligence (AI) based answer generator implemented using a processor(Pitis [p.4 right ¶3] Model Our primary experiments are carried out with the code-davinci-002 (“Codex”) model via the OpenAI API) to perform precisely the methods of Claim 1. Thus, Claim 17 is rejected for reasons set forth in Claim 1.
Claim 16, dependent on Claim 15, also recites precisely the methods of Claims 2-3 combined. Thus, Claims 16 are rejected for reasons set forth in Claims 2-3 combined.
Claims 17-19, dependent on Claim 15, also recite precisely the methods of Claims 4-6, respectively. Thus, Claims 17-19 are rejected for reasons set forth in Claims 4-6, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 7, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Pitis, in view of Yuan et al., “RRHF:RankResponses to Align Language Models with Human Feedback” (hereinafter Yuan).
Regarding Claim 7, Pitis respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Pitis does not teach, but Yuan further teaches:
The method of claim 1, further comprising: receiving feedback for the answer, obtained based on evaluation directed to the answer, from one or more human evaluators; (Yuan [p.2 ¶2] To alleviate the complex hyperparameter tuning and sophisticated training resource requirements of PPO, we propose a novel training paradigm RRHF (Rank Responses to align Human Feedback) that aligns model probabilities of multiple responses with human preferences by ranking loss, which can retain the performance of PPO and is much simpler… RRHF then leverages responses from various sources for training, scoring responses based on the log probability provided by the training language model. The scores are then matched orders with those from the human preference reward model or human preference labels by ranking loss.) (Note: the human preference labels correspond to feedback from human evaluators)
incorporating the feedback in a training data set for adapting the plurality of machine experts, wherein evaluation from each of the one or more human evaluators include a ranking of the answer, a cumulative fidelity score of the human evaluator, and optionally an alternative answer in place of the answer with an alternative reference used to support the alternative answer; (Yuan [p.2 fig.1, RRHF, “human label/reward model”]
PNG
media_image5.png
335
743
media_image5.png
Greyscale
) (Note: the score generated from the human label corresponds to the cumulative fidelity score; expert response corresponds to an alternative answer from an alternative reference)
and adapting the plurality of machine experts via machine learning based on the training data set. (Yuan [p.4 ¶2]
PNG
media_image6.png
327
764
media_image6.png
Greyscale
) (Note: calculating the new total loss corresponds to adapting the plurality of machine experts via machine learning)
Yuan and Pitis are analogous to the present invention because both are from the same field of endeavor of training large language models. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the RRHF method from Yuan into Pitis’s method of boosted prompting. The motivation would be to “RRHF only needs 1 to 2 models during tuning and can efficiently align language models with human preferences robustly without complex hyperparameter tuning.” (Yuan, abstract).
Claim 14 dependent on Claim 8, also recites precisely the methods of Claim 7. Thus, Claims 14 is rejected for reasons set forth in Claims 7.
Claim 20 dependent on Claim 15, also recites precisely the methods of Claim 7. Thus, Claims 20 is rejected for reasons set forth in Claims 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kim et al., US 12608564 B2, which teaches “a model that combines a large language model (LLM) that predicts words and sentences with reinforcement learning from human feedback (RLHF) for accelerating or improving agent learning based on human feedback” (Kim, 2. Description of the Related Art; 3rd ¶); Chandler et al., US 20250315662 A1, which teaches generating optimal prompt combinations based on the corresponding human decisions (Chandler [0008]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEP HAN whose telephone number is (703)756-1346. The examiner can normally be reached Mon-Fri 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.H./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights