DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does fall within at least one of the four categories of patent eligible subject matter because the claim is directed to one or more computer storage media. Applicant fails to define “computer storage media” in the specification. Applicant does define “computer readable media” in the specification, [0232]. Here, the definition of “computer readable media” is not limiting (…include[s] all forms of non volatile memory, media and memory devices…) and may be construed as reading on transitory waves, which are non-statutory subject matter. Applicant is encouraged to amend to “non-transitory computer storage media” to overcome this rejection.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-10 and 17-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mohtashami et al., “Social Learning: Towards Collaborative Learning with Large Language Models”, arXiv.org, 8 February 2024, 19 pages.
In regard to claim 1, Mohtashami disclosed a method performed by one or more computers, the method comprising:
receiving an input query for performing a task using a student language model neural network; (section 2.1 – “A user queries the student, section 4.1 – “the student’s language model”
processing a student input comprising the input query using the student language model neural network to generate, as output, a teacher query for a teacher language model neural network, wherein the teacher query characterizes the task while not including sensitive information of the input query; (Figure 1 – instruction / examples request)
providing the teacher query as an input to the teacher language model neural network; (Figure 1)
obtaining, as output from the teacher language model neural network and in response to the teacher query, a respective example response for each of one or more example queries for performing the task; (section 2.2.1 – “ask teacher agents to generate instructions based on their silo of private data. These instructions are then shared with the student who integrates the instructions in its prompt”)
processing an augmented input query that comprises (i) the input query, (ii) the one or more example queries, and (iii) the respective example responses for the example queries using the student language model neural network to generate a response to the input query; (section 2.1 – “In this work, we only consider the most basic version of the student at inference which replies to a user input by appending the input to a prompt, querying its language model, and returning the continuation”)
providing, as output, the response to the input query. (section 2.1 – returning the continuation)
In regard to claim 2, Mohtashami disclosed the method of claim 1, wherein the student language model neural network is deployed on a user device and the teacher language model neural network is deployed on one or more remote computers that are remote from the user device. (Figure 1, wherein the teacher models are separate to the student model)
In regard to claim 3, Mohtashami disclosed the method of claim 2, wherein providing the teacher query as input to the teacher language model neural network comprises providing the teacher query from the user device to the one or more remote computers over a data communication network. (Figure 1)
In regard to claim 4, Mohtashami disclosed the method of claim 3, wherein obtaining, as output from the teacher language model neural network and in response to the teacher query, a respective example response for each of one or more example queries comprises:
receiving, by the user device and over the data communication network, data comprising the respective example responses. (Figure 1)
In regard to claim 5, Mohtashami disclosed the method of claim 2, wherein the query input is received from a user of the user device. (Figure 1, user sends query to student model)
In regard to claim 6, Mohtashami disclosed the method of claim 1, wherein the teacher query comprises a natural language description of the input query that specifies one or more properties of the task. (section 2.2.1)
In regard to claim 7, Mohtashami disclosed the method of claim 6, wherein the output from the teacher language model neural network comprises one or more example queries and the respective example responses and is generated in response to an input that comprises the teacher query and a natural language instruction to generate example queries and corresponding example responses that have the one or more properties specified by the natural language description. (section 2.2.1 and 2.2.2)
In regard to claim 8, Mohtashami disclosed the method of claim 6, wherein the student input comprises the input query and (i) a natural language instruction to generate a natural language description of the input query that specifies the one or more properties of the input query, (ii) one or more example input query – natural language description pairs, or (iii) both. (section 2.2.1 and 2.2.2)
In regard to claim 9, Mohtashami disclosed the method of claim 1, wherein the teacher query comprises the example queries. (section 2.2.1 and 2.2.2)
In regard to claim 10, Mohtashami disclosed the method of claim 9, wherein the output from the teacher language model neural network comprises the respective example responses and is generated in response to an input that comprises the teacher query and a natural language instruction to generate responses to the example queries. (section 2.2.1 and 2.2.2)
Claim 17 is rejected for substantially the same reasons as claim 1.
Claim 18 is rejected for substantially the same reasons as claim 2.
Claim 19 is rejected for substantially the same reasons as claim 3.
Claim 20 is rejected for substantially the same reasons as claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Mohtashami in view of Yue et al., “Large Language Model Cascades with Mixture of Thought Representations for Cost-efficient Reasoning”, arXiv.org, Cornell University Library, 4 October 2023, 38 pages.
In regard to claim 13, Mohtashami failed to disclose wherein the teacher language model neural network has more parameters than the student language model neural network.
However, Yue disclosed wherein the teacher language model neural network has more parameters than the student language model neural network. See Section 2.1, Figure 1, weaker LLM and stronger LLM, wherein the stronger LLM would have more parameters than the weaker LLM.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use a stronger LLM with more parameters as the teacher language model in Mohtashami when training the student LLM in Mohtashami since the teacher LLM with more parameters would train the student LLM with less parameters.
Allowable Subject Matter
Claims are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bethancourt et al. US 12,602,508
Turuvekere Sreenivas et al. US 2025/0061340
Chen et al. US 2024/0428079
Zhang et al. “Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners.” arXiv.org. 3 March 2023. 17 pages.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jeffrey R. Swearingen whose telephone number is (571)272-3921. The examiner can normally be reached M-F 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oscar Louie can be reached at 571-270-1684. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Jeffrey R. Swearingen
Primary Examiner
Art Unit 2445
/Jeffrey R Swearingen/Primary Examiner, Art Unit 2445
Prosecution Insights