DETAILED ACTION
This action is responsive to the claims filed 07/23/2026, Claims 1, 2, 7-14, 18, 19 are pending in the case. Claims 1, 12, and 19 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/23/2026 has been entered.
Response to Arguments
Applicant's arguments filed 04/23/2026, with respect to the 35 U.S.C. 101 rejection, have been fully considered but they are not persuasive.
With respect to 35 U.S.C. 101:
Applicant begins by pointing out that the independent claims can not be performed in the mind and include additional elements which amount to significantly more.
Examiner disagrees. In summary the rejection does not make the assertion that the entirety of the claim are performed in the mind, only that certain limitations may be performed in the mind. While certain other limitations (additional elements) do not reflect the improvement and as such do not amount to significantly more.
Applicant highlights several sections of the specification noting that the models have a massive number of parameters which are difficult to deploy in limited applications. The Application proposes obtaining a model having resource limitations and determining whether an amount of training data exceeds a threshold. Applicant continues quoting many of the limitations of the claim, suggesting the claim reflects improving the technological field via applying compressed models analogous to the improvements reflected in Thales Visionix Inc v U.S. and McRO.
Examiner notes both Thales and McRO reflect improvements to the functioning of technology by explicitly reciting limitations which result in a particular improvement to technological functioning. Critically, the improvement is a consequence of the configuration and functioning of the technology, those functions inoperable within the mind. In contrast, the improvements related to improvements to resources efficiency are the result of the abstract ideas alone (i.e determining that a model should be compressed). The principal reason for the improvements in the instant application are the result of decision made in the mind rather than the particular functioning of the technology. While the instant claim recites obtaining models and information as well as training the original language model, these additional elements do not result in the improvements to the model size. The improvements are a result of the mental evaluations about the data and models themselves, rather than the particular functioning of technology. As noted in the rejection obtaining information and models as claimed amounts to mere data gathering and thus cannot reflect the improvement. Further, training the model is application at a high degree of generality which makes used of the judicial exception (see MPEP 2106.05(f)). Specific details which describe how the training leads to an improvement are not claimed thus not an additional element which reflects and improvement.
Applicant argues the recited limitations cannot be performed in the mind. Applicant suggests the claim is similar to the claim described in the MPEP for calculating an absolute position of a GPS which did not recite an abstract idea.
Examiner disagrees. While training as claimed is not a mental step, the claim recites many mental steps identified clearly in the rejection. The claim is not similar to the example at least because the claim recites abstract ideas. While the training may very well include “billions” of parameters, the other recited steps such as “extracting common knowledge” and “determining an amount of data” are reasonably considered decisions made in the mind about data. The fact that the data is large or that certain other steps such as training is computationally complex does not in any way suggest that certain other limitations can be performed in the mind. For example, one may determine in the mind that 1 trillion GBs of training data is too large for a model to process. This decision can be made regardless of the size and complexity of the model.
The same analysis follows for searching for a model, and establishing cross talk relationships. Nothing in the specification suggests that these processes require prohibitively complex analysis such that they can not be performed in the mind or with pen and paper as a tool. At most, the disclosure suggests the model itself is complex, it does not follow that decisions about the model are necessarily confined to analysis which can only be performed by a particular machine. In fact both Applicant and the disclosure are silent with respect to the particular analysis required to “determine whether an amount…exceeds a threshold”, “determine prompt information”, “establish…relationships”, “extract common knowledge” which would make these steps inoperable within the human mind. At most applicant reiterates that the associated model is complex.
Therefore, the rejection is maintained and updated in view of the amendments.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12-14, 18, 19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The limitation “and the target language model being a compressed version of the original language model with a fewer number of parameters and a faster inference speed” in claim 12 and 19 and 13 is a relative term which renders the claim indefinite. The term “fewer” and “faster” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Dependent claims 13-14 and 18 are rejected by virtue of their dependency on the rejected base claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 2, 7-14, 18 and 19 are rejected under 35 U.S.C. 101 because the claims are directed to an abstract idea without significantly more.
Regarding Claim 1
Under step 1, the claim is directed to A method implemented by a computing device, which is directed to a process, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations “determining whether an amount of training data for the original language model exceeds a threshold; determining that the original language model needs to be compressed after determining that the amount of training data exceeds the threshold…determining a task that needs to be processed by the original language model; … extracting common knowledge in the original language model as a first knowledge loss, and extracting knowledge corresponding to the task in the first language model as a second knowledge loss;… performing a search in a neural architecture search based at least in part on using an objective function to obtain the target language model, the objective function being obtained through a synthesis of the first knowledge loss and the second knowledge loss,”…”compressing the original language model, wherein compressing the original language model comprises:”, “wherein performing the search in the neural architecture search comprises: determining prompt information based at least in part on the first knowledge loss and the second knowledge loss, wherein determining the prompt information comprises :establishing cross-task relationships based at least in part on the first knowledge loss and the second knowledge loss, the cross-task relationships being used to indicate relationships between multiple tasks; and determining the prompt information based at least in part on the cross-task relationships; and searching for a model indicated by the prompt information in a hierarchical search space corresponding to the network architecture search, the model by the prompt information being determined as the target language model.”
Determinations and decisions about compression are decisions about data that can be performed in the human mind. The claim recites compression is involving both abstract idea steps and additional elements. The aspect of compression of the original language model which includes the abstract ideas recites an abstract idea. The training step contained within the compression claimed is evaluated in Step 2A as an additional element and not considered part of the abstract idea. Further, extracting knowledge and performing a search based on data features is an activity performed in the mind as they describe generalized data analysis, The claim does not recite details of how the search is performed to suggest that it is not an evaluation based on abstract data, rather the claim recites the search is merely based on certain claim features. These steps are not described as computer confined analysis such as computer confined memory manipulation.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. The claims recite the additional element(s) “… the target language model being a compressed version of the original language model with a fewer number of parameters … training the original language model based at least in part on features and the training data of the task to obtain a first language model, the first language model being a fine-tuned version of the original language model;” describes an application at a high degree of generality which makes use of the recited exception, see MPEP 2106.05(f).
In addition, the claim recites additional element(s) “obtaining a target language model to be deployed in a real-time application that has strict limitations on computing resources and inference times, obtaining the target language model…obtaining an original language model, the original language model being a pre- trained context characterization encoder;” that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).
The claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Further, the additional elements, “obtaining a target language model to be deployed in a real-time application that has strict limitations on computing resources and inference times, obtaining the target language model…obtaining an original language model” are insignificant extra-solution activities that are considered well-understood, routine, conventional activities, for the following reasons. Examiner notes this amounts to receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i). According to MPEP 2106.05(d)(II)(i), “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner”. As such, the insignificant extra-solution activities are considered well-understood, routine, conventional activities. Therefore, the claim is not patent eligible.
Accordingly, the recited additional elements, when taken alone or in combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea.
Regarding Claim 2
The claim is directed to a process. The claim recites the following limitations “…and determining the target language model based on the search result.” Under Step 2A Prong 1, these limitations describe a step performed in the mind. The claim sets no limits on the neural architecture search and includes an architecture search based entirely on decisions made in the human mind.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. In particular, the claims recite the additional element(s) the limitations “inputting features of the task into the neural architecture search to obtain a search result” amounts to mere instructions to apply a computer technology to an abstract idea, see MPEP 2106.05(f). In this case, the training is applied in order to determine a search result.
Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 7-8
The claim is directed to a process,
Similarly, claim 7 recites the following limitations:
“recording a first knowledge loss sequence of the original language model and a second knowledge loss sequence of the first language model in the knowledge aggregator, wherein the first knowledge loss sequence includes a knowledge loss of the original language model at at least one moment of training, the second knowledge loss sequence includes a second knowledge loss of the first language model at the at least one moment of training; clustering multiple tasks to obtain at least one meta-task group based on the first knowledge loss sequence of the original language model and the second knowledge loss sequence of the first language model, wherein the meta-task group includes at least two tasks whose similarity degree is greater than a first threshold; performing normalization based on a target value of the meta-task group to obtain a weight of the meta-task group, wherein the target value is used to indicate an average classification performance of the meta-task group; and establishing the cross-task relationships based on the weight of the meta-task group.”
Similarly, claim 8 recites the following limitations:
“extracting the common knowledge in the original language model as the first knowledge loss in a knowledge decomposer; and extracting the knowledge corresponding to the task in the first language model as the second knowledge loss including extracting the knowledge corresponding to the task in the first language model as the second knowledge loss in the knowledge decomposer.”
Under Step 2A Prong 1, these limitations describe a step performed in the mind. The claim sets no limits on the neural architecture search and includes an architecture search based entirely on decisions made in the human mind.
Furthermore, under step 2A Prong 2 and 2B, the claim does not recite additional elements to consider other than those considered in the independent claim.
Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 9
The claim is directed to a process. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
Furthermore, under step 2A Prong 2 and 2B: The judicial exception in not integrated into a practical application or provide significantly more. In particular, the claims recite the additional element(s) “wherein the knowledge decomposer comprises a set of probe classifiers obtained by training the original language model and the first language model.” which is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h). The limitation merely limits the language model to classification tasks.
Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 10
The claim is directed to a process. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
Furthermore, under step 2A Prong 2 and 2B: The judicial exception in not integrated into a practical application or provide significantly more. In particular, the claims recite the additional element(s) “adding target task parameters of the task to the original language model;
and training the target task parameters on a newly added corpus of the task to obtain the first language model.” which is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h). The limitation merely limits the language model to classification tasks of a particular corpus.
Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 11
The claim is directed to a process. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
Furthermore, under step 2A Prong 2 and 2B: The judicial exception in not integrated into a practical application or provide significantly more. In particular, the claims recite the additional element(s) “wherein parameters of the original language model remain unchanged when training the target task parameters on the newly added corpus of the task” which is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h). The limitation merely limits the language model to training particular parameters at a time.
Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 12
Under step 1, the claim is directed to One or more computer readable media storing executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts, which is directed to an article of manufacture, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations “determining a task corresponding to the textual information… compressing the original language model to obtain a target language comprising the original language model comprises… extracting common knowledge in the original language model as a first knowledge loss, and extracting knowledge corresponding to the task in the first language model as a second knowledge loss; …performing a search in a neural architecture search based at least in part on using an objective function to obtain the target language model, the objective function being obtained through a synthesis of the first knowledge loss and the second knowledge loss;” As previously noted, such steps can be performed in the mind for the reasons described in the rejection of claim 1.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. The claims recite the additional element(s) “wherein the task is processed by an original language model the original language model being a pre-trained context characterization encoder … processing the textual information based on the target language model to obtain a textual processing result… training the original language model based at least in part on features of the task to obtain a first language model, the first language model being a fine-tuned version of the original language model;” describes an application at a high degree of generality which makes use of the recited exception, see MPEP 2106.05(f).
In addition, the claim recites additional element(s) “obtaining textual information uploaded to a target platform… and outputting the textual processing result to the target platform.” that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).
In addition, the claim recites additional element(s) “and the target language model being a compressed version of the original language model with a fewer number of parameters and a faster inference speed;” is generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h)
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Further, the additional elements, obtaining and outputting information are insignificant extra-solution activities that are considered well-understood, routine, conventional activities, for the following reasons. Examiner notes this amounts to receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i). According to MPEP 2106.05(d)(II)(i), “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner”. As such, the insignificant extra-solution activities are considered well-understood, routine, conventional activities. Therefore, the claim is not patent eligible.
Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 13-14
The claim is directed to an article of manufacture. Claim 13 recites the following limitations:
“wherein the textual information comprises textual transaction information that is uploaded to a transaction platform when the target platform is the transaction platform”
Similarity claim 14 recites: “wherein the textual transaction information comprises at least one of: textual query information for querying a transaction object; textual information associated with a transaction operation performed by the transaction object; textual evaluation information for evaluating the transaction object; and textual search information for querying an associated object related to the transaction object.”
Under Step 2A Prong 1, these limitations only serve to describe the abstract idea addressed in the independent claim.
Furthermore, under step 2A Prong 2 and 2B, the claim does not recite additional elements to consider other than those considered in the independent claim. Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 18
The claim is directed to an article of manufacture. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
The claim is rejected for the reasons set forth in the rejection of claim 10
Regarding Claim 19
Under step 1, the claim is directed to an apparatus for using a target language model deployed in a real-time application that has strict limitations on computing resources and inference times, which is directed to an machine, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations “determining a task corresponding to the textual input information … compressing the original language model to obtain a target language comprising the original language model comprises …extracting common knowledge in the original language model as a first knowledge loss, and extracting knowledge corresponding to the task in the first language model as a second knowledge loss; and performing a search in a neural architecture search based at least in part on using an objective function to obtain the target language model, the objective function being obtained through a synthesis of the first knowledge loss and the second knowledge loss” As previously noted, such steps can be performed in the mind for the reasons described in the rejection of claim 1.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. the claims recite the additional element(s) the limitations “and memory storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:” amounts to mere instructions to apply a computer technology to an abstract idea, see MPEP 2106.05(f).
In addition, the limitations, “the target language model is obtained by…processing the textual input information based on the target language model that is read to obtain a textual processing result;… wherein the task is processed by an original language model… the original language model being a pre-trained context characterization encoder … training the original language model based at least in part on features of the task to obtain a first language model, the first language model being a fine- tuned version of the original language model;” describes an application at a high degree of generality which makes use of the recited exception, see MPEP 2106.05(f).
In addition, the claim recites additional element(s) “receiving textual input information, wherein the textual input information is collected based on at least one text collector associated with a textual processing system;… and reading a target language model… and outputting the textual processing result.” that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g).
In addition, the claim recites additional element(s) “and the target language model being a compressed version of the original language model with a fewer number of parameters and a faster inference speed;” is generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h)
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Further, the additional elements, obtaining and outputting information are insignificant extra-solution activities that are considered well-understood, routine, conventional activities, for the following reasons. Examiner notes this amounts to receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i). According to MPEP 2106.05(d)(II)(i), “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner”. As such, the insignificant extra-solution activities are considered well-understood, routine, conventional activities. Therefore, the claim is not patent eligible.
Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Allowable Subject Matter
Claim 1, 2, 7-14, 18 and 19 are rejected under 35 U.S.C 101
Claim 1, 2, 7-14, 18 and 19 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101.
The following is a statement of reasons for the indication of allowable subject matter:
From independent claim 1:
determining whether an amount of training data for the original language model exceeds a threshold;
determining that the original language model needs to be compressed after determining that the amount of training data exceeds the threshold
The closes prior art of Record, Zhou et al “Adaptive Quantization for Deep Neural Network” which describes determining the amount of compression based in part on a noise attribute of the training data. The amount of training data is not compared to a threshold, rather the accuracy which is a function of the noise present within the training data is compared to the threshold. Further, Leen US PG Document ID US 11444845 B1, describes Determining the training data quantity exceeds a threshold, and that the original complete model is used for processing instead of a compressed version, which is the converse of the claimed limitations. The previously cited art made of record do not discuss determining to compress the original language model after a determination that the amount of training data exceeds the claimed threshold. It would not have been obvious to one or ordinary skill in the art before the effective filing data to combine these references to teach at least the limitations above.
Conclusion
Prior art:
Tang et al. “Distilling Task-Specific Knowledge from BERT into Simple Neural Networks” addresses task specific learning using a BERT representation model. Which has been compressed from the larger original model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNATHAN R GERMICK whose telephone number is (571)272-8363. The examiner can normally be reached M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.R.G./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights