DETAILED ACTION
This action is responsive to the claims filed 04/23/2026, Claims 1, 2, 5-14, 18 and 19 are pending in the case. Claims 1, 12, and 19 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 04/23/2026, with respect to the 35 U.S.C. 101 rejection, have been fully considered but they are not persuasive.
With respect to 35 U.S.C. 101:
Applicant argues the claim includes features that are significantly more and integrate a practical application and cites the specification to suggest the improvements over large scale models noting that obtaining a target language model that reduces the computational cost improves the speed by decreasing the number of parameters compared to the original model.
Examiner disagrees. The MPEP notes that the improvement should be reflected in the additional elements. A mere description in the specification that the invention solves a problem does not make the claims eligible. Obtaining and converting a model to obtain a model with fewer parameters does not reflect and improvement. Critically, an improvement should be to the functioning or inner workings of the technology. The claim recites a mental step (searching for a model based on loss features) without reciting additional elements which describe any specific functioning which imparts the improvement. The improvement cannot be a result of a recited abstract idea alone. See MPEP 2106.05(a).
For example, the cited paragraph (0019) of the specification suggests that automatic compression for a target task enables the improvement. However, the claims do not describe the compression as an additional element which particularly enables the improvement, but rather the result of applying judicial exceptions (determining, extraction, searching) to obtained and trained models. The claims recite that the compression is a result directly of the “performing a search…[based on features]”
Applicant specifically provides the limitation “determining whether an amount of training data for the original language model exceeds a threshold” noting this reduces the cost and improves speed by reducing the parameters needed for the target model.
Examiner highlights it is clear that the improvements to the size/speed of the model are directly the result of the judicial exception alone, i.e the determination that a model should be compressed. Rather than an particular technological functions.
In the interest of compact prosecution, the specification was evaluated to consider elements which may suggest features which if amended to the claims would reflect and improvement or practical application.
Specification para. 0097 generally suggests that compressed implementation improves compute on resource constrained devices with no details describing the particular technical elements which result in the improvement (i.e. limitations which would not be considered part of the abstract idea recited).
The specification makes references frequently to Neural Architecture Search. Paragraph 0065 describes search as including a series of mental evaluations (i.e determinations about data). The specification does not appear to include any description of the inner workings or how it is performed specifically by a physical processor except via description of the results and inputs to the search algorithm
Similarly, the specification describes performing training conditioned on clustering and/or tasks and being based on various loss terms and input values, without description of the computer confined functions performed by training to consider.
The Specification describes a target platform to obtain and output information, without any additional description of the implementation of the target platform to suggest that it makes use of or applies the judicial exception such that it may be considered significantly more than the recited judicial exception.
Therefore, the rejection is maintained and updated in view of the amendments.
With respect to prior art:
Applicant notes that the cited art does not teach the amended claims. Examiner notes upon an updated search and further review
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 2, 5-14, 18 and 19 are rejected under 35 U.S.C. 101 because the claims are directed to an abstract idea without significantly more.
Regarding Claim 1
Under step 1, the claim is directed to A method implemented by a computing device, which is directed to a process, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations “determining whether an amount of training data for the original language model exceeds a threshold; determining that the original language model needs to be compressed after determining that the amount of training data exceeds the threshold…determining a task that needs to be processed by the original language model; … extracting common knowledge in the original language model as a first knowledge loss, and extracting knowledge corresponding to the task in the first language model as a second knowledge loss;… performing a search in a neural architecture search based at least in part on using an objective function to obtain the target language model, the objective function being obtained through a synthesis of the first knowledge loss and the second knowledge loss,”. Determinations and compression are decisions about data that can be performed in the human mind. The specification does not describe how any compression is performed, instead it describes the result of compressing. Further, extracting knowledge and performing a search based on data features is an activity performed in the mind as they describe generalized data analysis, The claim does not recite details of how the search is performed to suggest that it is not an evaluation based on abstract data, rather the claim recites the search is merely based on certain claim features. These steps are not described as computer confined analysis such as computer confined memory manipulation.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. The claims recite the additional element(s) “… the target language model being a compressed version of the original language model with a fewer number of parameters … training the original language model based at least in part on features and the training data of the task to obtain a first language model, the first language model being a fine-tuned version of the original language model;” describes an application at a high degree of generality which makes use of the recited exception, see MPEP 2106.05(f). In addition, the claim recites additional element(s) “obtaining a target language model to be deployed in a real-time application that has strict limitations on computing resources and inference times, obtaining the target language model…obtaining an original language model, the original language model being a pre- trained context characterization encoder;” that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Further, the additional elements, “obtaining a target language model to be deployed in a real-time application that has strict limitations on computing resources and inference times, obtaining the target language model…obtaining an original language model” are insignificant extra-solution activities that are considered well-understood, routine, conventional activities, for the following reasons. Examiner notes this amounts to receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i). According to MPEP 2106.05(d)(II)(i), “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner”. As such, the insignificant extra-solution activities are considered well-understood, routine, conventional activities. Therefore, the claim is not patent eligible.
Regarding Claim 2
The claim is directed to a process. The claim recites the following limitations “…and determining the target language model based on the search result.” Under Step 2A Prong 1, these limitations describe a step performed in the mind. The claim sets no limits on the neural architecture search and includes an architecture search based entirely on decisions made in the human mind.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. In particular, the claims recite the additional element(s) the limitations “inputting features of the task into the neural architecture search to obtain a search result” amounts to mere instructions to apply a computer technology to an abstract idea, see MPEP 2106.05(f). In this case, the training is applied in order to determine a search result.
Accordingly, the recited additional elements, when taken alone or in combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea.
Regarding Claim 5-8
The claim is directed to a process, claim 5 recites the following limitations:
“determining prompt information based on the first knowledge loss and the second knowledge loss; searching for a model indicated by the prompt information in an architecture search space corresponding to the neural architecture search; and determining the model indicated by the prompt information as the target language model.”
Similarly, claim 6 recites the following limitations:
“establishing cross-task relationships based on the first knowledge loss and the second knowledge loss in a knowledge aggregator, wherein the cross-task relationships are used to indicate relationships among multiple tasks; and determining the prompt information based on the cross-task relationships.”
Similarly, claim 7 recites the following limitations:
“recording a first knowledge loss sequence of the original language model and a second knowledge loss sequence of the first language model in the knowledge aggregator, wherein the first knowledge loss sequence includes a knowledge loss of the original language model at at least one moment of training, the second knowledge loss sequence includes a second knowledge loss of the first language model at the at least one moment of training; clustering multiple tasks to obtain at least one meta-task group based on the first knowledge loss sequence of the original language model and the second knowledge loss sequence of the first language model, wherein the meta-task group includes at least two tasks whose similarity degree is greater than a first threshold; performing normalization based on a target value of the meta-task group to obtain a weight of the meta-task group, wherein the target value is used to indicate an average classification performance of the meta-task group; and establishing the cross-task relationships based on the weight of the meta-task group.”
Similarly, claim 8 recites the following limitations:
“extracting the common knowledge in the original language model as the first knowledge loss in a knowledge decomposer; and extracting the knowledge corresponding to the task in the first language model as the second knowledge loss including extracting the knowledge corresponding to the task in the first language model as the second knowledge loss in the knowledge decomposer.”
Under Step 2A Prong 1, these limitations describe a step performed in the mind. The claim sets no limits on the neural architecture search and includes an architecture search based entirely on decisions made in the human mind.
Furthermore, under step 2A Prong 2 and 2B, the claim does not recite additional elements to consider other than those considered in the independent claim. Accordingly, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea.
Regarding Claim 9
The claim is directed to a process. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
Furthermore, under step 2A Prong 2 and 2B: The judicial exception in not integrated into a practical application or provide significantly more. In particular, the claims recite the additional element(s) “wherein the knowledge decomposer comprises a set of probe classifiers obtained by training the original language model and the first language model.” which is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h). The limitation merely limits the language model to classification tasks. Accordingly, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea. Accordingly, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea.
Regarding Claim 10
The claim is directed to a process. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
Furthermore, under step 2A Prong 2 and 2B: The judicial exception in not integrated into a practical application or provide significantly more. In particular, the claims recite the additional element(s) “adding target task parameters of the task to the original language model;
and training the target task parameters on a newly added corpus of the task to obtain the first language model.” which is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h). The limitation merely limits the language model to classification tasks of a particular corpus. Accordingly, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea. Accordingly, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea.
Regarding Claim 11
The claim is directed to a process. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
Furthermore, under step 2A Prong 2 and 2B: The judicial exception in not integrated into a practical application or provide significantly more. In particular, the claims recite the additional element(s) “wherein parameters of the original language model remain unchanged when training the target task parameters on the newly added corpus of the task” which is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h). The limitation merely limits the language model to training particular parameters at a time. Accordingly, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea. Accordingly, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea, nor do they amount to significantly more than the judicial exception because they do not impose any meaningful limits on practicing the abstract idea.
Regarding Claim 12
Under step 1, the claim is directed to One or more computer readable media storing executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts, which is directed to an article of manufacture, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations “determining a task corresponding to the textual information… extracting common knowledge in the original language model as a first knowledge loss, and extracting knowledge corresponding to the task in the first language model as a second knowledge loss; …performing a search in a neural architecture search based at least in part on using an objective function to obtain the target language model, the objective function being obtained through a synthesis of the first knowledge loss and the second knowledge loss;” As previously noted, such steps can be performed in the mind for the reasons described in the rejection of claim 1.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. The claims recite the additional element(s) “wherein the task is processed by an original language model the original language model being a pre-trained context characterization encoder and a target language model is obtained by… processing the textual information based on the target language model to obtain a textual processing result… training the original language model based at least in part on features of the task to obtain a first language model, the first language model being a fine-tuned version of the original language model;” describes an application at a high degree of generality which makes use of the recited exception, see MPEP 2106.05(f). In addition, the claim recites additional element(s) “obtaining textual information uploaded to a target platform… and outputting the textual processing result to the target platform.” that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Further, the additional elements, obtaining and outputting information are insignificant extra-solution activities that are considered well-understood, routine, conventional activities, for the following reasons. Examiner notes this amounts to receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i). According to MPEP 2106.05(d)(II)(i), “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner”. As such, the insignificant extra-solution activities are considered well-understood, routine, conventional activities. Therefore, the claim is not patent eligible.
Regarding Claim 13-14
The claim is directed to an article of manufacture. Claim 13 recites the following limitations:
“wherein the textual information comprises textual transaction information that is uploaded to a transaction platform when the target platform is the transaction platform”
Similarity claim 14 recites: “wherein the textual transaction information comprises at least one of: textual query information for querying a transaction object; textual information associated with a transaction operation performed by the transaction object; textual evaluation information for evaluating the transaction object; and textual search information for querying an associated object related to the transaction object.”
Under Step 2A Prong 1, these limitations only serve to describe the abstract idea addressed in the independent claim.
Furthermore, under step 2A Prong 2 and 2B, the claim does not recite additional elements to consider other than those considered in the independent claim. Accordingly, the claim does not provide a practical application and is not considered to be significantly more.
Regarding Claim 18
The claim is directed to an article of manufacture. Each of the limitations described in the claim, under Step 2A Prong 1, do not recite any additional abstract ideas beyond those described in the independent claim
The claim is rejected for the reasons set forth in the rejection of claim 10
Regarding Claim 19
Under step 1, the claim is directed to an apparatus for using a target language model deployed in a real-time application that has strict limitations on computing resources and inference times, which is directed to an machine, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations “determining a task corresponding to the textual input information … extracting common knowledge in the original language model as a first knowledge loss, and extracting knowledge corresponding to the task in the first language model as a second knowledge loss; and performing a search in a neural architecture search based at least in part on using an objective function to obtain the target language model, the objective function being obtained through a synthesis of the first knowledge loss and the second knowledge loss” As previously noted, such steps can be performed in the mind for the reasons described in the rejection of claim 1.
Step 2A Prong Two Analysis: The judicial exception in not integrated into a practical application. the claims recite the additional element(s) the limitations “and memory storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:” amounts to mere instructions to apply a computer technology to an abstract idea, see MPEP 2106.05(f). In addition, the limitations, “the target language model is obtained by…processing the textual input information based on the target language model that is read to obtain a textual processing result;… wherein the task is processed by an original language model… the original language model being a pre-trained context characterization encoder … training the original language model based at least in part on features of the task to obtain a first language model, the first language model being a fine- tuned version of the original language model;” describes an application at a high degree of generality which makes use of the recited exception, see MPEP 2106.05(f). In addition, the claim recites additional element(s) “receiving textual input information, wherein the textual input information is collected based on at least one text collector associated with a textual processing system;… and reading a target language model… and outputting the textual processing result.” that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Further, the additional elements, obtaining and outputting information are insignificant extra-solution activities that are considered well-understood, routine, conventional activities, for the following reasons. Examiner notes this amounts to receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i). According to MPEP 2106.05(d)(II)(i), “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner”. As such, the insignificant extra-solution activities are considered well-understood, routine, conventional activities. Therefore, the claim is not patent eligible.
Allowable Subject Matter
Claim 1, 2, 5-14, 18 and 19 are rejected under 35 U.S.C 101
Claim 1, 2, 5-14, 18 and 19 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101.
The following is a statement of reasons for the indication of allowable subject matter:
From independent claim 1:
determining whether an amount of training data for the original language model exceeds a threshold;
determining that the original language model needs to be compressed after determining that the amount of training data exceeds the threshold
The closes prior art of Record, Zhou et al “Adaptive Quantization for Deep Neural Network” which describes determining the amount of compression based in part on a noise attribute of the training data. The amount of training data is not compared to a threshold, rather the accuracy which is a function of the noise present within the training data is compared to the threshold. Further, Leen US PG Document ID US 11444845 B1, describes Determining the training data quantity exceeds a threshold, and that the original complete model is used for processing instead of a compressed version, which is the converse of the claimed limitations. The previously cited art made of record do not discuss determining to compress the original language model after a determination that the amount of training data exceeds the claimed threshold. It would not have been obvious to one or ordinary skill in the art before the effective filing data to combine these references to teach at least the limitations above.
Conclusion
Prior art:
Tang et al. “Distilling Task-Specific Knowledge from BERT into Simple Neural Networks” addresses task specific learning using a BERT representation model. Which has been compressed from the larger original model.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNATHAN R GERMICK whose telephone number is (571)272-8363. The examiner can normally be reached M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.R.G./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights