Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. This communication is responsive to the amendment filed on 12/29/2025.
3. Claims 1-20 are currently pending in this Office action. This action is made Final.
Claim Objections
4. The claim objection made in the prior Office action is withdrawn in view of the claim amendment filed on 12/29/2025.
Double Patenting
5. The examiner acknowledges the applicant’s decision to hold the double patenting rejection in abeyance. Thus, the previously issued double patenting rejection is maintained.
6. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
7. Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of U.S. Patent No. 12,264,564. Although the claims at issue are not identical, they are not patentably distinct from each other because the claimed invention of the instant application is a similar version of the claimed invention of the above identified U.S. Patent with the similar intended scope as shown below:
Instant Application
Patent No. 12,265,564
Claim 1. A method of instance-wise adaptive knowledge injection in a large language pre-trained language model, the method being executed by at least one processor, the method comprising:
determining whether external knowledge is needed for a query to be input to a target large scale pre-trained language model to perform a task based on a thrust score of the query, wherein determining the thrust score comprises:
generating one or more clusters based on the target large scale pre-trained language model;
for each cluster, determining a respective unit vector associated with the query that points from a query vector of the query to a center of a respective cluster; and
determining the thrust score for the query based on a sum vector of one or more unit vectors weighted by a size of each of the one or more clusters;
[Claim 4. The method of claim 1, wherein the query is represented using last layer hidden states of the target large scale pre-trained language model associated with the query.]
based on determining that external knowledge is needed for the query, augmenting the query with the external knowledge;
generating a combined dataset based on combining a first dataset and the augmented query; and
applying the combined dataset to the target large scale pre-trained language model so that the target large scale pre-trained language model performs the task based on the augmented query.
Claim 1. A method of instance-wise adaptive knowledge injection in a large language pre-trained language model (PTLM), the method being executed by at least one processor, the method comprising:
determining whether external knowledge is needed for a respective query in a plurality of queries of a first dataset based on a thrust score of the respective query using internal knowledge of a target large scale pre-trained language model, wherein determining the thrust score comprises:
generating a query distribution based on the target large scale pre-trained language model;
generating one or more clusters based on the query distribution; for the respective query among the plurality of queries, determining one or more unit vectors associated with the query that pointing from a query vector of the query to a center of each cluster among the one or more clusters, wherein each unit vector is associated with the query and a respective cluster among the one or more clusters; and
determining the thrust score for the respective query based on a sum vector of the one or more unit vectors weighted by a size of each of the one or more clusters, and
wherein each query in the or more clusters is represented using last layer hidden states of the target large scale pre-trained language model associated with each query;
based on determining that external knowledge is needed for one or more queries among the plurality of queries of the first dataset, augmenting the one or more queries with respective pieces of external knowledge;
generating a combined dataset based on combining the first dataset and the one or more augmented queries; and
applying the combined dataset to the target large scale pre-trained language model.
Claim 9. An apparatus for instance-wise adaptive knowledge injection in a pre-trained language model, the apparatus comprising:
at least one memory configured to store computer program code;
at least one processor configured to access the computer program code and operate as instructed by the computer program code, the computer program code including:
first determining code configured to cause the at least one processor to determine whether external knowledge is needed for a query to be input to a target large scale pre-trained language model to perform a task based on a thrust score of the query,
[Claim 12. The apparatus of claim 7, wherein one or more last layers of decoders of the target large scale pre-trained language model are used to generate the query distribution.]
based on determining that external knowledge is needed for the query, first augmenting code configured to cause the at least one processor to augment the query with the external knowledge;
first generating code configured to cause the at least one processor to generate a combined dataset based on combining a first dataset and the augmented query; and
first applying code configured to cause the at least one processor to apply the combined dataset to the target large scale pre-trained language model so that the target large scale pre-trained language model performs the task based on the augmented query,
wherein the first determining code comprises:
second generating code configured to cause the at least one processor to generate one or more clusters based on the target large scale pre-trained language model;
second determining code configured to cause the at least one processor to determine, for each cluster, determining a respective unit vector associated with the query that points from a query vector of the query to a center of a respective cluster; and
third determining code configured to cause the at least one processor to determine the thrust score for the query based on a sum vector of one or more unit vectors weighted by a size of each of the one or more clusters.
Claim 7. An apparatus for instance-wise adaptive knowledge injection in a pre-trained language model (PTLM), the apparatus comprising:
at least one memory configured to store computer program code;
at least one processor configured to access the computer program code and operate as instructed by the computer program code, the computer program code including:
first determining code configured to cause the at least one processor to determine whether external knowledge is needed for a respective query in a plurality of queries of a first dataset based on a thrust score of the respective query using internal knowledge of a target large scale pre-trained language model,
wherein each query in the one or more clusters is represented using last layer hidden states of the target large scale pre-trained language model associated with each query;
based on determining that external knowledge is needed for one or more queries among the plurality of queries of the first dataset, first augmenting code configured to cause the at least one processor to augment the one or more queries with respective pieces of external knowledge;
first generating code configured to cause the at least one processor to generate a combined dataset based on combining the first dataset and the one or more augmented queries; and
first applying code configured to cause the at least one processor to apply the combined dataset to the target large scale pre-trained language model,
wherein the first determining code comprises:
second generating code configured to cause the at least one processor to generate a query distribution based on the target large scale pre-trained language model;
third generating code configured to cause the at least one processor to generate one or more clusters based on the query distribution;
second determining code configured to cause the at least one processor to determine, for the respective query among the plurality of queries, one or more unit vectors associated with the query that pointing from a query vector of the query to a center of each cluster among the one or more clusters, wherein each unit vector is associated with the query and a respective cluster among the one or more clusters; and
third determining code configured to cause the at least one processor to determine the thrust score for the query based on a sum vector of the one or more unit vectors weighted by a size of each of the one or more clusters.
Claim 17. A non-transitory computer-readable medium storing computer code that is configured to, when executed by at least one processor, cause the at least one processor to implement instance-wise adaptive knowledge injection in a pre-trained language model that:
determines whether external knowledge is needed for a query to be input to a target large scale pre-trained language model to perform a task based on a thrust score of the query, wherein determining the thrust score comprises:
[Claim 18. The non-transitory computer-readable medium of claim 17, wherein determining whether the external knowledge is needed is based on whether the target large scale pre-trained language model has no relevant knowledge, the target large scale pre-trained language model is not familiar with the query, or the target large scale pre-trained language model includes controversial knowledge associated with the query.]
generating one or more clusters based on the target large scale pre-trained language model,
for each cluster, determining a respective unit vector associated with the query that points from a query vector of the query to a center of a respective cluster,
determining the thrust score for the query based on a sum vector of one or more unit vectors weighted by a size of each of the one or more clusters; and
[Claim 20. The non-transitory computer-readable medium of claim 18, wherein the query is represented using last layer hidden states of the target large scale pre-trained language model associated with the query.]
based on determining that external knowledge is needed for the query, augments the query with the external knowledge;
generates a combined dataset based on combining a first dataset and the augmented query; and
applies the combined dataset to the target large scale pre-trained language model so that the target large scale pre-trained language model performs the task based on the augmented query.
Claim 13.. A non-transitory computer-readable medium storing computer code that is configured to, when executed by at least one processor, cause the at least one processor to implement instance-wise adaptive knowledge injection in a pre-trained language model (PTLM) that:
determines whether external knowledge is needed for a respective query in a plurality of queries of a first dataset based on a thrust score of the respective query using internal knowledge of a target large scale pre-trained language model, wherein determining the thrust score comprises:
generating a query distribution based on the target large scale pre-trained language model,
generating one or more clusters based on the query distribution,
for the respective query among the plurality of queries,
determining one or more unit vectors associated with the query that pointing from a query vector of the query to a center of each cluster among the one or more clusters, wherein each unit vector is associated with the query and a respective cluster among the one or more clusters,
determining the thrust score for the respective query based on a sum vector of the one or more unit vectors weighted by a size of each of the one or more clusters, and
wherein each query in the one or more clusters is represented using last layer hidden states of the target large scale pre-trained language model associated with each query;
based on determining that external knowledge is needed for one or more queries among the plurality of queries of the first dataset, augments the one or more queries with respective pieces of external knowledge;
generates a combined dataset based on combining the first dataset and the one or more augmented queries; and
applies the combined dataset to the target large scale pre-trained language model.
Claim Rejections - 35 USC § 101
8. Applicant’s arguments [Remarks filed on 12/29/2025] are not persuasive and the examiner maintains the rejection as set forth below:
9. Claims 1-3, 6-11 and 14-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The eligibility analysis in support of these findings is provided below, in accordance with the 2019 Revised Patent Subject Matter Eligibility Guidance, hereinafter 2019 PEG.
Step 1. In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), it is noted that the method(s) of claims 1-3, 6-11 and 14-19 are directed to one of the eligible categories of subject matter and therefore satisfy Step 1.
Step 2A. In accordance with Step 2A, prog one of the 2019 PEG:
In claim 1-3 and 6-8, the limitations directed to additional elements include: at least one processor; in claims 9-11 and 14-16, the limitations directed to additional elements include: at least one memory, at least one processor; and in claim 17-19, the limitations directed to additional elements include: at least one processor.
In exemplary claim 1, limitations reciting the abstract idea are as follows:
(1) determining whether external knowledge is needed for a query to be input to a target large scale pre-trained language model to perform a task based on a thrust score of the query; (2) generating one or more clusters based on the target large scale pre-trained language model; (3) for each cluster determining a respective unit vector associated with the query that points from a query vector of the query to a center of a respective cluster; and (4) determining the thrust score for the query based on a sum vector of one or more unit vectors weighted by a size of the one or more clusters; (5) augmenting the query with the external knowledge; (6) generating a combined dataset based on combining a first dataset and the augmented query; and (7) applying the combined dataset to the target large scale pre-trained model.
These limitations, under the broadest reasonable interpretation, recite mental processes because these limitations can be performed in the human mind or using pen and paper. The examiner believes that the steps disclosed in claim 1 [generating one or more clusters, determining the thrust score, augmenting the query, generating a combined dataset, applying the combined dataset to the target scale pre-trained language model] can be performed by a human, using observation, evaluation, and judgment, because the steps involve making determinations and augmentations, which are mental tasks humans routinely perform in the course of producing and performing queries.
Claim 9 and 17 recite the similar limitations as claim 1. Thus, claims 9 and 17 are rejected due to the similar reasons set forth regarding claim 1.
A claim recites a mental process when the claim encompasses acts the person can perform using the mind or pen and paper [determining that a claim whose ‘steps can be performed in the human mind, or by a human using a pen and paper’ is directed to an unpatentable mental process]. This is true even if the claim recites, as they do here, that a generic computer component performs the acts.
As noted above, if a claim, under its broadest reasonable interpretation, covers performance in the mind but for the recitation of generic computer components, then it is still in the mental processes category unless the claim cannot practically be performed in the mind. Here, the examiner is not persuaded that the aforementioned steps in claims 1, 9 or 17 cannot practically be performed in the human minds, or using pen and paper, but for the generic computing device.
Step 2A. In accordance with Step 2A, prog two of the 2019 PEG:
With respect to Step 2A, prog two, the judicial exception is not integrated into a practical application. The additional elements are directed to at least one processor and/or at least one memory. However, these elements do not (1) improve the functioning of a computer or other technology; (2) are not applied with any particular machine (except for a generic computer); (3) do not effect a transformation of a particular article to a different state; and (4) are not applied in any meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. In other words, the aforementioned additional element (or combination of elements) recited in the claims do not integrate the judicial exception into a practical application.
In other words, the claimed processes fail to improve the functioning of either at least one processor and/or at least one memory. Rather, these additional elements merely link the underlying abstract idea (i.e., mental processes or using pen and paper) to a particular technological environment, i.e., search query processing. Thus, the claimed process uses conventional computers to automate tasks that would have otherwise been very labor intensive by a human searcher. Such claims are not patent eligible. See OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363 (Fed. Cir. 2015) (“relying on a computer to perform routine tasks more quickly or more accurately is insufficient to render a claim patent eligible”).
Since the analysis of Step 2A prong one and prong two results in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception.
Step 2B. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional limitations are directed to at least one processor and/or at least one memory, at a very high level of generality and without imposing meaningful limitations on the scope of the claim. Such general-purpose computing device, high-level, and nominal involvement of a computer or computer-based elements for carrying out the invention merely serves to tie the abstract idea to a particular technological environment, which is not enough to render the claims patent-eligible.
The additional elements are broadly applied to the abstract idea at a high level of generality and they operate in a well-understood, routine, and conventional manner. Courts have held computer-implemented processes not to be significantly more than an abstract idea (and thus ineligible) where the claim as a whole amount to nothing more than generic computer functions merely used to implement an abstract idea, such as an idea that could be done by a human analog.
The dependent claims have been fully considered, however, similar to the findings for claims above, these claims are similarly directed to the “Mental Processes” grouping of abstract ideas set forth in the 2019 PEG, without integrating it into a practical application and with, at most, a general-purpose computer that serves to tie the idea to a particular technological environment, which does not add significantly more to the claims. The ordered combination of elements in the dependent claims (including the limitations inherited from the parent claim(s)) add nothing that is not already present as when the elements are taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. The collective functions merely provide conventional computer implementation. Accordingly, the subject matter encompassed by the dependent claims fail to amount to significantly more than the abstract idea.
Conclusion
10. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MONICA M PYO whose telephone number is (571)272-8192. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, APU MOFIZ can be reached at 571-272-4080. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MONICA M PYO/ Primary Examiner, Art Unit 2161