DETAILED ACTION
Notice of Pre-AIA or AIA Status
01. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
02. The information disclosure statement (IDS) filed on 10/02/2025 has been considered by the examiner and made of record in the application file.
Priority
03. Applicant’s claim for domestic priority under 35 U.S.C. 119(e) is acknowledged.
Drawings
04. The drawings were received on 01/31/2025. These drawings are accepted.
Claim Rejections – 35 USC § 112
05. The following is a quotation of the second paragraph of 35 U.S.C. 112(b):
(b) CONCLUSION. – The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of the second paragraph of pre-AIA 35 U.S.C. 112:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the application regards as his invention.
06. Claims 3 and 16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 3 recites the limitation “retraining the machine-learning model”. There is insufficient antecedent basis for this limitation in the claim, as the machine-learning model has not yet been described as having been trained, so it could not be “retrained”. Examiner believes that claim 3 was either intended to depend from claim 2 (which recites training the machine-learning model) or be recited has “training” instead of “retraining”. The same analysis applies to claim 16. Appropriate correction is required.
Claim Rejections - 35 USC § 101
07. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
08. Claims 1, 4, 5 – 14, 17, and 18 are rejected under 35 U.S.C. 101 because the claims are directed to an abstract idea without significantly more. The claims are directed to determining a quality metric for a model, which amounts to an abstract idea, as explained in detail below. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional computer elements, which are recited at a high level of generality, provide conventional computer functions that do not add meaningful limits to practicing the abstract idea.
Step 1: Claim 1 recites a system comprising a combination of devices that receive and generate data. Thus, the claim is directed to a machine, which is one of the statutory categories of invention.
Step 2A, prong one: Claim 1 recites the limitation of “determining… a corresponding quality metric for the machine learning model”. Although the claim recites some hardware (processor and memory), nothing in the claim elements precludes the step from practically being performed in the human mind. For example, the “determining” step in the claim encompasses an observation, evaluation, judgment, or opinion, in that a person could give a score (i.e. quality metric) to a system or model. The claim does not explain what the score represents or how it is calculated, just that some quality metric is determined, which is commonplace for people to do by making judgments in their mind about things.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claims recites an abstract idea.
Step 2A, prong two: The judicial exception is not integrated into a practical application. In particular, the claim includes the additional limitations of: “receiving first training data”, “receiving one or more challenge queries”, “generating…a plurality of associated training vectors”, and “generating…a plurality of associate challenge vectors”, as discussed below. The claim also recites the additional components of a processor and memory. However, they are recited so generically (no details whatsoever are provided other than that they are a processor and memory) that they represent no more than mere instructions to apply the judicial exception on a computer. These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer.
The claim recites the additional elements of “receiving first training data” and “receiving one or more challenge queries”. However, these steps/elements represent mere data gathering that is necessary for use of the recited judicial exception and is recited at a high level of generality. These steps merely recite receiving different types of data, and do not perform any meaningful transformation to the claims or the identified abstract idea. These elements are also insignificant extra-solution activity because it merely gathers data for use in making the determination for a quality metric.
The claim recites the additional elements of “generating…a plurality of associated training vectors that embed…”, and “generating…a plurality of associated challenge vectors that embed…”. However, these steps/elements represent mere data gathering that is necessary for use of the recited judicial exception and is recited at a high level of generality. These steps merely recite generating data for vectors, which in this instance is some type of data, such as numbers or objects. This does not perform any meaningful transformation to the claims or the identified abstract idea, as these elements merely gather/generate data for use in making the determination for a quality metric.
Even when viewed in combination, the additional elements in this claim do no more than perform the process on generic computing components. This does not provide an improvement to the computers and other technology that are recited in the claim. Thus this claim cannon improve computer functionality or other technology.
Step 2B: As discussed previously with respect to Step 2A prong two, the controller in the claim amounts to no more than mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B, i.e., mere instructions to apply an exception using a generic computer component cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B. The claims do not include additional elements that are sufficient to amount to significantly more than the abstract idea, but are instead limited to appending well-understood, routine, and conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (abstract idea).
In this instance, the claims include the determining of a neighborhood density for the received query. However, the neighborhood density is not expressly defined as being used in the quality metric determination step, or at least how it would be used, as the claim merely explains that a neighborhood density is determined, and then that the quality metric is determined. Furthermore, there is no recitation as to what a neighborhood density is exactly or how it would be useful for a quality metric, and appears to be something that would be well-understood, routine, and conventional activity, and which can be performed by generic computing components.
The same analysis is applied to dependent claims 4, 5, and 7 – 18, because the limitations recite additional mental processes and/or mathematical calculations and do not integrate into a practical application. Further, they do not include additional elements that amount to significantly more.
Claims 2, 3, 15, and 16 are not rejected under 35 USC 101, as they positively recite training the machine-learning model. The training of a machine learning model is understood to be an improvement to the functioning of computers, and is considered to go beyond the above identified abstract idea.
Claims 4 and 5 recite the additional element of “selecting the machine-learning model”. However, the mere making of a selection does not add anything significantly more to the claims. There is no change/improvement to the system as a result of making a selection.
Claim 6 recites that the training data “has been used” to train the machine-learning model. Although similar to claims 2 and 3, this claim limitation does not appear to actually recite that the machine-learning model is subsequently trained based on some other actions or calculations taken. Since the claim merely recites that the model had been trained, i.e. before the functionality recited in the claim, it does not add anything significantly more to the claims.
Claim 7 recites the additional element of the machine-learning model being a large language model. However, just defines the machine-learning model as being a specific type, and does not anything significantly more to the abstract idea.
Claim 8 recites the additional element of the training data being either language, image, or video data. However, this just defines the type of data that is used, and the claims do not even positively recite the training of the machine-learning model. This limitation does not add anything significantly more to the claims.
Claim 9 recites that the challenge queries have a format. However, all queries are known to have some type of format, and this does not add anything significantly more to the claims.
Claim 10 recites the additional element of an embedding function. This appears to be some type of mathematical calculation, and therefore appears to recite an abstract idea (mathematical concept). This does not integrate the claim into a practical application or include significantly more for the abstract idea.
Claim 11 recites that a subsample of training data is embedded. However, this is merely indicating the technological environment in which the judicial exception is applied to, and does not amount to significantly more than the abstract idea.
Claim 12 recites the additional limitation of the neighborhood density being a count of a number of training vectors within a threshold distance. This appears to be some type of mathematical calculation, and therefore appears to recite an abstract idea (mathematical concept). This does not integrate the claim into a practical application or include significantly more for the abstract idea.
Claim 13 recites the additional limitation of the neighborhood density being an average distance to nearest training vectors. This appears to be some type of mathematical calculation, and therefore appears to recite an abstract idea (mathematical concept). This does not integrate the claim into a practical application or include significantly more for the abstract idea.
Claims 14, 17, and 18 recite the same claim limitations as claims 1, 4, and 5, and are rejected under the same rational provided above.
Claim Rejections - 35 USC § 102
09. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
11. Claims 1 – 6 and 8 – 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Osuala et al. (US PGPub 2024/0111794), hereinafter “Osuala”.
Consider claim 1, Osuala discloses a system for objective characterization of machine-learning models (abstract, paragraph [0040], machine learning models are used to perform different functions), the system comprising:
one or more processors; and computer memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: (paragraphs [0133], [0135], [0138], the system comprises various computing hardware, including processors and memories, which are used to perform the functionality of the system);
receiving first training data formatted to be used in the training of a machine-learning model (paragraphs [0028], [0031], [0032], a dataset of training records are obtained that are used to train models, such as a machine learning model);
receiving one or more challenge queries formatted to be run on the machine-learning model (paragraphs [0026], [0033], [0035], queries are submitted to the system that are executed through the machine learning model);
generating, for the first training data, a plurality of associated training vectors that embed at least some of the first training data into a vector space (paragraphs [0027], [0028], [0032], [0116], the training process includes a process to embed vectors into a vector space, which are based on the input data for the machine learning model);
generating, for each of the one or more challenge queries, a plurality of associated challenge vectors that embed at least some of the challenge queries into the vector space (paragraphs [0028] – [0031], an embedding generation model allows for the input query to be embedded into a vector space, which includes vectors for the associated queries);
determining, for each challenge query, a corresponding quality metric for the machine-learning model by determining a neighborhood density for each of the challenge queries in the vector space (paragraphs [0038], [0096], [0111], a similarity score is determined that is based on the embeddings obtained by a k-nearest neighbor search).
Consider claim 2, and as applied to claim 1 above, Osuala discloses a system comprising:
responsive to determining, for each challenge query, a corresponding quality
metric for the machine-learning model, creating the machine-learning model comprising training the machine-learning model using the first training data (paragraphs [0031] – [0034], the machine learning model is trained by way of using training data).
Consider claim 3, and as applied to claim 1 above, Osuala discloses a system comprising:
responsive to determining, for each challenge query, a corresponding quality metric for the machine-learning model, retraining the machine-learning model using second training data that comprises at least some of the first training data and at least some of the challenge queries (paragraphs [0028], [0031], a machine learning model can be pre-trained, and then trained again based on additional inputs).
Consider claim 4, and as applied to claim 1 above, Osuala discloses a system comprising:
responsive to determining, for each challenge query, a corresponding quality metric for the machine-learning model, selecting the machine-learning model for use in processing at least one of the challenge queries (paragraphs [0038], [0105], the similarity scores are used to determine which embeddings for the machine learning model are to be used).
Consider claim 5, and as applied to claim 1 above, Osuala discloses a system comprising:
responsive to determining, for each challenge query, a corresponding quality metric for the machine-learning model, selecting the machine-learning model for use in processing other queries similar to at least one of the challenge queries (paragraphs [0038], [0045], [0053], the similarity scores are used to determine which embeddings for the machine learning model are to be used, such that future queries use a particular selected model).
Consider claim 6, and as applied to claim 1 above, Osuala discloses a system comprising:
first training data has been used to train the machine-learning model (paragraph [0032], the training datasets are used to train the machine learning model).
Consider claim 8, and as applied to claim 1 above, Osuala discloses a system comprising:
the first training data comprises data in a first format selected from the group consisting of i) natural language strings, ii) image data, and iii) video data (paragraph [0026], the training data can be image data, sound data, and/or text data).
Consider claim 9, and as applied to claim 8 above, Osuala discloses a system comprising:
the challenge queries are in the first format (paragraphs [0026], [0029], a query is in a certain format, such as containing spoken language or input words).
Consider claim 10, and as applied to claim 1 above, Osuala discloses a system comprising:
generating, for the first training data, the plurality of associated training vectors that embed at least some of the first training data into a vector space comprises using a first embedding function (paragraphs [0032], [0048], [0054], a function is used in order to embed the data into the vectors);
generating, for each of the one or more challenge queries, a plurality of challenge vectors that embed at least some of the challenge queries into the vector space comprises using the first embedding function (paragraphs [0032], [0048], [0054], a function is used in order to embed the data into the vectors).
Consider claim 11, and as applied to claim 1 above, Osuala discloses a system comprising:
the plurality of associated training vectors that embed at least some of the first training data into the vector space embed a statistically representative subsample of the first training data into the vector space (paragraphs [0029], [0033], [0036], a subset of the data is used in order to embed data into the vector space).
Consider claim 12, and as applied to claim 1 above, Osuala discloses a system comprising:
determining the neighborhood density for each of the challenge queries in the vector space comprises determining a count of a number of training vectors within a threshold distance of each of the challenge vectors in the vector space (paragraphs [0038], [0041], [0062], [0096], a distance function is used to determine the amount of neighbors that impact the vector space).
Consider claim 13, and as applied to claim 1 above, Osuala discloses a system comprising:
determining the neighborhood density for each of the challenge queries in the vector space comprises finding an average distance to N nearest training vectors in the vector space (paragraphs [0038], [0041], [0062], [0096], a distance function is used to determine the neighbors, which determines an average distance in the vector space).
Claims 14 – 18 recite the same embodiments as those found in claims 1 – 5, except that either a system or method is claimed. Since the same claim limitations are otherwise present, claims 14 – 18 are rejected under the same rationale provided above with respect to claims 1 – 5.
Claim Rejections - 35 USC § 103
12. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
13. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
14. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Osuala et al. (US PGPub 2024/0111794), hereinafter “Osuala”, in view of Ordorica de la Torre et al. (US PGPub 2025/0139353), hereinafter “Ordorica”
Consider claim 7, and as applied to claim 1 above, Osuala discloses various types of machine learning models, such as deep neural networks (paragraph [0172]). However, Osuala does not explicitly teach a large language model.
In the same field of endeavor, Ordorica discloses a system comprising:
the machine-learning model is a large language model (paragraphs [0027], [0077], a large language model is used to provide for processing of queries through vector space embeddings).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the large language model taught by Ordorica into the querying of machine learning models taught by Osuala for the purpose of allowing different types of models to be able to take advantage of the ability to embed queries into vector space so that more complex operations could be performed to obtain additional data about the machine learning model.
Relevant Prior Art Directed to State of Art
15. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
a) Neelakantan et al. (US PGPub 2024/0249186) discloses a method of querying with natural language input in order to access an embedding space for storing vector representations generated by a machine learn model that has been trained.
Conclusion
16. Any inquiry concerning this communication or earlier communications from the Examiner should be directed to Christopher Raab whose telephone number is (571) 270-1090. The Examiner can normally be reached on Monday-Friday from 9:00am to 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, Ajay Bhatia can be reached on (571) 272-3906. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free) or 703-305-3028.
/CHRISTOPHER J RAAB/Primary Examiner, Art Unit 2156
February 06, 2026