DETAILED ACTION
This Office Action is in response to the amendments filed on 04/27/2026.
Claims 1, 15, and 19 are currently amended.
Claims 1-16, and 18-20 are currently pending in this application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
In reference to Applicant’s arguments on page(s) 9-13 regarding rejections made under 35 U.S.C. 101:
Claims 1-20 are rejected under 35 U.S.C. § 101 for allegedly being directed to a judicial exception (e.g., an abstract idea) without significantly more. More specifically, the claims allegedly recite a mathematical relationship/concept (i.e., generating numerical values and minimizing distances) and a mental process (i.e., the limitations contained therein can be performed in the mind or with pen and paper).
The Office Action characterizes the prior version of claim 1 as being directed to abstract concepts such as generating numerical values, minimizing distances, and predicting confidence scores. Applicant respectfully submits that as amended, claim 1 is directed to a specific computer-implemented process for managing a stored data structure, not to an abstract idea. The focus of the claim is therefore not the mathematical generation of embeddings or confidence scores in isolation, but rather a computer-centric technique for selectively compressing a large embedding matrix while preserving recoverability of deleted entries. Such subject matter is directed to a technological process that operates on a concrete data structure, not an abstract idea.
Even assuming arguendo that certain computational steps could be characterized as abstract, the amended claim integrates those steps into a practical application, consistent with USPTO guidance. Firstly, the confidence scores control a concrete machine operation. In the amended claim, confidence scores are not merely reported, evaluated, or displayed. Instead, they are used to control a specific system behavior-namely, whether corresponding entries in a stored embedding matrix are deleted from memory. This is a paradigmatic example of meaningful application, where mathematical outputs directly govern the operation of a computer system.
Secondly, the amendment improves computer storage and resource usage. Applicant's specification explains that large embedding matrices in industrial NLP systems may include tens of millions of entries and consume tens of gigabytes of storage. The amended claim implements a lossy compression mechanism that selectively removes entries that can be reliably regenerated, thereby: reducing memory requirements; enabling more efficient storage of embedding data; and balancing storage costs with computational regeneration. Applicant respectfully submits that improvements to data storage efficiency and memory management are recognized indicators of a practical application and fall squarely within the types of computer improvements deemed eligible under § 101. Thirdly, the claim operates within a closed, technical feedback loop.
Examiner’s response:
Applicant’s arguments have been fully considered but are found to be not persuasive.
Applicant argues that the instant application is not directed toward an abstract idea. Examiner disagrees. The independent claims recite actions of generating embeddings and minimizing distances between said embeddings, predicting confidence scores, comparing said confidence scores to a threshold value, compressing a matrix embedding by deleting entries according to the confidence score threshold, and keeping deleted values in memory so that they are recoverable, all of which can be reasonably performed in the human mind. The action of generating embeddings and minimizing distances between embeddings is a mathematical concept/calculation of optimization. The actions of predicting confidence scores, comparing said score to a threshold value, and removing matrix entries based on said comparison are mental processes of making judgements. The action of having recently deleted entries be recoverable is a mental process of recalling data.
Applicant argues that the amended claim integrates the steps into a practical application. Examiner disagrees. The use of confidence scores, identified above as a mental process of making a judgement, to determine what entries of a matrix to remove is not a practical application of the mathematical outputs because a judgement is still being cast on which matrix entries are considered to be under the threshold value.
Applicant argues that the amendment improves computer storage and resource usage. Examiner disagrees. While the storage and resource usage is lessened as a result of the actions amended in the claims, the improvement therein relies on the reduced amount of data to be processed. The reduction of data relies on the abstract ideas mentioned above, namely the prediction of confidence scores and the comparing of said scores to the threshold value and a technological improvement cannot arise from an abstract idea.
In light of the amendments made on the claims, the rejections made under 35 U.S.C. 101 are maintained and updated below.
In reference to Applicant’s arguments on page(s) 13-15 regarding rejections made under 35 U.S.C. 103:
Claims 1-3, 15-16, and 19-20 are rejected under 35 U.S.C. § 103 as being allegedly unpatentable over U.S. Pat. Pub. No. 2023/0022845 (hereinafter "Meng"), in view of U.S. Pat. No. 12,499,879 (hereinafter "Soliman").
Claims 6 and 9 are rejected under 35 U.S.C. § 103 as being allegedly unpatentable over Meng, in view of Soliman, and further in view of U.S. Pat. Pub. No. 2019/0370337 (hereinafter "Lee").
Specifically, Applicant traverses the rejection of the claims as amended for at least the reason that the prior art of record fails to disclose the limitations of amended independent claim 1, in so far as it now recites "...performing a lossy compression of the embedding matrix by deleting entries of the embedding matrix whose corresponding confidence scores satisfy the threshold, thereby reducing storage requirements of the embedding matrix, and wherein embeddings associated with deleted entries are regenerable on-the-fly using the trained model." As set forth in the Office Action dated January 26, 2026, Examiner applied the § 103 rejection to claims 1-3, 6, 9, 15-16, and 19-20, but did not apply that rejection to claims 4-5, 7-8, 10-14, and 17-18. Amended independent claim 1 now expressily incorporates the limitations of un-rejected claim 17 (now canceled), which relate to performing a lossy compression of an embedding marix based on inferred confidence scores. Independent claims 15 and 19 have also been amended to incorporate the limitations of un-rejected claim 17. Because claim 17 was not rejected under 35 U.S.C. § 103, the Office has already determined, based on the cited prior art, that the limitations now incorporated into claims 1, 15, and 19 were not shown or suggested by the applied combinations. Thus, the combination relied upon for the § 103 rejection of former claim 1 no longer accounts for all limitations of amended claim 1, and the § 103 rejection does not extend to the amended claim.
Therefore, at least for the above reasons, Applicant believes independent claims 1, 15, and 19 are patentable over Meng, in view of Soliman.
Accordingly, Applicant respectfully requests withdrawal of the rejection and reconsideration of the rejected claims.
The other claims are dependent from one of the independent claims (i.e., claims 1, 15, and 19) discussed above, and are therefore believed patentable for at least the same reasons. Specifically, rejected claims 2-14 depend from amended independent claim 1, rejected claims 16-18 depend from amended independent claim 15, and rejected claim 20 depends from amended independent claim 19. The independent claims being allowable, the rejection of the dependent claims is moot. Since each dependent claim is also deemed to define an additional aspect of the invention, however, the individual reconsideration of the patentability of each on its own merits is respectfully requested. Similarly, because Applicant maintains that all claims are allowable for at least the reasons presented hereinabove, in the interests of brevity, this response does not comment on each and every comment made by the Examiner in the Office Action. This should not be taken as acquiescence of the substance of those comments, and Applicant reserves the right to address such comments.
Examiner’s response:
Applicant’s arguments have been fully considered and are found to be persuasive.
Applicant has amended the independent claims to incorporate the subject matter of previously unrejected claim 17, which was flagged as allowable subject matter in the previous action. As the subject matter of Claim 17 was previously examined, no new matter is introduced that would require further search and consideration.
In light of the amendments made on the claims, the rejections made under 35 U.S.C. 103 are withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because they are directed toward an abstract idea without significantly more.
Step 1 analysis:
Independent Claims 1 and 15 recite, in part, a computer-implemented method, therefore falling into the statutory category of process. Independent Claim 19 recites, in part, a computer program product, therefore falling into the statutory category of manufacture.
Regarding Claim 1:
Step 2A: Prong 1 analysis:
Claim 1 recites in part:
“generating, with the term encoder, second embeddings from numerical representations of word subunits of the training terms with an objective of minimizing distances between the first embeddings and the second embeddings”. As drafted and under its broadest reasonable interpretation, this limitation covers a mathematical relationship/concept (generating numerical values and minimizing distances).
“predicting confidence scores based on the minimized distances”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgement, or opinion) or with the aid of pencil and paper. For example, this limitation encompasses predicting scores based on calculated values.
“comparing the corresponding confidence scores to a threshold”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgement, or opinion) or with the aid of pencil and paper. For example, this limitation encompasses comparing values.
“performing a lossy compression of the embedding matrix by deleting entries of the embedding matrix whose corresponding confidence scores satisfy the threshold, thereby reducing storage requirements of the embedding matrix”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgement, or opinion) or with the aid of pencil and paper. For example, this limitation encompasses deleting matrix entries according to a score comparison.
“wherein embeddings associated with deleted entries are regenerable on-the-fly using the trained model”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgement, or opinion) or with the aid of pencil and paper. For example, this limitation encompasses maintaining recently deleted values in memory so they can be regenerated.
Accordingly, at Step 2A: Prong 1, the claim is directed to an abstract idea.
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein, prior to the training, the training dataset is obtained as an embedding matrix”. This additional elements is recited at a high level of generality and amounts to extra-solution activity of gathering data i.e. pre-solution activity of gathering data for use in the claimed process.
“training the model on a training dataset that associates training terms with first embeddings of the training terms”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (machine learning model) (See MPEP 2106.05(f)).
“wherein the word subunits form part of a predetermined set of word subunits”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (word/semantic data) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
“deploying the model as part of an executable algorithm to allow a user to infer third embeddings and corresponding confidence scores from any input terms written based on word subunits of the predetermined set”. This additional element is recited at a high level of generality such that the claim recites only the idea of a solution or outcome (deploy a model) i.e., the claim fails to recite details of how a solution to a problem is accomplished.
“accessing, in memory, a set of terms corresponding to entries of the embedding matrix”. This additional elements is recited at a high level of generality and amounts to extra-solution activity of gathering data i.e. pre-solution activity of gathering data for use in the claimed process.
“executing the model on the set of terms to infer third embeddings and corresponding confidence scores for the set of terms”. This additional element is recited at a high level of generality such that the claim recites only the idea of a solution or outcome (inferring third embeddings) i.e., the claim fails to recite details of how a solution to a problem is accomplished.
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “wherein, prior to the training, the training dataset is obtained as an embedding matrix” and “accessing, in memory, a set of terms corresponding to entries of the embedding matrix” is/are recited at a high level of generality and amount(s) to extra-solution activity of receiving data i.e., pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
As discussed above, the additional element(s) of “training the model on a training dataset that associates training terms with first embeddings of the training terms” is/are recited at a high-level of generality such that it/they amount(s) to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
The additional element(s) of “wherein the word subunits form part of a predetermined set of word subunits” is/are directed to particular field(s) of use (word/semantic data) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
As discussed above, the additional element(s) of “deploying the model as part of an executable algorithm to allow a user to infer third embeddings and corresponding confidence scores from any input terms written based on word subunits of the predetermined set” and “executing the model on the set of terms to infer third embeddings and corresponding confidence scores for the set of terms” is/are recited at a high-level of generality such that the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 2:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the word subunits of the training terms are characters”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (word/semantic data) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
“wherein the model is trained to generate the second embeddings based on numerical representations of characters of the training terms”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (machine learning model) (See MPEP 2106.05(f)).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “wherein the word subunits of the training terms are characters” is/are directed to particular field(s) of use (word/semantic data) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
As discussed above, the additional element(s) of “wherein the model is trained to generate the second embeddings based on numerical representations of characters of the training terms s” is/are recited at a high-level of generality such that it/they amount(s) to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 3:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the training terms are captured as tokens, and wherein at least some of the tokens capture respective sets of multiple words, and wherein each of the sets of multiple words are tokenized into a respective single token”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (word/semantic data) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “wherein the training terms are captured as tokens, and wherein at least some of the tokens capture respective sets of multiple words, and wherein each of the sets of multiple words are tokenized into a respective single token” is/are directed to particular field(s) of use (word/semantic data) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 4:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“prior to training the model, obtaining the training dataset as an embedding matrix which maps the tokens to the first embeddings”. This additional elements is recited at a high level of generality and amounts to extra-solution activity of gathering data i.e. pre-solution activity of gathering data for use in the claimed process.
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “prior to training the model, obtaining the training dataset as an embedding matrix which maps the tokens to the first embeddings” is/are recited at a high level of generality and amount(s) to extra-solution activity of receiving data i.e., pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 5:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“prior to obtaining the embedding matrix, running a natural language preprocessing pipeline on text data of one or more text corpora to tokenize the text data using a sequence tagging model designed to identify named entities, so as to tokenize multiple words corresponding to the name entities into single tokens”. This additional elements is recited at a high level of generality and amounts to extra-solution activity of gathering data i.e. pre-solution activity of gathering data for use in the claimed process.
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “prior to obtaining the embedding matrix, running a natural language preprocessing pipeline on text data of one or more text corpora to tokenize the text data using a sequence tagging model designed to identify named entities, so as to tokenize multiple words corresponding to the name entities into single tokens” is/are recited at a high level of generality and amount(s) to extra-solution activity of receiving data i.e., pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 6:
Step 2A: Prong 1 analysis:
Claim 6 recites in part:
“identifying the word subunits of the training terms”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgement, or opinion) or with the aid of pencil and paper. For example, this limitation encompasses identifying characters of words in a training set.
Accordingly, at Step 2A: Prong 1, the claim is directed to an abstract idea.
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the term encoder includes a word subunit decomposition layer, a word subunit embedding layer, and one or more trainable layers, and wherein the word subunit decomposition layer is connected to the word subunit embedding layer, itself connected to the one or more trainable layers”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (machine learning model layers) (See MPEP 2106.05(f)).
“through the word subunit decomposition layer”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (machine learning model layers) (See MPEP 2106.05(f)).
“obtaining numerical representations of the identified word subunits through the word subunit embedding layer”. This additional elements is recited at a high level of generality and amounts to extra-solution activity of gathering data i.e. pre-solution activity of gathering data for use in the claimed process.
“training the one or more trainable layers to generate the second embeddings from the obtained numerical representations in accordance with an objective function defining the objective”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (objective functions) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
As discussed above, the additional element(s) of “wherein the term encoder includes a word subunit decomposition layer, a word subunit embedding layer, and one or more trainable layers, and wherein the word subunit decomposition layer is connected to the word subunit embedding layer, itself connected to the one or more trainable layers” and “through the word subunit decomposition layer” is/are recited at a high-level of generality such that it/they amount(s) to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
The additional element(s) of “obtaining numerical representations of the identified word subunits through the word subunit embedding layer” is/are recited at a high level of generality and amount(s) to extra-solution activity of receiving data i.e., pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
The additional element(s) of “training the one or more trainable layers to generate the second embeddings from the obtained numerical representations in accordance with an objective function defining the objective” is/are directed to particular field(s) of use (objective functions) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 7:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the one or more trainable layers include several layers that are configured as a multilayer perceptron”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (multilayer perceptron) (See MPEP 2106.05(f)).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
As discussed above, the additional element(s) of “wherein the one or more trainable layers include several layers that are configured as a multilayer perceptron” is/are recited at a high-level of generality such that it/they amount(s) to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 8:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the one or more trainable layers further include at least one long short-term memory layer interfacing the word subunit embedding layer with the multilayer perceptron”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (LSTM) (See MPEP 2106.05(f)).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
As discussed above, the additional element(s) of “wherein the one or more trainable layers further include at least one long short-term memory layer interfacing the word subunit embedding layer with the multilayer perceptron” is/are recited at a high-level of generality such that it/they amount(s) to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 9:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the model further includes an estimator connected by the one or more trainable layers, and wherein training the model further comprises training the estimator to predict the confidence scores”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (estimator) (See MPEP 2106.05(f)).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
As discussed above, the additional element(s) of “wherein the model further includes an estimator connected by the one or more trainable layers, and wherein training the model further comprises training the estimator to predict the confidence scores” is/are recited at a high-level of generality such that it/they amount(s) to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 10:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the objective function is a first objective function, and wherein the estimator is trained to predict the confidence scores in accordance with a second objective function, the second objective function defining an objective of minimizing a difference between the confidence scores predicted by the estimator and the first objective function as evaluated based on the minimized distances”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (objective functions) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “training the one or more trainable layers to generate the second embeddings from the obtained numerical representations in accordance with an objective function defining the objective” is/are directed to particular field(s) of use (objective functions) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 11:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the one or more trainable layers are trained to learn parameters of distributions, from which respective ones of the first embeddings are drawn and wherein the second embeddings and the corresponding confidence scores are obtained from the learned parameters of the distributions”. This additional elements is recited at a high level of generality such that the claim recites only the idea of a solution or outcome (train a model) i.e., the claim fails to recite details of how a solution to a problem is accomplished.
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
As discussed above, the additional element(s) of “wherein the one or more trainable layers are trained to learn parameters of distributions, from which respective ones of the first embeddings are drawn and wherein the second embeddings and the corresponding confidence scores are obtained from the learned parameters of the distributions” is/are recited at a high-level of generality such that the claim recites only the idea of a solution or outcome (train a model) i.e., the claim fails to recite details of how a solution to a problem is accomplished (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 12:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the parameters learned for each distribution of the distributions include a mean and a variance, the mean corresponding to a respective one of the second embeddings, while a corresponding one of the confidence scores is obtained based on a negative log-likelihood of the each distribution”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (probability distributions) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “wherein the parameters learned for each distribution of the distributions include a mean and a variance, the mean corresponding to a respective one of the second embeddings, while a corresponding one of the confidence scores is obtained based on a negative log-likelihood of the each distribution” is/are directed to particular field(s) of use (probability distributions) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 13:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the objective function is defined based on negative log-likelihoods of the distributions with respect to the first embeddings”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (objective functions) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “wherein the objective function is defined based on negative log-likelihoods of the distributions with respect to the first embeddings” is/are directed to particular field(s) of use (objective functions) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 14:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the objective function is designed to define a further objective, in addition to the objective of minimizing said distances, the further objective causing to push the second embeddings towards embeddings of semantically related training terms upon training the model”. This limitation merely indicates a field of use or technological environment in which the judicial exception is performed (objective functions) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “wherein the objective function is designed to define a further objective, in addition to the objective of minimizing said distances, the further objective causing to push the second embeddings towards embeddings of semantically related training terms upon training the model” is/are directed to particular field(s) of use (objective functions) (MPEP 2106.05(h)) and therefore do not provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 15:
Due to claim language similar to that of Claim 1, Claim 15 is rejected for the same reasons as presented above in the rejection of Claim 1.
Regarding Claim 16:
Step 2A: Prong 1 analysis:
Claim 16 recites in part:
“accepting or rejecting the third embeddings based on the corresponding confidence scores”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgement, or opinion) or with the aid of pencil and paper. For example, this limitation encompasses accepting or rejecting data based on a calculated score.
Accordingly, at Step 2A: Prong 1, the claim is directed to an abstract idea.
Step 2A: Prong 2 analysis:
The claim does not recite any additional elements that integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Regarding Claim 18:
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“wherein the objective function is designed to define a further objective, in addition to the objective of minimizing said distances, the further objective causing to push the second embeddings towards embeddings of semantically related training terms upon training the model”. This additional elements is recited at a high level of generality and amounts to extra-solution activity of gathering data i.e. pre-solution activity of gathering data for use in the claimed process.
“executing the model on the additional words to controllably update entries of an embedding matrix in accordance with confidence scores inferred for the embeddings generated for the additional words”. This additional elements is recited at a high level of generality such that the claim recites only the idea of a solution or outcome (train a model) i.e., the claim fails to recite details of how a solution to a problem is accomplished.
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
The additional element(s) of “wherein the objective function is designed to define a further objective, in addition to the objective of minimizing said distances, the further objective causing to push the second embeddings towards embeddings of semantically related training terms upon training the model” is/are recited at a high level of generality and amount(s) to extra-solution activity of receiving data i.e., pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
As discussed above, the additional element(s) of “executing the model on the additional words to controllably update entries of an embedding matrix in accordance with confidence scores inferred for the embeddings generated for the additional words” is/are recited at a high-level of generality such that the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 19:
Due to claim language similar to the of Claims 1 and 15, Claim 19 is rejected for the same reasons as presented above in the rejections of Claims 1 and 15, with the exception of the limitation(s) covered below.
Step 2A: Prong 2 analysis:
The judicial exception is not integrated into practical application. In particular, the claim recites the additional elements of:
“one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor capable of performing a method”. This additional element is recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (storage and processor) (See MPEP 2106.05(f)).
Accordingly at Step 2A: Prong 2, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
As discussed above, the additional element(s) of “one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor capable of performing a method” is/are recited at a high-level of generality such that it/they amount(s) to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Regarding Claim 20:
Due to claim language similar to that of Claim 2, Claim 20 is rejected for the same reasons as presented above in the rejection of Claim 2.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230022845 A1 – one or more machine learning models (e.g., a modified transformer) to predict a type of data that one or more numerical characters and/or one or more natural language word characters of a document (e.g., an invoice) correspond to
US 12499879 B1 – identifying functionalities (i.e., user experiences) that are requested by users but are not supported by natural understanding (NU) processing
US 20190370337 A1 – embedding of content of a natural language document
US 12494200 B1 – Techniques for performing an action with respect to displayed content
US 12164877 B2 – Techniques for interacting with users in a discussion environment
US 11568143 B2 – system and methods for fine-tuning a pre-trained Universal Language model (encoder) based on transformer architecture
US 20220300711 A1 – A system and method for natural language processing for document sequences
US 20220245348 A1 – self-supervised semantic shift detection and alignment
US 11393456 B1 – A system is provided for a self-learning policy engine that can be used by various spoken language understanding (SLU) processing components
US 20220020355 A1 – A method and apparatus for generating speech through neural text-to-speech (TTS) synthesis
US 20200311115 A1 – systems and methods for mapping of text phrases to a taxonomy
US 20200175360 A1 – Methods, systems and computer program products for updating a word embedding model
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to COREY M SACKALOSKY whose telephone number is (703)756-1590. The examiner can normally be reached M-F 7:30am-3:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COREY M SACKALOSKY/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights