Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Regarding U.S. Provisional Patent Application No. 18/048,197 (filed 10/20/2022) Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) is acknowledged.
Information Disclosure Statement
The information disclosure statement submitted on 4/26/2023 has been considered.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: 309a, 309d, 407, 410, 1007, 1008, 1105.
Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 15-18, 22-25, 30, 34-35, 40, and 44-45 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Regarding claim 15, line 10, the term “near to” is a relative term which renders the claim indefinite. The term “near to” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if a “input candidate” is “near to” a “heterogeneous cluster” or a “singleton” in the fine tuned vector space. The examiner suggests amending this to recite “nearest to” or to introduce the concept of a threshold for determining if an “input candidate” is sufficiently “near to” a “heterogeneous cluster” or a “singleton” in the fine tuned vector space.
Claims 16-18 depend from claim 15, do not remedy the deficiencies of claim 15, and are therefore rejected for the same reasons explained with respect to claim 15.
Regarding claim 22, line 3, the term “low confidence score” is a relative term which renders the claim indefinite. The term “low confidence score” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if an “input candidate” has a “low confidence score”. The examiner suggests amending this to recite “lowest confidence score” or to introduce the concept of a threshold for determining if an “input candidate” has a sufficiently low confidence score.
Regarding claim 23, line 3, the term “low confidence score” is a relative term which renders the claim indefinite. The term “low confidence score” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if an “input candidate” has a “low confidence score”. The examiner suggests amending this to recite “lowest confidence score” or to introduce the concept of a threshold for determining if an “input candidate” has a sufficiently low confidence score.
Claims 24 depends from claim 22, does not remedy the deficiencies of claim 22, and is therefore rejected for the same reasons explained with respect to claim 22.
Claims 25 depends from claim 23, does not remedy the deficiencies of claim 23, and is therefore rejected for the same reasons explained with respect to claim 23.
Regarding claim 30, line 10, the term “near to” is a relative term which renders the claim indefinite. The term “near to” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if a “input candidate” is “near to” a “heterogeneous cluster” or a “singleton” in the fine tuned vector space. The examiner suggests amending this to recite “nearest to” or to introduce the concept of a threshold for determining if an “input candidate” is sufficiently “near to” a “heterogeneous cluster” or a “singleton” in the fine tuned vector space.
Regarding claim 34, line 3, the term “low confidence score” is a relative term which renders the claim indefinite. The term “low confidence score” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if an “input candidate” has a “low confidence score”. The examiner suggests amending this to recite “lowest confidence score” or to introduce the concept of a threshold for determining if an “input candidate” has a sufficiently low confidence score.
Regarding claim 35, line 3, the term “low confidence score” is a relative term which renders the claim indefinite. The term “low confidence score” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if an “input candidate” has a “low confidence score”. The examiner suggests amending this to recite “lowest confidence score” or to introduce the concept of a threshold for determining if an “input candidate” has a sufficiently low confidence score.
Regarding claim 40, line 10, the term “near to” is a relative term which renders the claim indefinite. The term “near to” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if a “input candidate” is “near to” a “heterogeneous cluster” or a “singleton” in the fine tuned vector space. The examiner suggests amending this to recite “nearest to” or to introduce the concept of a threshold for determining if an “input candidate” is sufficiently “near to” a “heterogeneous cluster” or a “singleton” in the fine tuned vector space.
Regarding claim 44, line 4, the term “low confidence score” is a relative term which renders the claim indefinite. The term “low confidence score” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if an “input candidate” has a “low confidence score”. The examiner suggests amending this to recite “lowest confidence score” or to introduce the concept of a threshold for determining if an “input candidate” has a sufficiently low confidence score.
Regarding claim 45, line 4, the term “low confidence score” is a relative term which renders the claim indefinite. The term “low confidence score” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It would be unclear to one of ordinary skill in the art how to determine if an “input candidate” has a “low confidence score”. The examiner suggests amending this to recite “lowest confidence score” or to introduce the concept of a threshold for determining if an “input candidate” has a sufficiently low confidence score.
Claim Rejections - 35 USC § 101
Claims 1-45 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Step 1 of the Alice/Mayo framework, Claims 1-25 are directed to a method (a process), Claims 26-35 are directed to a system (a machine), and Claims 36-45 are directed to a non-transitory computer-readable medium (an article of manufacture), which each fall within one of the four statutory categories of inventions.
Regarding Claim 1
Step 2A, prong 1 (Is the claim directed to a law of nature, a natural phenomenon or an abstract idea).
Claim 1 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “pretrained model”).
selecting a first plurality of input candidates from a corpus of data; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally select a plurality of words from a corpus of data, such as an encyclopedia)
mapping the first plurality of input candidates onto a pretrained vector space of a pretrained model; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pretrained vector space of a pretrained model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the pretrained vector space)
clustering the first plurality of input candidates in the pretrained vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pretrained vector space of a pretrained model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the pretrained vector space, and then draw circles around clusters of data points that are close to each other)
adding the first plurality of input candidates to a plurality of queues for labelling; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally add the selected words to a plurality of queues for labeling, where such queues can be mental lists of words for labeling)
labelling the first plurality of input candidates. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label words (the input candidates))
Step 2A, prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?).
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements (e.g., “pretrained model”) which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding the “pretrained model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a pretrained model. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a generic pretrained model). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?)
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements (e.g., “pretrained model”) are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding the “pretrained model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 2
Step 2A, Prong 1
wherein labelling the first plurality of input candidates is performed by humans. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label words with mentally-derived labels)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 3
Step 2A, Prong 1
wherein labelling the first plurality of input candidates is performed algorithmically. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label words using an algorithm to determine the label, e.g., if the word starts with a letter from a-m, label it with a “0” and if it starts with a letter from n-z, label it with a “1”)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 4
Step 2A, Prong 1
wherein labelling comprises identifying cluster centroids in the pretrained vector space. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally identify cluster centroids, such as by mentally selecting a data point of the cluster as a centroid, where such clusters are drawn around points on a x-y axis that represents a 2-dimensional pretrained vector space)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 5
Step 2A, Prong 1
wherein the pretrained vector space is created by mapping input to sparse/dense distributed representations. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map input words to a sparse/dense distributed representations that have few zero values)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 6
Step 2A, Prong 1
wherein the pretrained vector space comprises learned parameters of a probability distribution. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally (or using pencil and paper) calculate a probability distribution and map such probability distribution to the pretrained vector space)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 7
Step 2A, Prong 2
Regarding the “the pretrained vector space is learned by performing density estimation” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result (e.g., any manner of training a model to learn a vector space that uses density estimation in any manner). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “the pretrained vector space is learned by performing density estimation” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 8
Step 2A, Prong 2
Regarding the “wherein the pretrained model is selected from a group consisting of transformers, convolutional neural networks, recurrent neural networks, graph neural networks, and combinations thereof” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (specific types of architectures for the pretrained model. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate a judicial exception into a practical application. Moreover, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of specific types of architectures for the pretrained model. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (specific types of architectures for the pretrained model). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the pretrained model is selected from a group consisting of transformers, convolutional neural networks, recurrent neural networks, graph neural networks, and combinations thereof” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use as explained above, which does not amount to significantly more than the judicial exception. MPEP 2106.05(h). Moreover, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 9
Step 2A, Prong 1
further comprising partitioning the labeled first plurality of input candidates into a train set, a development set, a test set, and an out-of-distribution set, wherein partitioning comprises: (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally partition input candidates (such as 4 different words), into each of the recited sets (1 in each))
adding labeled cluster centroids in the pretrained vector space from the first plurality of input candidates to the train set; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally (or using pencil and paper), add a word corresponding to a cluster centroid to the train set as recited by this limitation)
adding labeled cluster children in the pretrained vector space from the first plurality of input candidates to one of the development set and the test set; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally(or using pencil and paper), add a word corresponding to a cluster child to one of the development set and the test set as recited by this limitation)
adding labeled singletons in the pretrained vector space from the first plurality of input candidates to one of the train set and the out-of-distribution set. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally(or with pencil and paper), for example, a human can mentally(or using pencil and paper), add a word corresponding to a singleton to one of the train set and the out-of-distribution set as recited by this limitation)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 10
Step 2A, Prong 2
Regarding the “further comprising creating a fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result (e.g., any technique for fine tuning a model is covered). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “further comprising creating a fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 11
Step 2A, Prong 2
Regarding the “wherein creating the fine tuned model comprises using the pretrained model to create the fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result (e.g., any technique for fine tuning a pre-trained model is covered). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein creating the fine tuned model comprises using the pretrained model to create the fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 12
Step 2A, Prong 2
Regarding the “further comprising assigning a first plurality of outputs using the fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a fine tuned model. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a fine tuned model). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “further comprising assigning a first plurality of outputs using the fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 13
Step 2A, Prong 2
Regarding the “wherein the fine tuned model is selected from a group consisting of transformers, convolutional neural networks, recurrent neural networks, graph neural networks, and combinations thereof” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (specific types of architectures for the pretrained model. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate a judicial exception into a practical application. Moreover, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of specific types of architectures for the pretrained model. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (specific types of architectures for the pretrained model). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the fine tuned model is selected from a group consisting of transformers, convolutional neural networks, recurrent neural networks, graph neural networks, and combinations thereof” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use as explained above, which does not amount to significantly more than the judicial exception. MPEP 2106.05(h). Moreover, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 14
Step 2A, Prong 1
further comprising evaluating performance of the fine tuned model on the test set, wherein evaluating the performance of the fine tuned model comprises: (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally evaluate the performance of the fine tuned model on the test set, such as mentally evaluating whether the performance is good or bad in the human’s opinion)
mapping the test set onto a fine tuned vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the fine tuned vector space)
clustering the test set in the fine tuned vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the fine tuned vector space, and then draw circles around clusters of data points that are close to each other)
quantifying heterogeneity of test set clusters in the fine tuned vector space; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally review clusters in the test set as mapped and graphed onto a x-y axis, and mentally determine the distance from the centroid to the farthest point in the cluster to quantify the difference between the centroid and farthest point)
providing a confidence score for the fine tuned model. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally evaluate the performance of the fine tuned model and mentally generate a confidence score (e.g., from 1 to 10) about how confident the human is in its analysis of the performance of the model)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 15
Step 2A, Prong 1
further comprising labelling a second plurality of input candidates from the corpus of data, wherein labelling the second plurality of input candidates comprises: (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label a second set of words from the same corpus)
mapping the train set and development set onto the pretrained vector space and the fine tuned vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pretrained vector space of a pretrained model and a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the pretrained vector space and fine tuned vector space, respectively)
clustering the train set and development set in the pretrained vector space and the fine tuned vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pretrained vector space of a pretrained model and a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the pretrained vector space and fine tuned vector space, respectively, and then draw circles around clusters of data points that are close to each other)
identifying heterogeneous clusters and singletons in the fine tuned vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally identify clusters that have diversity (corresponding to “heterogeneous clusters”) and clusters that only have a single data point)
selecting the second plurality of input candidates such that the second plurality of input candidates are near to at least one of the heterogeneous clusters and singletons in the fine tuned vector space; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally review heterogeneous clusters and singletons in the fine tuned vector space and select points that are near to such clusters and/or singletons)
labelling the second plurality of input candidates. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label the second plurality of input candidates)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 16
Step 2A, Prong 1
further comprising partitioning the labeled second plurality of input candidates into the train set, the development set, the test set, and the out-of-distribution set, wherein partitioning comprises: (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally partition input candidates (such as 4 different words), into each of the recited sets (1 in each))
adding labeled cluster centroids from the second plurality of input candidates to the train set; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally (or using pencil and paper), add a word corresponding to a cluster centroid to the train set as recited by this limitation)
adding labeled cluster children from the second plurality of input candidates to one of the development set and the test set; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally(or using pencil and paper), add a word corresponding to a cluster child to one of the development set and the test set as recited by this limitation)
adding labeled singletons from the second plurality of input candidates to the one of the train set and the out-of-distribution set. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally(or with pencil and paper), for example, a human can mentally(or using pencil and paper), add a word corresponding to a singleton to one of the train set and the out-of-distribution set as recited by this limitation)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 17
Step 2A, Prong 1
wherein labelling of the second plurality of input candidates comprises algorithmically labelling the second plurality of input candidates. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label words using an algorithm to determine the label, e.g., if the word starts with a letter from a-m, label it with a “0” and if it starts with a letter from n-z, label it with a “1”)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 18
Step 2A, Prong 1
further comprising assigning the confidence score for the labelling of the second plurality of input candidates using a bipartite graph of the pretrained vector space and the fine tuned vector space. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally (or with pencil and paper), for example, a human can mentally (or using pencil and paper), draw a bipartite graph of the pretrained vector space and the fine tuned vector space, and utilize such graph when determining a confidence score to assign)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 19
Step 2A, Prong 1
evaluating performance of an ensemble of two or more fine tuned models on the test set, wherein evaluating the performance of the ensemble of two or more fine tuned models comprises determining whether the ensemble of two or more fine tuned models concur on an output; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally determine if the ensemble of two or more fine tuned models concur on an output, and if so, evaluate the performance as being good)
mapping the train, development, and test sets onto one or more pairs of pretrained vector spaces and fine tuned vector spaces; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pair of a pretrained vector space of a pretrained model and a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on x-y axes corresponding to the basis of the pair of the pretrained model vector space and the fine tuned vector space, respectively)
assigning a confidence score for each of the two or more fine tuned models using a bipartite graph for each of the one or more pairs of pretrained vector spaces and fine tuned vector spaces. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally (or with pencil and paper), for example, a human can mentally (or using pencil and paper), draw bipartite graphs for the pairs of the pretrained vector space and the fine tuned vector space, and utilize such graphs when determining a confidence score to assign)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 20
Step 2A, Prong 1
selecting a third plurality of input candidates from the corpus of data; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally select a third plurality of words from a corpus of data, such as an encyclopedia)
labeling the third plurality of input candidates ... ; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label words (the input candidates))
mapping the third plurality of input candidates onto the pretrained vector space and a fine tuned vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pretrained vector space of a pretrained model and a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the pretrained vector space and fine tuned vector space, respectively)
clustering the third plurality of input candidates in the pretrained vector space and the fine tuned vector space; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pretrained vector space of a pretrained model and a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on an x-y axis corresponding to the basis of the pretrained vector space and fine tuned vector space, respectively, and then draw circles around clusters of data points that are close to each other)
identifying heterogeneous clusters and singletons in the fine tuned vector space; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally identify clusters that have diversity (corresponding to “heterogeneous clusters”) and clusters that only have a single data point)
assigning a confidence score for the labelling of the third plurality of input candidates using a bipartite graph of the pretrained vector space and the fine tuned vector space. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally (or with pencil and paper), for example, a human can mentally (or using pencil and paper), draw a bipartite graph of the pretrained vector space and the fine tuned vector space, and utilize such graph when determining a confidence score to assign)
Step 2A, Prong 2
Regarding the “using the fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a fine tuned model. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a generic fine tuned model). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “using the fine tuned model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 21
Step 2A, Prong 1
labeling a third plurality of input candidates ... ; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally label a third plurality of input candidates)
determining whether the ensemble of two or more fine tuned models concur on labeling of the third plurality of input candidates; (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally determine if the ensemble of two or more fine tuned models concur on labeling of the third plurality of words)
mapping the third plurality of input candidates onto one or more pairs of pretrained vector spaces and fine tuned vector spaces; and (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally map words onto a pair of a pretrained vector space of a pretrained model and a fine tuned vector space of a fine tuned model, such as if the vector space is 2-dimensional, and mapping words involves plotting such words on x-y axes corresponding to the basis of the pair of the pretrained model vector space and the fine tuned vector space, respectively)
assigning a confidence score for each of the two or more fine tuned models using a bipartite graph for each of the one or more pairs of pretrained vector spaces and fine tuned vector spaces. (under the broadest reasonable interpretation, this limitation can be performed by a human mentally (or with pencil and paper), for example, a human can mentally (or with pencil and paper), for example, a human can mentally (or using pencil and paper), draw bipartite graphs for the pairs of the pretrained vector space and the fine tuned vector space, and utilize such graphs when determining a confidence score to assign)
Step 2A, Prong 2
Regarding the “using an ensemble of two or more fine tuned models on the third plurality of input candidates” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of an ensemble of two or more fine tuned models. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (generic fine tuned models in an ensemble). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “using an ensemble of two or more fine tuned models on the third plurality of input candidates” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 22
Step 2A, Prong 1
further comprising selecting a plurality of failed inputs for examination, wherein the plurality of failed inputs are inputs of the third plurality of inputs candidates that have