Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-8 and 10-15 are pending for examination. Claims 1 and 15 are independent.
Response to Amendment
The office action is responsive to the amendments filed on 07/24/2026. As
directed by the amendments claims 1, 4-5, 13 and 15 are amended. Claim 9 is canceled.
Response to Arguments
Applicant's arguments filed 07/24/2026 have been fully considered but they are not fully persuasive.
Applicant arguments regarding 35 U.S.C. § 101:
Examiner response: Examiner respectfully disagrees, training of a ML model was not addressed as a mental step under step 2A Prong 1. Instead, it was addressed in step 2A Prong 2 & 2B as “apply it” (or an equivalent) with the judicial exception - see MPEP 2106.05(f). Claim 1 describes training as “a first/second ML model component trained by applying the first/second subset”. Example 47 from the “July 2024 Subject Matter Eligibility Examples” also describes generally training and applying a neural network. The feature selection process described in the claims for selecting subsets is practically performable in the human mind and is understood to be a recitation of a mental process. Measuring classification performance is also practically performable in the human mind and is understood to be a recitation of a mental process.
Examiner respectfully disagrees, under broadest reasonable interpretation claim 1 does not incorporate the features described in the specification for “using a quantum annealer-based process to solve a QUBO (quadratic unconstrained binary optimization) formulation”. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. MPEP 2106.05(a) also indicates not only does the specification have to disclose the improvement for a claim not to be an abstract idea, the claim elements themselves have to provide limitations that reflect the improvement. Specifying the details for “using a quantum annealer-based process to solve a QUBO (quadratic unconstrained binary optimization) formulation” into claim 1 can potentially overcome the 101 rejection.
MPEP §2106.05(a) reads:
During examination, the examiner should analyze the "improvements" consideration by evaluating the specification and the claims to ensure that a technical explanation of the asserted improvement is present in the specification, and that the claim reflects the asserted improvement.
However, the MPEP (§2106.05(a)(II)) also warns, “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.” Here, the alleged improvement in the form of a feature-selection processes an improvement to the abstract idea of selecting data sets to apply to models.
Claim 1 describes applying data subsets to a first and second machine learning model. The first and second ML models are the ensemble architecture which is describing applying the feature selected data to ML models. This is understood as “apply it” (or an equivalent) with the judicial exception - see MPEP 2106.05(f).
Desjardins describes a completely different application and improvement from appellant’s application when describing training a machine learning model to learn new tasks while protecting knowledge about previous tasks to overcome the problem of "catastrophic forgetting". Instead, applicants claims describe a feature selection process to apply to machine learning models. Applicants claimed improvement is describing an improvement to an abstract idea, MPEP 2106.05(a) says an improvement in the abstract idea itself is not an improvement in technology. Overall, the claims are a combination of mental steps under step 2A Prong 1, and additional elements under steps 2A Prong 2 & 2B as detailed in the 101 rejection below.
Applicant arguments regarding 35 U.S.C. § 102/103:
Examiner response: Applicant’s arguments with respect to claim(s) have been considered but are moot because the new ground of rejection.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8, and 10-15 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
According to the first part of the analysis, in the instant case, claims 1-8 and 10-14 are directed to a method, and claim 15 is directed to a system. Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Regarding Claim 1:
2A Prong 1:
extracting a plurality of global features from the sample dataset; (This step for extracting features is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
applying a feature selection process for selecting a first subset of the plurality of global features, the first subset comprising fewer features than the plurality of global features; (This step for selecting a subset of features is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
measuring a baseline classification performance of the ML model fed the plurality of global features; (This step for measuring classification performance is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
measuring a classification performance of the ML model fed the first subset; (This step for measuring classification performance is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
evaluating the classification performance of the ML model fed the first subset relative to the baseline classification performance to determine significant degradation in performance; (This step for evaluating classification performance is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
analyzing the classification performance of the ML model fed the first subset, to identify an error in classification by the ML model, wherein the identification of the error in classification is in response to the determination of significant degradation in performance; (This step for analyzing performance to identify error is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
identifying a subset of the sample dataset related to the error; (This step for identifying a subset related to error is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
extracting a plurality of second features from the subset of the sample data; (This step for extracting features is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
applying the feature selection process for selecting a second subset of the plurality of second features, the second subset comprising fewer features than the plurality of second features; (This step for selecting a subset of features is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
A computer implemented method of generating a reduced version of a Machine Learning (ML) model, comprising: (The computer implemented method is understood to be generic computer elements – See MPEP 2106.05(f).)
obtaining a sample dataset; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
creating a reduced version of the ML model that uses fewer extracted features than the ML model, comprising an ensemble of: a first ML model component trained by applying the first subset of the plurality of global features to the sample dataset, and a second ML model component trained by applying the second subset of the plurality of second features to the subset of the sample data related to the error, wherein the second ML model component is configured to resolve the error in classification identified when the ML model is fed the first subset. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying machine learning as a tool to perform the abstract idea - see MPEP 2106.05(f).)
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are insignificant extra solution activity in combination of generic computer functions that are implemented to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
A computer implemented method of generating a reduced version of a Machine Learning (ML) model, comprising: (The computer implemented method is understood to be generic computer elements – See MPEP 2106.05(f).)
obtaining a sample dataset; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
creating a reduced version of the ML model that uses fewer extracted features than the ML model, comprising an ensemble of: a first ML model component trained by applying the first subset of the plurality of global features to the sample dataset, and a second ML model component trained by applying the second subset of the plurality of second features to the subset of the sample data related to the error, wherein the second ML model component is configured to resolve the error in classification identified when the ML model is fed the first subset. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying machine learning as a tool to perform the abstract idea - see MPEP 2106.05(f).)
The additional elements as disclosed above in combination of the abstract idea
are not sufficient to amount to significantly more than the judicial exception as they are
well, understood, routine and conventional activity as disclosed in combination
of generic computer functions that are implemented to perform the disclosed abstract idea above.
Regarding Claim 15: see the rejection of claim 1 above. Same rationale applies.
2A Prong 2 & 2B: The claim recites another additional element “A system for generating a reduced version of a Machine Learning (ML) model, comprising: at least one processor executing a code for:” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 2
2A Prong 1: The claim does not recite an Abstract idea.
2A Prong 2 & 2B:
wherein during inference, input data is first fed into the first ML model component to obtain a classification outcome, and in response to the classification outcome denoting the error in classification comprising ambiguity in the classification outcome, the input data is fed into the second ML model component to obtain a resolution to the classification outcome. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying machine learning as a tool to perform the abstract idea - see MPEP 2106.05(f).)
Regarding Claim 3
2A Prong 1:
wherein the feature selection process comprises defining a target function according to the plurality of global features or second features and a correlation with an expected outcome of the ML model being fed the plurality of global features or second features, and finding a minimum of the target function, the minimum representing the first subset or the second subset. (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 4
2A Prong 1: The claim does not recite an Abstract idea.
2A Prong 2 & 2B:
further comprising applying a quantum annealer based process for finding a minimum of the cost function. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying quantum annealer as a tool to perform the abstract idea - see MPEP 2106.05(f).)
Regarding Claim 5
2A Prong 1: The claim does not recite an Abstract idea.
2A Prong 2:
wherein the error comprises at least two classification categories of a plurality of classification categories for which the ML model performance incorrect classification at a rate above a threshold (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the error - See MPEP 2106.05(h).), wherein an input data is fed into the second ML model component when the first ML model component classifies input data into the at least two classification categories (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).), wherein the second ML model component classifies the input data into one of the at least two classification categories. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying machine learning as a tool to perform the abstract idea - see MPEP 2106.05(f).)
2B:
wherein the error comprises at least two classification categories of a plurality of classification categories for which the ML model performance incorrect classification at a rate above a threshold (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the error - See MPEP 2106.05(h).), wherein the input data is fed into the second ML model component when the first ML model component classifies input data into the at least two classification categories (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i)))), wherein the second ML model component classifies the input data into one of the at least two classification categories. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying machine learning as a tool to perform the abstract idea - see MPEP 2106.05(f).)
Regarding Claim 6
2A Prong 1:
wherein the at least two classification categories are merged into a single classification category, (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
2A Prong 2 & 2B:
and the first ML model component is trained to classify the input data into the single classification category or other classification categories of the plurality of classification categories. (Training a machine learning model is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
Regarding Claim 7
2A Prong 1: The claim does not recite an Abstract idea.
2A Prong 2 & 2B:
wherein the second ML model component resolves ambiguity of the single classification category by classifying the input data into one of the at least two classification categories merged into the single classification category. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying machine learning as a tool to perform the abstract idea - see MPEP 2106.05(f).)
Regarding Claim 8
2A Prong 1:
further comprising computing a confusion matrix of the ML model fed the first subset to identify the error. (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 10
2A Prong 1:
further comprising: analyzing the classification performance of the second ML model component fed the second subset, to identify a second error in classification by the second ML model component; identifying a second subset of the sample dataset related to the second error; extracting a plurality of third features from the second subset of the sample data; applying the feature selection process for selecting a third subset of the plurality of third features; (These steps are practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
2A Prong 2 & 2B:
creating a third ML model component for inclusion in the reduced version of the ML model, the third ML model component trained by applying the third subset of features to the sample dataset, wherein input data is fed into the third ML model component when the second ML model component performs the second error in classification. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying machine learning as a tool to perform the abstract idea - see MPEP 2106.05(f).)
Regarding Claim 11
2A Prong 1:
further comprising: iterating the analyzing the classification performance, the identifying the subset, the extracting, the applying the feature selection process, and the creating the reduced version, for creating a hierarchical tree of ML model components, wherein each lower level ML model component is for resolving classification ambiguity of a higher level ML model component. (These steps are practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 12
2A Prong 1: The claim does not recite an Abstract idea.
2A Prong 2 & 2B:
wherein the plurality of second features are the same as the plurality of global features. (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the second features - See MPEP 2106.05(h).)
Regarding Claim 13
2A Prong 1:
extracting a first subset of features from the input data;
analyzing the first classification outcome to determine whether an error in classification occurred,
in response to determining the error, extracting a second subset of features from the input data; (These steps are practically performable in the human mind and is understood to be a recitation of a mental process (i.e., judgment/evaluation).)
2A Prong 2:
A computer implemented method of inference by a reduced version of a Machine Learning (ML) model, comprising: (The computer implemented method is understood to be generic computer elements – See MPEP 2106.05(f).)
obtaining input data; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
feeding the first subset of features into a first ML model component of the reduced version of the ML model; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
obtaining a first classification outcome from the first ML model component; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
feeding the second subset of features into a second ML model component of the reduced version of the ML model; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)and
obtaining a second classification outcome comprising a resolution to the error of the first classification outcome from the second ML model, wherein the reduced version of the ML model is created according to claim 1. (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
2B:
A computer implemented method of inference by a reduced version of a ML model, comprising: (The computer implemented method is understood to be generic computer elements – See MPEP 2106.05(f).)
obtaining input data; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
feeding the first subset of features into a first ML model component of the reduced version of the ML model; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
obtaining a first classification outcome from the first ML model component; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
feeding the second subset of features into a second ML model component of the reduced version of the ML model; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i)))) and
obtaining a second classification outcome comprising a resolution to the error of the first classification outcome from the second ML model, wherein the reduced version of the ML model is created according to claim 1. (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
Regarding Claim 14
2A Prong 1: The claim does not recite an Abstract idea.
2A Prong 2 & 2B:
wherein the error in classification occurred when the first classification outcome comprises a single classification category representing a merger of a plurality of different classification categories, wherein the first ML model component is unable to accurately classify into one of the plurality of different classification categories, wherein the second ML model component accurately classifies into one of the plurality of different classification categories. (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the error - See MPEP 2106.05(h).)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 12-13, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramirez et al. (US 2019/0228527 A1, hereinafter "Ramirez") in view of Narsky (US 9,189,750 B1, hereinafter "Narsky").
Regarding Claim 1
Ramirez discloses: A computer implemented method of generating a reduced version of a Machine Learning (ML) model ([Para 0052, 0110-0112, and Fig 1] describes a computer and ensemble of classifiers with lower dimensionality.), comprising:
obtaining a sample dataset; ([Para 0056, 0082, 0100, and Fig 6-7] describes acquiring an image dataset.)
extracting a plurality of global features from the sample dataset; ([Para 0057-0060, 0113 and Fig 6-7] describes a subset of extracted features form a larger common pool of features.)
applying a feature selection process for selecting a first subset of the plurality of global features, the first subset comprising fewer features than the plurality of global features; ([Para 0079, 0081, 0119, and Fig 6-7] describes feature selection to select a subset of features from all of the features available in a dataset.)
analyzing the classification performance of the ML model fed the first subset, to identify an error in classification by the ML model, wherein the identification of the error in classification is in response to the determination of significant degradation in performance; ([Para 0087-0093, 0095, and Fig 6-7] describes evaluate the performance of the first level model and high rates of error. Examiner interprets high error as significant degradation.)
identifying a subset of the sample dataset related to the error; ([Para 0094 and Fig 6-7] describes images from the training/validation dataset associated with problematic classification probability may be selected for the second level classifier.)
extracting a plurality of second features from the subset of the sample data; ([Para 0096, 0126-0128, and Fig 6-7] describes features used in the second level model may be different from the ones used in the first level and extract features for Level 2 classifier.)
applying the feature selection process for selecting a second subset of the plurality of second features, the second subset comprising fewer features than the plurality of second features; ([Para 0096, 0119-0120, and Fig 6-7] describes a new feature selection process carried out using the subset of training data selected.) and
creating a reduced version of the ML model that uses fewer extracted features than the ML model, comprising an ensemble of ([Para 0097, 0111-0112 and Fig 6]): a first ML model component trained by applying the first subset of the plurality of global features to the sample dataset ([Para 0083-0084, 0119 and Fig 6-7] describes level 1 classifier that is machine learning and uses reduced feature set.), and a second ML model component trained by applying the second subset of the plurality of second features to the subset of the sample data related to the error ([0096-0098 and Fig 6-7] describes level 2 classifier trained with data associated with low classification rates with a new feature selection performed using the selected training data subset.), wherein the second ML model component is configured to resolve the error in classification identified when the ML model is fed the first subset. ([Para 0083, 0095-0098] describe the second level model for hard to classify samples identified by the first level model.)
Ramirez does not explicitly disclose: measuring a baseline classification performance of the ML model fed the plurality of global features; measuring a classification performance of the ML model fed the first subset; evaluating the classification performance of the ML model fed the first subset relative to the baseline classification performance to determine significant degradation in performance;
However, Narsky discloses in the same field of endeavor: measuring a baseline classification performance of the ML model fed the plurality of global features; ([Col 11 lines 45-67, Col 12 lines 1-5, and Fig 5-7] describes calculating a classification error for the full feature set.)
measuring a classification performance of the ML model fed the first subset; ([Col 11 lines 45-67, Col 12 lines 1-5, and Fig 5-7] describes calculating a classification error for the second feature set. [Col 2 lines 18-20] states “wherein the second feature set is a subset of the first feature set and is a candidate for the reduced feature set.”)
evaluating the classification performance of the ML model fed the first subset relative to the baseline classification performance to determine significant degradation in performance; ([Col 12 lines 5-50, Col 15 lines 7-12, and Fig 5-7] describes comparing the classification error of the full feature set and second feature set. [Col 11 lines 5-8] states “the model using the potential second feature set may perform significantly worse than the model using the first feature set.”)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of feature selection disclosed by Narsky into the method of Classification disclosed by Ramirez to evaluate the classification performance of a ML model fed a first subset relative to a baseline classification performance. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of feature selection disclosed by Narsky as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to remove less important features from a full feature set and compare performance.
Regarding Claim 15
Ramirez in view of Narsky discloses: A system for generating a reduced version of a ML model, comprising: at least one processor executing a code for ([Para 0052, 0110-0112, and Fig 1], Ramirez describes a computer and ensemble of classifiers with lower dimensionality.): (Claim 15 is a system claim that corresponds to claim 1 and the rest of the limitations are rejected on the same ground)
Regarding Claim 2
Ramirez in view of Narsky discloses: The computer implemented method of claim 1, wherein during inference, input data is first fed into the first ML model component to obtain a classification outcome, and in response to the classification outcome denoting the error in classification comprising ambiguity in the classification outcome, the input data is fed into the second ML model component to obtain a resolution to the classification outcome. ([Para 0083, 0095-0098], Ramirez describes the second level model for hard to classify samples identified by the first level model.)
Regarding Claim 12
Ramirez in view of Narsky discloses: The computer implemented method of claim 1, wherein the plurality of second features are the same as the plurality of global features. ([Abstract, Col 2 lines 17-20, and Claim 1], Narsky describes the secondary feature set as a subset of the primary feature set.)
Regarding Claim 13
Ramirez in view of Narsky discloses: A computer implemented method of inference by a reduced version of a Machine Learning (ML) model, comprising:
obtaining input data; ([Para 0056, 0082, 0100, and Fig 6-7], Ramirez describes acquiring an image dataset.)
extracting a first subset of features from the input data; ([Para 0057-0060, 0100, 0113 and Fig 6-7] Ramirez describes a subset of extracted features form the images.)
feeding the first subset of features into a first ML model component of the reduced version of the ML model; ([Para 0057-0060, 0100, 0113 and Fig 6-7] Ramirez describes the subset of extracted features selected and mapped into a first level model.)
obtaining a first classification outcome from the first ML model component; ([Para 0057-0060, 0100, 0113 and Fig 6-7] Ramirez describes determining a classification by the first level model.)
analyzing the first classification outcome to determine whether an error in classification occurred, ([Para 0087-0093, 0095, 0100 and Fig 6-7] Ramirez describes evaluate the performance of the first level model and higher rates of error.)
in response to determining the error, extracting a second subset of features from the input data; ([Para 0094-0096, 0100, 0126-0128, and Fig 6-7] Ramirez describes images from the training/validation dataset associated with problematic classification probability may be selected for the second level classifier.)
feeding the second subset of features into a second ML model component of the reduced version of the ML model; ([Para 0083, 0100, and Fig 6-7] Ramirez describes a second classifiers to classify samples identified during the validation of the first level model.) and
obtaining a second classification outcome comprising a resolution to the error of the first classification outcome from the second ML model, wherein the reduced version of the ML model is created according to claim 1. ([Para 0083, 0100, and Fig 6-7] Ramirez describes a second classifiers to classify samples for hard to classify samples of the first level model.)
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramirez in view of Narsky and Herbster et al. (US 20200005154 A1, hereinafter "Herbster")
Regarding Claim 3
Ramirez in view of Narsky discloses: The computer implemented method of claim 1,
Ramirez in view of Narsky does not explicitly disclose: wherein the feature selection process comprises defining a target function according to the plurality of global features or second features and a correlation with an expected outcome of the ML model being fed the plurality of global features or second features, and finding a minimum of the target function, the minimum representing the first subset or the second subset.
However, Herbster discloses in the same field of endeavor: wherein the feature selection process comprises defining a target function according to the plurality of global features or second features and a correlation with an expected outcome of the ML model being fed the plurality of global features or second features, and finding a minimum of the target function, the minimum representing the first subset or the second subset. ([Para 0068-0073] describes minimizing the value of a loss function thereby optimizing the accuracy of the updated feature vector generated.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Data encoding and Classification disclosed by Herbster into the method Ramirez in view of Narsky. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Data encoding and Classification disclosed by Herbster as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to minimize the value of a loss function thereby optimizing the accuracy of updated feature vectors.
Regarding Claim 4
Ramirez in view of Narsky in view of Herbster discloses: The computer implemented method of claim 3, further comprising applying a quantum annealer based process for finding a minimum of the cost function. ([Para 0073], Herbster describes minimizing a loss function and quantum annealer.)
Claim(s) 5-7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramirez in view of Narsky and Alon et al. (US 20220156524 A1, hereinafter "Alon").
Regarding Claim 5
Ramirez in view of Narsky discloses: The computer implemented method of claim 1, wherein the error comprises at least two classification categories of a plurality of classification categories for which the ML model performance incorrect classification ([Para 0048, 0056, 0098, Fig 2and Fig 6-7] Ramirez describes the classifier assigning images into different classes or categories .), wherein an input data is fed into the second ML model component when the first ML model component classifies input data into the at least two classification categories, wherein the second ML model component classifies the input data into one of the at least two classification categories. ([Para 0083, 0100, and Fig 6-7] Ramirez describes a second classifiers to classify samples identified during the validation of the first level model.)
Ramirez in view of Narsky does not explicitly disclose: the ML model performance incorrect classification at a rate above a threshold,
However, Alon discloses in the same field of endeavor: the ML model performance incorrect classification at a rate above a threshold ([Para 0004, 0035, 0063, and Fig 4] describes a sub-threshold correctness metric.), wherein an input data is fed into the second ML model component when the first ML model component classifies input data into the at least two classification categories, wherein the second ML model component classifies the input data into one of the at least two classification categories ([Para 0004, 0035, 0063, and Fig 4] describes applying the second machine learning model to the input to generate a second model output.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Ensembles and Cascades disclosed by Alon into the method of Ramirez in view of Narsky to disclose a classification threshold. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Ensembles and Cascades disclosed by Alon as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to render an increase in a combined output accuracy for executing additional machine learning model(s).
Regarding Claim 6
Ramirez in view of Narsky and Alon discloses: The computer implemented method of claim 5, wherein the at least two classification categories are merged into a single classification category, and the first ML model component is trained to classify the input data into the single classification category or other classification categories of the plurality of classification categories.([Para 0021-0022, 0069, and 0075], Alon describes combining model outputs.)
Regarding Claim 7
Ramirez in view of Narsky and Alon discloses: The computer implemented method of claim 6, wherein the second ML model component resolves ambiguity of the single classification category by classifying the input data into one of the at least two classification categories merged into the single classification category. ([Para 0021-0022, 0069, and 0075], Alon describes combining model outputs.)
Regarding Claim 14
Ramirez in view of Narsky and Alon discloses: The computer implemented method of claim 13, wherein the error in classification occurred when the first classification outcome comprises a single classification category representing a merger of a plurality of different classification categories ([Para 0021-0022, 0069, and 0075], Alon describes combining model outputs.), wherein the first ML model component is unable to accurately classify into one of the plurality of different classification categories, wherein the second ML model component accurately classifies into one of the plurality of different classification categories. ([Para 0035] Alon describes A second, conditionally-executed model trained only on those examples in the training dataset for which the first model performed poorly.)
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramirez in view of Narsky and Brennan et al. (US 20180075368 A1, hereinafter "Brennan").
Regarding Claim 8
Ramirez in view of Narsky discloses: The computer implemented method of claim 1,
Ramirez in view of Narsky does not explicitly disclose: further comprising computing a confusion matrix of the ML model fed the first subset to identify the error.
However, Brennan discloses in the same field of endeavor: further comprising computing a confusion matrix of the ML model fed the first subset to identify the error. ([Para 0038, 0046, 0050, and Fig 1-4] describes the confusion matrix to identify confusion/misclassification.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of confusion matrix’s disclosed by Brennan into the method of Ramirez in view of Narsky to apply a third ML model. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of confusion matrix’s disclosed by Brennan as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to utilize confusion matrix to identify misclassification.
Claim(s) 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ramirez in view of Narsky and Farhadi et al. (US 20220067453 A1, hereinafter "Farhadi").
Regarding Claim 10
Ramirez in view of Narsky discloses: The computer implemented method of claim 1,
Ramirez in view of Narsky does not explicitly disclose: further comprising: analyzing the classification performance of the second ML model component fed the second subset, to identify a second error in classification by the second ML model component; identifying a second subset of the sample dataset related to the second error; extracting a plurality of third features from the second subset of the sample data; applying the feature selection process for selecting a third subset of the plurality of third features; and creating a third ML model component for inclusion in the reduced version of the ML model, the third ML model component trained by applying the third subset of features to the sample dataset, wherein input data is fed into the third ML model component when the second ML model component performs the second error in classification.
However, Farhadi discloses in the same field of endeavor: further comprising: analyzing the classification performance of the second ML model component fed the second subset, to identify a second error in classification by the second ML model component; identifying a second subset of the sample dataset related to the second error; extracting a plurality of third features from the second subset of the sample data; applying the feature selection process for selecting a third subset of the plurality of third features; and creating a third ML model component for inclusion in the reduced version of the ML model, the third ML model component trained by applying the third subset of features to the sample dataset, wherein input data is fed into the third ML model component when the second ML model component performs the second error in classification. ([Para 0085, Claims 14-15, and Fig 6] describes a third neural network for deeper feature extractions.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Hierarchical Neural networks disclosed by Farhadi into the method of Ramirez in view of Narsky to apply a third ML model. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Hierarchical Neural networks disclosed by Farhadi as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to utilize deeper models to gain a higher accuracy.
Regarding Claim 11
Ramirez in view of Narsky and Farhadi discloses: The computer implemented method of claim 1, further comprising: iterating the analyzing the classification performance, the identifying the subset, the extracting, the applying the feature selection process, and the creating the reduced version, for creating a hierarchical tree of ML model components, wherein each lower level ML model component is for resolving classification ambiguity of a higher level ML model component. ([Para 0006, 0028, 0085, Claims 13-15, and Fig 6] Farhadi describes hierarchical convolutional neural networks cascading down from the first neural network.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Sharma et al. (US 20220374783 A1) describes feature selection for models.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TEWODROS E MENGISTU whose telephone number is (571)270-7714. The examiner can normally be reached Mon-Fri 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ABDULLAH KAWSAR can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TEWODROS E MENGISTU/Primary Examiner, Art Unit 2127
