DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The following is a non-final office action. Claims [1-20] are currently pending and have been examined based on their merits.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception that is an abstract idea without a practical application or significantly more.
Step 1: claims 1-7 recite a system, claims 8-14 recite a computer implemented method (i.e. a series of steps), and claim 15-20 recite a computer storage medium, and therefore each claim falls within one of the four statutory categories.
Step 2A prong 1 (Is a judicial exception recited?):
The representative claim 1 recites: identify standard data features in labeled training data; generate engineered data features using the identified standard data features; select training data features from the standard data features and the engineered data features; generate a balanced training data subset using the labeled training data; classify data entries as being in a multi-party anonymization transaction (MAT) class using the selected training data features and the balanced training data subset; and classify an input data entry as being in the MAT class.
Claim 8: A method comprising: identifying standard data features in labeled training data; generating engineered data features using the identified standard data features; selecting training data features from the standard data features and the engineered data features; generating a balanced training data subset using the labeled training data; classify data entries as being in one of a first class and a second class using the selected training data features and the balanced training data subset, wherein the labeled training data includes more entries in the second class than entries in the first class; and classifying an input data entry as being in the first class.
Claim 15: identify standard data features in labeled training data; generate engineered data features using the identified standard data features; select training data features from the standard data features and the engineered data features; generate a balanced training data subset using the labeled training data; classify data entries as being in a multi-party anonymization transaction (MAT) class using the selected training data features and the balanced training data subset; and classify an input data entry as being in the MAT class.
The examiner further finds that the claims are directed to a mental process. The claims recite a method of identifying standard data features in labeled training data, generating engineered data features, selecting training data features, generating a balanced training data subset, and classifying an input data entry as being in a first class. The claims recite a mental process as a person is capable of mentally, or by using simple tools such as pen and paper, identifying and selecting data features from a data set, generating a training data subset, and classifying input data entries into a class based on the training data. Additionally, the claims recite steps and procedures that are similar to concepts the courts have identified as a mental process such as observations, evaluations, judgements, and opinions. Similar types of claims the courts have identified as being mental processes include claims to collect information, analyze it, and display a certain result. Merely evaluating training data to generate a refined training data subset and classifying data entries into a class are mental processes. The courts have further stated that claims that use a computer as a tool to perform a mental process or performing a mental process on a generic computer are also found to recite mental processes. Therefore, the claims recite an abstract idea.
Step 2A Prong 2 (Is the exception integrated into a practical application?): The claims additionally recite;
Claim 1: A system comprising: a processor; and a memory comprising computer program code, the memory and the computer program code configured to cause the processor to: train an entry classifier model to classify data entries
Claim 8: A computer and training an entry classifier model to classify data entries.
Claim 15: A computer storage medium has computer-executable instructions that, upon execution by a processor, cause the processor to at least: train an entry classifier model to classify data entries.
The additional elements of generic computer elements to train a classifier model to perform the abstract idea of classifying input data as being in a class based on training data is not an improvement to a technology or technical field. Therefore, the limitations merely amount to adding the words “apply it” (or an equivalent) to the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, as discussed in MPEP 2106.05(f). Receiving and processing data to generate training data and training a model is not an improvement to the technology or technical field of classifier models as this is a basic and fundamental step in using a model to perform the abstract idea of classifying input data entries into a class. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Step 2B (Does the claim recite additional elements that amount to significantly more that the judicial exception?): As discussed above, the additional imitations amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, as discussed in MPEP 2106.05(f). The additional elements do not recite an improvement to a technology or technical field but merely utilize the generic computer elements to perform the abstract idea of classifying data entries as being in a first designated class. Therefore, the additional elements do not direct the claims to significantly more.
The dependent claims 2-7, 9-14, and 16-20 further narrow the abstract idea of generating balanced training data subsets and classifying input data entries based on the generated training data, recited in the independent claims 1, 8, and 15 and are therefore directed towards the same abstract idea.
The dependent claims recite the following additional elements:
Claims 3, 5-6, 10, 12-13, 17, and 19: a first and second candidate entry classifier model.
However, the additional elements are directed to merely “apply it” or being applied to perform the abstract idea.
Therefore, claims 1-20 are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-5, 7-12, 14-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gunn (US 2025/0165848) in view of Dalal (US 2023/0376594).
Claims 1, 8, and 15: Gunn discloses A system comprising: a processor; and a memory comprising computer program code, the memory and the computer program code configured to cause the processor to: identify standard data features in labeled training data (Paragraph [0001-0003]; [0011-0013]; [0026]; Fig. 2, methods and systems are described for selecting machine learning features using iterative batch feature reduction. For example, methods and systems may generate a lightweight model from a full machine learning model using iterative feature selection. The lightweight model may be generated using an alternate set of features determined using explainability vectors from candidate models each trained on a portion of the full feature set. The system may iteratively remove features from candidate models using ranking in its corresponding explainability vector. Some aspects described comprise training a plurality of candidate models based on a plurality of feature groups split from a first set of features, wherein each candidate model in the plurality of candidate models takes as input a feature group from the plurality of feature groups, wherein each feature group includes no more than a first threshold number of features. For each candidate model in the plurality of candidate models: process the candidate model to extract an explainability vector, wherein each entry in the explainability vector corresponds to a feature in the feature group associated with the candidate model and is indicative of a correlation between the feature and output of the model; based on the explainability vector, selecting a second threshold number of features to generate a slim feature group; training a slim candidate model, wherein the slim candidate model takes as input the slim feature group; generate a second set of features by combining features from a plurality of slim candidate models; training a final candidate model which takes as input the second set of features. The system may receive training data which may contain a set of features and may be used as input by a machine learning model. The system may process the training data using a data cleansing process to generate a processed dataset. The system may train a first machine learning model);
generate engineered data features using the identified standard data features (Paragraph [0001-0003]; [0011-0013]; [0026-0028]; Fig. 2, The system may receive training data which may contain a set of features and may be used as input by a machine learning model. The system may process the training data using a data cleansing process to generate a processed dataset. The system may train a first machine learning model. The system may perform feature selection which generates a features set);
select training data features from the standard data features and the engineered data features (Paragraph [0001-0003]; [0011-0013]; [0026-0028]; Fig. 2, The system may receive training data which may contain a set of features and may be used as input by a machine learning model. The system may process the training data using a data cleansing process to generate a processed dataset. The system may train a first machine learning model. The system may perform feature selection which generates a features set);
generate a balanced training data subset using the labeled training data (Paragraph [0001-0003]; [0011-0013]; [0026-0028]; Fig. 2, methods and systems are described for selecting machine learning features using iterative batch feature reduction. For example, methods and systems may generate a lightweight model from a full machine learning model using iterative feature selection. The lightweight model may be generated using an alternate set of features determined using explainability vectors from candidate models each trained on a portion of the full feature set. The system may iteratively remove features from candidate models using ranking in its corresponding explainability vector. Some aspects described comprise training a plurality of candidate models based on a plurality of feature groups split from a first set of features, wherein each candidate model in the plurality of candidate models takes as input a feature group from the plurality of feature groups, wherein each feature group includes no more than a first threshold number of features. For each candidate model in the plurality of candidate models: process the candidate model to extract an explainability vector, wherein each entry in the explainability vector corresponds to a feature in the feature group associated with the candidate model and is indicative of a correlation between the feature and output of the model; based on the explainability vector, selecting a second threshold number of features to generate a slim feature group; training a slim candidate model, wherein the slim candidate model takes as input the slim feature group; generate a second set of features by combining features from a plurality of slim candidate models; training a final candidate model which takes as input the second set of features. The system may receive training data which may contain a set of features and may be used as input by a machine learning model. The system may process the training data using a data cleansing process to generate a processed dataset. The system may train a first machine learning model).
Gunn discloses a system of training a machine learning model to classify data into groups based on specific features. However, Gunn does not specifically disclose the following claim limitations: (Claims 1 and 15) train an entry classifier model to classify data entries as being in a multi-party anonymization transaction (MAT) class using the selected training data features and the balanced training data subset; and classify an input data entry as being in the MAT class using the trained entry classifier model. (Claim 8) training an entry classifier model to classify data entries as being in one of a first class and a second class using the selected training data features and the balanced training data subset, wherein the labeled training data includes more entries in the second class than entries in the first class; and classifying an input data entry as being in the first class using the trained entry classifier model.
In the same field of endeavor of training a machine learning model to classify input data based on features Dalal teaches (Claims 1 and 15) train an entry classifier model to classify data entries as being in a multi-party anonymization transaction (MAT) class using the selected training data features and the balanced training data subset; (Claim 8) training an entry classifier model to classify data entries as being in one of a first class and a second class using the selected training data features and the balanced training data subset, wherein the labeled training data includes more entries in the second class than entries in the first class (Paragraph [0003-0004]; [0006-0008]; [0043]; [0050]; [0063]; in some variations a method for identifying illegal digital currency transactions is provided. The method includes obtaining one or more blockchains of transaction blocks for transactions involving a digital currency, deriving from the one or more blockchains of transaction blocks a transaction graph of sequential transactions, applying clustering processing to the transaction graph to generate resultant one or more entity graphs representative of likely chains of digital currency transfers by respective one or more entities, extracting graph feature data based on the resultant one or more entity graphs, and applying classification processing to the extracted graph feature data to identify a suspected malicious entity from the one or more entities associated with the one or more entity graphs. Applying the classification processing may include applying a machine learning classification process to the extracted graph feature data to determine the suspected malicious entity. The machine learning classification process may be trained using initial address data comprising one or more digital currency addresses associated with one or more rouge transactions. Because rouge ransomware actors tend to make, upon receiving a ransom payment, multiple successive transactions in an effort to evade detection, the derivation process of the transaction graph can be simplified. For example, graphs generated focus on temporally local behavior of the address. Specifically to define local behavior such as coinjoin transactions. For the actor-to-actor graph creation, mixing coinjoin transactions should be identified. (The examiner notes that the broadest reasonable interpretation of identifying multi-party anonymization transactions includes a process to use a classifier model to detect transactions in a blockchain where multiple participants pool their transactions together into a large transaction ledger in an attempt to hide their identity which includes processes such as “coinjoin” where users pool inputs into a single signed transaction to obfuscate ownership));
(Claims 1 and 15) and classify an input data entry as being in the MAT class using the trained entry classifier model; (Claim 8) and classifying an input data entry as being in the first class using the trained entry classifier model. (Paragraph [0003-0004]; [0006-0008]; [0043]; [0050]; [0063]; in some variations a method for identifying illegal digital currency transactions is provided. The method includes obtaining one or more blockchains of transaction blocks for transactions involving a digital currency, deriving from the one or more blockchains of transaction blocks a transaction graph of sequential transactions, applying clustering processing to the transaction graph to generate resultant one or more entity graphs representative of likely chains of digital currency transfers by respective one or more entities, extracting graph feature data based on the resultant one or more entity graphs, and applying classification processing to the extracted graph feature data to identify a suspected malicious entity from the one or more entities associated with the one or more entity graphs. Applying the classification processing may include applying a machine learning classification process to the extracted graph feature data to determine the suspected malicious entity. The machine learning classification process may be trained using initial address data comprising one or more digital currency addresses associated with one or more rouge transactions. Because rouge ransomware actors tend to make, upon receiving a ransom payment, multiple successive transactions in an effort to evade detection, the derivation process of the transaction graph can be simplified. For example, graphs generated focus on temporally local behavior of the address. Specifically to define local behavior such as coinjoin transactions. For the actor-to-actor graph creation, mixing coinjoin transactions should be identified).
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to modify the system of iteratively training a plurality of classifiers to detect a slim feature group from a set of input data as disclosed by Gunn with the system of train an entry classifier model to classify data entries as being in a multi-party anonymization transaction (MAT) class using the selected training data features and the balanced training data subset; and classify an input data entry as being in the MAT class using the trained entry classifier model. Training an entry classifier model to classify data entries as being in one of a first class and a second class using the selected training data features and the balanced training data subset, wherein the labeled training data includes more entries in the second class than entries in the first class; and classifying an input data entry as being in the first class using the trained entry classifier model as taught by Dalal (Dalal [0043]). With the motivation of helping a classifier to detect a class of malicious software in a blockchain environment by classifying specific groups of data (Dalal [0002]).
Claims 2, 9, and 16: Modified Gunn discloses the system as per claim 1, the computerized method as per claim 8, and the computer storage medium as per claim 15. Gunn further discloses wherein generating the balanced training data subset includes: generating a first balanced training data subset with a first MAT class frequency, the first MAT class frequency indicating a percentage of entries in the first balanced training data subset that are in the MAT class (Paragraph [0021-0025] in another example the candidate model may contain a set of parameters comprising a hyperplane matrix for a support vector machine algorithm. Explainability subsystem may use a counterfactual explanation method to extract explainability vectors. The counterfactual explanation method looks for input data which are identical or extremely close in values for all features except one. Then the difference in the divergent value. This process is repeated on each feature for all pairs of available input vectors and the aggregated result is measured for the effect of each feature on the output of the model, which may be formed into the explainability vector. Using the explainability vector corresponding to a candidate model, the system may select a slim feature group from the feature groups associated with the candidate model. In some embodiments, the slim feature group may be features satisfying a percentile cutoff based on their values. The system may iteratively repeat the process of processing a candidate model to extract an explainability vector, generating a slim feature group, and training a slim candidate model based on the slim feature group);
and generating a second balanced training data subset with a second MAT class frequency, the second MAT class frequency indicating a percentage of entries in the second balanced training data subset that are in the MAT class, wherein the second MAT class frequency is greater than the first MAT class frequency (Paragraph [0021-0025] in another example the candidate model may contain a set of parameters comprising a hyperplane matrix for a support vector machine algorithm. Explainability subsystem may use a counterfactual explanation method to extract explainability vectors. The counterfactual explanation method looks for input data which are identical or extremely close in values for all features except one. Then the difference in the divergent value. This process is repeated on each feature for all pairs of available input vectors and the aggregated result is measured for the effect of each feature on the output of the model, which may be formed into the explainability vector. Using the explainability vector corresponding to a candidate model, the system may select a slim feature group from the feature groups associated with the candidate model. In some embodiments, the slim feature group may be features satisfying a percentile cutoff based on their values. The system may iteratively repeat the process of processing a candidate model to extract an explainability vector, generating a slim feature group, and training a slim candidate model based on the slim feature group. The first-generation candidate model may be associated with a first explainability vector from the first-generation candidate model. Using the first explainability vector, the system may generate a second-generation slim feature group, the system may train a second-generation candidate model, using the same algorithm and training procedures as the first-generation candidate model).
Claims 3, 10, and 17: Modified Gunn discloses the system as per claim 2, the computerized method as per claim 9, and the computer storage medium as per claim 16. Gunn further discloses wherein training the entry classifier model includes: training a first candidate entry classifier model using the first balanced training data subset; training a second candidate entry classifier model using the second balanced training data subset (Paragraph [0001-0003]; [0011-0013]; [0026-0028]; Fig. 2, methods and systems are described for selecting machine learning features using iterative batch feature reduction. For example, methods and systems may generate a lightweight model from a full machine learning model using iterative feature selection. The lightweight model may be generated using an alternate set of features determined using explainability vectors from candidate models each trained on a portion of the full feature set. The system may iteratively remove features from candidate models using ranking in its corresponding explainability vector. Some aspects described comprise training a plurality of candidate models based on a plurality of feature groups split from a first set of features, wherein each candidate model in the plurality of candidate models takes as input a feature group from the plurality of feature groups, wherein each feature group includes no more than a first threshold number of features. For each candidate model in the plurality of candidate models: process the candidate model to extract an explainability vector, wherein each entry in the explainability vector corresponds to a feature in the feature group associated with the candidate model and is indicative of a correlation between the feature and output of the model; based on the explainability vector, selecting a second threshold number of features to generate a slim feature group; training a slim candidate model, wherein the slim candidate model takes as input the slim feature group; generate a second set of features by combining features from a plurality of slim candidate models; training a final candidate model which takes as input the second set of features. The system may receive training data which may contain a set of features and may be used as input by a machine learning model. The system may process the training data using a data cleansing process to generate a processed dataset. The system may train a first machine learning model);
testing the first candidate entry classifier model and the second candidate entry classifier model; and selecting the trained entry classifier model from the first candidate entry classifier model and the second candidate entry classifier model based on a result of the testing (Paragraph [0001-0003]; [0011-0013]; [0023]; [0026-0028]; Fig. 2, methods and systems are described for selecting machine learning features using iterative batch feature reduction. For example, methods and systems may generate a lightweight model from a full machine learning model using iterative feature selection. The lightweight model may be generated using an alternate set of features determined using explainability vectors from candidate models each trained on a portion of the full feature set. The system may iteratively remove features from candidate models using ranking in its corresponding explainability vector. Some aspects described comprise training a plurality of candidate models based on a plurality of feature groups split from a first set of features, wherein each candidate model in the plurality of candidate models takes as input a feature group from the plurality of feature groups, wherein each feature group includes no more than a first threshold number of features. For each candidate model in the plurality of candidate models: process the candidate model to extract an explainability vector, wherein each entry in the explainability vector corresponds to a feature in the feature group associated with the candidate model and is indicative of a correlation between the feature and output of the model; based on the explainability vector, selecting a second threshold number of features to generate a slim feature group; training a slim candidate model, wherein the slim candidate model takes as input the slim feature group; generate a second set of features by combining features from a plurality of slim candidate models; training a final candidate model which takes as input the second set of features. The system may receive training data which may contain a set of features and may be used as input by a machine learning model. The system may process the training data using a data cleansing process to generate a processed dataset. The system may train a first machine learning model. The first-generation candidate model may be associated with a first candidate performance score. In some embodiments the generation of a candidate model which performs at a threshold may be selected).
Claims 4 and 11: Modified Gunn discloses the system as per claim 1 and the computerized method as per claim 8. Gunn further discloses wherein selecting the training data features from the standard data features and the engineered data features includes: generating a random forest including the standard data features and the engineered data features; determining impurity values for the standard data features and the engineered data features; calculating weight values for the standard data features and the engineered data features, wherein the calculated weight values indicate probabilities of associated nodes being reached in the generated random forest (Paragraph [0013]; [0021-0025]; [0046] the system may train a first machine learning model. First machine learning model may take as input vector of feature values for the entirety of first set of features and output a score. The first machine learning model may use one or more algorithms like a random forest to achieve quantitative prediction. In another example the candidate model may contain a set of parameters comprising a hyperplane matrix for a support vector machine algorithm. Explainability subsystem may use a counterfactual explanation method to extract explainability vectors. The counterfactual explanation method looks for input data which are identical or extremely close in values for all features except one. Then the difference in the divergent value. This process is repeated on each feature for all pairs of available input vectors and the aggregated result is measured for the effect of each feature on the output of the model, which may be formed into the explainability vector. Using the explainability vector corresponding to a candidate model, the system may select a slim feature group from the feature groups associated with the candidate model. In some embodiments, the slim feature group may be features satisfying a percentile cutoff based on their values. The system may iteratively repeat the process of processing a candidate model to extract an explainability vector, generating a slim feature group, and training a slim candidate model based on the slim feature group. The first-generation candidate model may be associated with a first explainability vector from the first-generation candidate model. Using the first explainability vector, the system may generate a second-generation slim feature group, the system may train a second-generation candidate model, using the same algorithm and training procedures as the first-generation candidate model. The first machine learning model may include one or more parameters that is uses to translate input into outputs. For example, an artificial neural network contains a matrix of weights, each weight in which is a real number. The combination of weights transform input values into output values. The system may measure the performance of the first machine learning model using a method to generate first performance metric);
combining the impurity values with the calculated weight values to form weighted impurity values of the standard data features and the engineered data features; calculating feature importance scores using the weighted impurity values, wherein the weighted impurity values are averaged across component trees of the generated random forest and normalized for each feature; and selecting the training data features using the calculated feature importance scores (Paragraph [0021-0025]; [0046]; [0048]; [0053] Explainability subsystem may use a counterfactual explanation method to extract explainability vectors. The counterfactual explanation method looks for input data which are identical or extremely close in values for all features except one. Then the difference in the divergent value. This process is repeated on each feature for all pairs of available input vectors and the aggregated result is measured for the effect of each feature on the output of the model, which may be formed into the explainability vector. Using the explainability vector corresponding to a candidate model, the system may select a slim feature group from the feature groups associated with the candidate model. In some embodiments, the slim feature group may be features satisfying a percentile cutoff based on their values. The system may iteratively repeat the process of processing a candidate model to extract an explainability vector, generating a slim feature group, and training a slim candidate model based on the slim feature group. The first-generation candidate model may be associated with a first explainability vector from the first-generation candidate model. Using the first explainability vector, the system may generate a second-generation slim feature group, the system may train a second-generation candidate model, using the same algorithm and training procedures as the first-generation candidate model. The first machine learning model may include one or more parameters that is used to translate input into outputs. For example, an artificial neural network contains a matrix of weights, each weight in which is a real number. The combination of weights transform input values into output values. The system may measure the performance of the first machine learning model using a method to generate first performance metric. The explainability vector contains one entry for each feature in the set of features in the input to the candidate model being processed, and the entry reflects the importance of that feature to the model. The candidate model may contain a matrix of weights for an algorithm. Explainability subsystem may use a gradient class activation mapping method to extract explainability vectors. The technique performs backpropagation on the output of the model to compute derivatives of features in the inputs. The derivatives may then be used as indications of importance of features to a model).
Claims 5, 12, and 18: Modified Gunn discloses the system as per claim 4, the computerized method as per claim 11, and the computer storage medium as per claim 15. Gunn further discloses wherein training the entry classifier model includes: training a first candidate entry classifier model using a first quantity of training data features associated with highest calculated feature importance scores; training a second candidate entry classifier model using a second quantity of training data features associated with highest calculated feature importance scores (Paragraph [0021-0025]; [0046]; [0048]; [0053]; Fig. 4, Explainability subsystem may use a counterfactual explanation method to extract explainability vectors. The counterfactual explanation method looks for input data which are identical or extremely close in values for all features except one. Then the difference in the divergent value. This process is repeated on each feature for all pairs of available input vectors and the aggregated result is measured for the effect of each feature on the output of the model, which may be formed into the explainability vector. Using the explainability vector corresponding to a candidate model, the system may select a slim feature group from the feature groups associated with the candidate model. In some embodiments, the slim feature group may be features satisfying a percentile cutoff based on their values. The system may iteratively repeat the process of processing a candidate model to extract an explainability vector, generating a slim feature group, and training a slim candidate model based on the slim feature group. The first-generation candidate model may be associated with a first explainability vector from the first-generation candidate model. Using the first explainability vector, the system may generate a second-generation slim feature group, the system may train a second-generation candidate model, using the same algorithm and training procedures as the first-generation candidate model. The first machine learning model may include one or more parameters that is used to translate input into outputs. For example, an artificial neural network contains a matrix of weights, each weight in which is a real number. The combination of weights transform input values into output values. The system may measure the performance of the first machine learning model using a method to generate first performance metric. The explainability vector contains one entry for each feature in the set of features in the input to the candidate model being processed, and the entry reflects the importance of that feature to the model. The candidate model may contain a matrix of weights for an algorithm. Explainability subsystem may use a gradient class activation mapping method to extract explainability vectors. The technique performs backpropagation on the output of the model to compute derivatives of features in the inputs. The derivatives may then be used as indications of importance of features to a model);
testing the first candidate entry classifier model and the second candidate entry classifier model; and selecting the trained entry classifier model from the first candidate entry classifier model and the second candidate entry classifier model based on a result of the testing (Paragraph [0021-0025]; [0046]; [0048]; [0053] Explainability subsystem may use a counterfactual explanation method to extract explainability vectors. The counterfactual explanation method looks for input data which are identical or extremely close in values for all features except one. Then the difference in the divergent value. This process is repeated on each feature for all pairs of available input vectors and the aggregated result is measured for the effect of each feature on the output of the model, which may be formed into the explainability vector. Using the explainability vector corresponding to a candidate model, the system may select a slim feature group from the feature groups associated with the candidate model. In some embodiments, the slim feature group may be features satisfying a percentile cutoff based on their values. The system may iteratively repeat the process of processing a candidate model to extract an explainability vector, generating a slim feature group, and training a slim candidate model based on the slim feature group. The first-generation candidate model may be associated with a first explainability vector from the first-generation candidate model. Using the first explainability vector, the system may generate a second-generation slim feature group, the system may train a second-generation candidate model, using the same algorithm and training procedures as the first-generation candidate model. The first machine learning model may include one or more parameters that is used to translate input into outputs. For example, an artificial neural network contains a matrix of weights, each weight in which is a real number. The combination of weights transform input values into output values. The system may measure the performance of the first machine learning model using a method to generate first performance metric. The explainability vector contains one entry for each feature in the set of features in the input to the candidate model being processed, and the entry reflects the importance of that feature to the model. The candidate model may contain a matrix of weights for an algorithm. Explainability subsystem may use a gradient class activation mapping method to extract explainability vectors. The technique performs backpropagation on the output of the model to compute derivatives of features in the inputs. The derivatives may then be used as indications of importance of features to a model).
Claims 7, 14, and 20: Modified Gunn discloses the system as per claim 1, the computerized method as per claim 8, and the computer storage medium as per claim 15. Gunn further discloses wherein generating the balanced training data subset includes: determining a MAT class frequency; stochastically selecting a first quantity of data entries that are in the MAT class from the labeled training data; stochastically selecting a second quantity of data entries that are in a non-MAT class from the labeled training data, wherein the second quantity is based on the determined MAT class frequency and the first quantity; and combining the selected first quantity of data entries and the selected second quantity of data entries into the balanced training data subset (Paragraph [0021-0025] in another example the candidate model may contain a set of parameters comprising a hyperplane matrix for a support vector machine algorithm. Explainability subsystem may use a counterfactual explanation method to extract explainability vectors. The counterfactual explanation method looks for input data which are identical or extremely close in values for all features except one. Then the difference in the divergent value. This process is repeated on each feature for all pairs of available input vectors and the aggregated result is measured for the effect of each feature on the output of the model, which may be formed into the explainability vector. Using the explainability vector corresponding to a candidate model, the system may select a slim feature group from the feature groups associated with the candidate model. In some embodiments, the slim feature group may be features satisfying a percentile cutoff based on their values. The system may iteratively repeat the process of processing a candidate model to extract an explainability vector, generating a slim feature group, and training a slim candidate model based on the slim feature group. The first-generation candidate model may be associated with a first explainability vector from the first-generation candidate model. Using the first explainability vector, the system may generate a second-generation slim feature group, the system may train a second-generation candidate model, using the same algorithm and training procedures as the first-generation candidate model).
Claim(s) 6, 13, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gunn (US 2025/0165848) in view of Dalal (US 2023/0376594) further in view of Kirshenbaum (US 2008/0154820).
Claims 6, 13, and 19: Modified Gunn discloses the system as per claim 5, the computerized method as per claim 12, and the computer storage medium as per claim 18. However, Gunn does not disclose wherein testing the first candidate entry classifier model and the second candidate entry classifier model includes: measuring a first false positive rate (FPR) and a first false negative rate (FNR) of the first candidate entry classifier model; measuring a second FPR and a second FNR of the second candidate entry classifier model; and wherein the result of the testing includes an indicator that a combination of the first FPR and the first FNR is optimized relative to a combination of the second FPR and the second FNR or that the combination of the second FPR and second FNR is optimized relative to the combination of the first FPR and the first FNR.
In the same field of endeavor of training classifiers Kirshenbaum teaches wherein testing the first candidate entry classifier model and the second candidate entry classifier model includes: measuring a first false positive rate (FPR) and a first false negative rate (FNR) of the first candidate entry classifier model; measuring a second FPR and a second FNR of the second candidate entry classifier model; and wherein the result of the testing includes an indicator that a combination of the first FPR and the first FNR is optimized relative to a combination of the second FPR and the second FNR or that the combination of the second FPR and second FNR is optimized relative to the combination of the first FPR and the first FNR (Paragraph [0013-0015]; [0038-0040]; Fig. 3, in accordance with some embodiments of the invention a feature selector for selecting features for building classifiers is able to select the output of a first classifier as a feature for building a second classifier. A feature refers to a predictive indicator to predict whether any given case has a particular property or belongs to one or more classes. A first approach for developing a partial order is a desirability matrix is built where the desirability matrix in one embodiment can be in the form of a table in which the rows are the classes for which classifiers are to be used as input features, and the columns are the classes to be used as input features. The measure of desirability for each value in the table can be based on metrics used to judge performance of a classifier. Such as an accuracy measure, an error rate, a true positive rate, a false negative rate, a true negative rate, a false positive rate, an area under an ROC curve (area under a curve that is a plot of true positive rate versus false positive rate for different threshold values for a classifier), a mean absolute rate, or some combination).
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to modify the system of iteratively training a plurality of classifiers to detect a slim feature group from a set of input data as disclosed by Gunn with the system of wherein testing the first candidate entry classifier model and the second candidate entry classifier model includes: measuring a first false positive rate (FPR) and a first false negative rate (FNR) of the first candidate entry classifier model; measuring a second FPR and a second FNR of the second candidate entry classifier model; and wherein the result of the testing includes an indicator that a combination of the first FPR and the first FNR is optimized relative to a combination of the second FPR and the second FNR or that the combination of the second FPR and second FNR is optimized relative to the combination of the first FPR and the first FNR as taught by Kirshenbaum (Kirshenbaum [0039]). With the motivation of helping to improve the accuracy of a classifier in detecting specific desired classes of data in a set (Kirshenbaum [0005]).
Therefore, claims 1-20 are rejected under 35 U.S.C. 103.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Heumann (US 2006/0074827) Method and apparatus for detecting temporal process variation and for managing and predicting performance of automatic classifiers.
McConnell (US 2013/0268465) Methods and systems for computer-based selection of identifying input for class differentiation.
Kheiri (US 2024/0273418) Aggregating unique training data.
Raharja (US 2025/0036941) A method of classification.
Li (US 2021/0027145) Fraudulent transaction detection method based on sequence wide and deep learning.
Wahrstätter, Anton, et al. "Improving cryptocurrency crime detection: Coinjoin community detection approach." IEEE Transactions on Dependable and Secure Computing 20.6 (2023): 4946-4956.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to COREY RUSS whose telephone number is (571)270-5902. The examiner can normally be reached on M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lynda Jasmin can be reached on 5712726782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be
obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COREY RUSS/Primary Examiner, Art Unit 3629
Prosecution Insights