DETAILED ACTION
Status of Claims
Claim(s) 1-20 are pending and are examined herein.
Claim(s) 1-20 are rejected under 35 U.S.C. §§§ 101 and 102/103.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement IDS(s) submitted on November 28, 2023 is in compliance with the provisions of 37 CFR 1.97 and have been considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself. Applicant is advised to consult MPEP 2106 for more details of the analysis.
Under Step 1 analysis,
Claims 1-7 recite a system (representing a machine);
Claims 8-14 recite a method (representing a process); and
Claims 15-20 recite a non-transitory machine-readable medium (representing an article of manufacture).
Therefore, each set of the claims falls into one of the four statutory categories (i.e., process, machine, article of manufacture, or composition of matter).
Claim(s)
1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more, and hence is not patent-eligible subject matter.
Regarding Claim 1,
Step 2A Prong 1: The claim recites an abstract idea enumerated in the 2019 PEG.
determining, using a first machine learning model, a classification for the transaction based on data associated with the transaction, (An abstract idea of a mental process. Examiner’s note: the “determining” step, as drafted, and under its broadest reasonable interpretation (BRI), covers concepts that can be practically performed in the human mind. But for the recitation of a machine learning model, that is nothing other than using a computer component to perform the abstract idea. See MPEP § 2106.04(a)(2)(III). The broader recitation of determining a classification for the transaction based on data associated with the transaction falls under the mental process category. For example, an individual can mentally predicts whether a transaction is fraudulent or not based on the features or information associated with the transaction. This is falls under evaluation and decision-making process that can be practically performed in the human mind.)
processing the request based on the classification. (An abstract idea of a mental process. Examiner’s note: the “processing” step, as drafted, and under its broadest reasonable interpretation (BRI), covers concepts that can be practically performed in the human mind. The plain meaning of the “processing” step merely suggest that when a decision is made regarding whether the transaction is appropriate or fraudulent, the action performed either completing the request (e.g., approving the payment) or rejecting the request (e.g., refusing the payment). Accordingly, an individual can manually process a transaction request based on the decision made whether the transaction is “good” or “bad.” This is a mental process. See MPEP § 2106.04(a)(2)(I) & (III).)
Step 2A Prong 2: Under this prong, we evaluate whether the claim recites additional elements that integrate the abstract idea into a practical application by considering the claim as a whole. The judicial exception is not integrated into a practical application.
Additional Elements Analysis:
The claim recite the additional element such as:
receiving a request for processing a transaction; (This amounts to adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g). The “receiving” step merely defines a received request to perform an abstract idea on the obtained transaction. This merely defines a generic computer function (i.e., data gathering in conjunction with the abstract idea.)
“using a machine learning model” (This amounts to no more than merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). In other words, the claim invokes computer and/or other machinery in its ordinary capacity merely as a tool to perform the abstract idea.)
“wherein the first machine learning model is trained using first training data having verified labels and second training data having inferred labels, and wherein the inferred labels of the second training data are generated based on a distribution of classifications associated with the first training data;” (This amounts to no more than merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). In other words, the claim invokes computer and/or other machinery in its ordinary capacity merely as a tool to an existing process. The examiner notes that the high-level recitation of training the machine learning model on a previously determined data amount to no more than merely applying computer program to perform the abstract idea.)
Step 2B: Under this prong, the claim must include additional elements that amount to significantly more than the judicial exception. These elements must not be well-understood, routine, or conventional in the relevant field. When viewed individually and as an ordered combination, the claim does not include any such additional elements that are sufficient to amount to significantly more (i.e., inventive concept).
Additional Elements Analysis:
As explained above, the claimed additional elements merely represents generic computer component (i.e., conventional models) configured to perform the abstract ideas and high-level machine learning training using predetermined data. This amounts to generally invoking computer component as a tool to perform an existing process. As described in MPEP § 2106.05(f), additional elements that invoke computers or other machinery merely as a tool to perform an existing process will generally not amount to significantly more than a judicial exception.
Additionally, the “receiving” and/or “accessing” steps remain insignificant extra-solution activity to the judicial exception. These additional limitations represent generic computer functions of receiving information and accessing computer programs to implement the abstract idea on a computer. They amount to either a data gathering and/or routine data access function. Courts have recognized computer functions such as “receiving or transmitting data over a network” and “storing and retrieving information in memory” as well‐understood, routine, and conventional functions. Accordingly, these elements remain insignificant extra-solution activities and do not provide an inventive concept. See MPEP § 2106.05(d).
Therefore, claim 1 does not recite patent-eligible subject matter.
Regarding Claim 2,
Step 2A Prong 1: Claim 2, which incorporates the rejection of claim 1, recites further limitation such as:
wherein the classifications include a first classification and a second classification, wherein the first classification corresponds to non-fraudulent transactions, and the second classification corresponds to fraudulent transactions. (That is part of the abstract idea recited in claim 1. This limitation merely defines the classifications where the first classification represents legitimate/good transactions that should be approved and fraudulent/bad transactions that should be declined. This is a mental process of classifying transactions.)
Step 2A Prong 2: The claim does not recite additional element that integrates the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
Therefore, claim 2 is ineligible.
Regarding Claim 3,
Step 2A Prong 1: Claim 3, which incorporates the rejection of claim 1, recites further limitation such as:
wherein the distribution of classifications comprises a first distribution of scores generated by a second machine learning model for first transactions associated with a first portion of the first training data classified as a first classification, wherein the distribution of classifications further comprises a second distribution of scores generated by the second machine learning model for second transactions associated with a second portion of the first training data classified as a second classification, (That is part of the abstract idea recited in claim 1. The claim limitations introduce the first and second distribution scores. This involves generate scores for each of the verified transactions and then performing statistical analysis on the generated scores (e.g., plotting histograms for the bad and good transactions). This covers concepts that can be practically performed in the human mind with the physical aid (e.g., pen and paper). See MPEP § 2106.04(a)(2)(I) & (III). The recitation of “the second machine learning model” to generate the first and second scores corresponding to the first and second portions of the first training data represents a computer instructions to perform the abstract idea (i.e., scoring transactions).)
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
The recitation of “the second machine learning model” to generate the first and second scores corresponding to the first and second portions of the first training data represents a computer instructions to perform the abstract idea (i.e., scoring transactions). In other words, this additional element amounts to no more than merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
wherein the second machine learning model was trained based on the first training data, and not the second training data. (This amounts to no more than merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). Examiner’s Note: High-level recitation of applying the trained model to perform predictions/scoring. The training and executing a machine learning model, recited at a high level of generality, amounts to merely using a general purpose computer to practice the abstract idea (see MPEP 2106.05(f)). Accordingly, claim 6 does not provide additional elements sufficient to integrate the abstract idea into a practical application.
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As explained above, the additional elements identified above do not provide significantly more than the abstract idea. As described in MPEP § 2106.05(f), additional elements that invoke computers or other machinery merely as a tool to perform an existing process will generally not amount to significantly more than a judicial exception.
Therefore, claim 3 is ineligible.
Regarding Claim 4,
Step 2A Prong 1: Claim 4, which incorporates the rejection of claim 1, recites further limitation such as:
determining a first score threshold and a second score threshold based on the distribution of classifications; and inferring, for each transaction record in the second training data, a label based on at least the first score threshold and the second score threshold. (An abstract idea of a mental process. Examiner’s note: the “determining” and “inferring” steps, as drafted, and under their broadest reasonable interpretation (BRI), cover concepts that can be practically performed in the human mind. The claim steps covers statistical analysis, evaluation, comparison, and decision making process. These concepts fall within the mental process category of abstract idea. For example, using score distribution plots for good and bad transactions, selecting two cutoff points (i.e., first threshold for good transactions and second threshold for bad transactions), and using the two cutoff thresholds as decision rule to label the unlabeled declined transactions as good or bad (See Spec e.g., paragraphs [0016]-[0017] and [0037]-[0038]). These are mental steps that can be practically performed in the human mind and/or with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III).)
Step 2A Prong 2: The claim does not recite additional element that integrates the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
Therefore, claim 4 is ineligible.
Regarding Claim 5,
Step 2A Prong 1: Claim 5, which incorporates the rejection of claim 4, recites further limitation such as:
generating a first score using a second machine learning model based on a first transaction record in the second training data; (That is part of the abstract idea recited in claim 4. Examiner’s note: the “generating” step, as drafted, and under its broadest reasonable interpretation (BRI), covers concepts that can be practically performed in the human mind with the aid of pen and paper. But for the recitation of the second machine learning model, that is not other than using a computer component to perform the abstract idea. See MPEP § 2106.04(a)(2)(III). This score generation steps merely represents the process of determining a probability score for the declined transactions, for example, the probability of a declined transaction being fraud. This steps falls under the mental process which can be practically performed in the human mind.) and
assigning a first label to the first transaction record when the first score falls below the first score threshold or assigning a second label to the first transaction record when the first score exceeds the second score threshold. (That is part of the abstract idea recited in claim 4. Examiner’s note: the “assigning” steps, as drafted, and under their broadest reasonable interpretation (BRI), cover concepts that can be practically performed in the human mind with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III). The claimed step merely involves evaluation, comparison, and decision-making process. This includes comparing the generated to the cutoff thresholds and assigning “good” label for those scores below the cutoff threshold and “bad” label for those scores above the cutoff threshold. Thus, scoring the declined transactions and labeling them based on the cutoff thresholds is a process that can be performed in the human mind and/or with the aid of pen and paper.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
The recitation of “using a second machine learning model” to generate scores based on the second transactions dataset amounts to merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). This limitation represents high-level applying a machine learning model to generate prediction, which amounts to no more than invoking computer or other machinery in its ordinary capacity as a tool to perform the abstract idea.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As explained above, the additional element identified above does not provide significantly more than the abstract idea. The high-level recitation of using the machine learning model amounts to generic and conventional computer component performing the abstract idea. This cannot provide an inventive concept. See MPEP § 2106.05(d).
Therefore, claim 5 is ineligible.
Regarding Claim 6,
Step 2A Prong 1: Claim 6, which incorporates the rejection of claim 4, recites further limitation such as:
generating a second score using a second machine learning model based on a second transaction record in the second training data; (That is part of the abstract idea recited in claim 4. Examiner’s note: the “generating” step, as drafted, and under its broadest reasonable interpretation (BRI), covers concepts that can be practically performed in the human mind with the aid of pen and paper. But for the recitation of the second machine learning model, that is not other than using a computer component to perform the abstract idea. See MPEP § 2106.04(a)(2)(III). This score generation steps merely represents the process of determining a probability score for the declined transactions, for example, the probability of a declined transaction being fraud. This steps falls under the mental process which can be practically performed in the human mind.)
in response to determining that the second score falls between the first score threshold and the second score threshold, generating a duplicated second transaction record; (That is part of the abstract idea recited in claim 4. The claim merely introduces a conditional comparison that checks if the score is between the first and second thresholds to generate a duplicate of the same transaction. This falls within the mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III).)
assigning a first label and a first weight to the second transaction record; and assigning a second label and a second weight to the duplicated second transaction record. (That is part of the abstract idea recited in claim 4. Examiner’s note: the “assigning” steps, as drafted, and under their broadest reasonable interpretation (BRI), cover concepts that can be practically performed in the human mind with the aid of pen and paper. See MPEP § 2106.04(a)(2)(III). The claimed step merely involves evaluation, comparison, and decision-making process. This includes comparing the generated to the cutoff thresholds and assigning the label “good” with high weight for those scores below the cutoff threshold and the label “bad” with low weight for those scores above the cutoff threshold. Thus, scoring the declined transactions and labeling/weighting them based on the cutoff thresholds is a process that can be performed in the human mind and/or with the aid of pen and paper.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
The recitation of “using a second machine learning model” to generate scores amounts to merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). This limitation represents high-level applying a machine learning model to generate prediction, which amounts to no more than invoking computer or other machinery in its ordinary capacity as a tool to perform the abstract idea.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As explained above, the additional element identified above does not provide significantly more than the abstract idea. The high-level recitation of using the machine learning model amounts to generic and conventional computer component performing the abstract idea. This cannot provide an inventive concept. See MPEP § 2106.05(d).
Therefore, claim 6 is ineligible.
Regarding Claim 7,
Step 2A Prong 1: Claim 7, which incorporates the rejection of claim 1, recites further limitation such as:
wherein the first training data includes previous transactions that have been approved, and the second training data includes previous transactions that have been declined. (That is part of the abstract idea. The claim limitation merely defines the first and second training data as being verified/approved transactions and declined/unlabeled transactions.)
Step 2A Prong 2: The claim does not recite additional element that integrates the judicial exception into a practical application.
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Therefore, claim 7 is ineligible.
Regarding Claim 8,
Step 2A Prong 1: The claim recites an abstract idea enumerated in the 2019 PEG.
determining, using a first machine learning model, a classification for the transaction based on data associated with the transaction, (An abstract idea of a mental process. Examiner’s note: the “determining” step, as drafted, and under its broadest reasonable interpretation (BRI), covers concepts that can be practically performed in the human mind. But for the recitation of a machine learning model, that is nothing other than using a computer component to perform the abstract idea. See MPEP § 2106.04(a)(2)(III). The broader recitation of determining a classification for the transaction based on data associated with the transaction falls under the mental process category. For example, an individual can mentally predicts whether a transaction is fraudulent or not based on the features or information associated with the transaction. This is falls under evaluation and decision-making process that can be practically performed in the human mind.)
processing the request based on the classification. (An abstract idea of a mental process. Examiner’s note: the “processing” step, as drafted, and under its broadest reasonable interpretation (BRI), covers concepts that can be practically performed in the human mind. The plain meaning of the “processing” step merely suggest that when a decision is made regarding whether the transaction is appropriate or fraudulent, the action performed either completing the request (e.g., approving the payment) or rejecting the request (e.g., refusing the payment). Accordingly, an individual can manually process a transaction request based on the decision made whether the transaction is “good” or “bad.” This is a mental process. See MPEP § 2106.04(a)(2)(I) & (III).)
Step 2A Prong 2: Under this prong, we evaluate whether the claim recites additional elements that integrate the abstract idea into a practical application by considering the claim as a whole. The judicial exception is not integrated into a practical application.
Additional Elements Analysis:
The claim recites the additional element such as:
receiving a request for processing a transaction; (This amounts to adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g). The “receiving” step merely defines a received request to perform an abstract idea on the obtained transaction. This merely defines a generic computer function (i.e., data gathering in conjunction with the abstract idea.)
“accessing a first machine learning model trained using first training data having verified labels and second training data having inferred labels, wherein the inferred labels of the second training data were generated using a second machine learning model based on a distribution of classifications associated with the first training data, and wherein the second machine learning model was trained using the first training data having the verified labels;” (These limitations amount to: adding insignificant extra-solution activity to the judicial exception, merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f) & (g). The “accessing” part of the claimed step merely represent a generic computer function configured to execute or retrieve computer instruction (i.e., accessing computer program). The recited first and second trained machine learning models represents a high-level recitation of generic computer components configured to perform the abstract idea. In other words, training and executing a machine learning model, recited at a high level of generality (both the first and second training/applying limitations are recited at a high level of generality), amount to merely using a general purpose computer to perform an existing process (see MPEP 2106.05(f)).)
“using a machine learning model” to determine a classification for the transaction amounts to no more than invoking computer component as a tool to perform the abstract idea, as discussed in MPEP § 2106.05(f). In other words, the claim invokes computer and/or other machinery in its ordinary capacity merely as a tool to perform the abstract idea.)
Step 2B: Under this prong, the claim must include additional elements that amount to significantly more than the judicial exception. These elements must not be well-understood, routine, or conventional in the relevant field. When viewed individually and as an ordered combination, the claim does not include any such additional elements that are sufficient to amount to significantly more (i.e., inventive concept).
Additional Elements Analysis:
As explained above, the claimed additional elements merely represents generic computer component (i.e., conventional models) configured to perform the abstract ideas and high-level machine learning training using predetermined data. This amounts to generally invoking computer component as a tool to perform an existing process. As described in MPEP § 2106.05(f), additional elements that invoke computers or other machinery merely as a tool to perform an existing process will generally not amount to significantly more than a judicial exception.
Additionally, the “receiving” and “accessing” steps remain insignificant extra-solution activity to the judicial exception. These additional limitations represent generic computer functions of receiving information and accessing computer programs to implement the abstract idea on a computer. They constitute either a data gathering and/or routine data access function. Courts have recognized computer functions such as “receiving or transmitting data over a network” and “storing and retrieving information in memory” as well‐understood, routine, and conventional functions. Accordingly, these elements remain insignificant extra-solution activities and do not provide an inventive concept. See MPEP § 2106.05(d).
Therefore, claim 8 does not recite patent-eligible subject matter.
Regarding Claim 9,
The claim recites similar limitations as corresponding claim 2. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 2, as described above, is equally applicable to claim 9.
Therefore, claim 9 is ineligible.
Regarding Claim 10,
The claim recites similar limitations as corresponding claim 3. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 3, as described above, is equally applicable to claim 10.
Therefore, claim 10 is ineligible.
Regarding Claim 11,
The claim recites similar limitations as corresponding claim 4. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 4, as described above, is equally applicable to claim 11.
Therefore, claim 11 is ineligible.
Regarding Claim 12,
The claim recites similar limitations as corresponding claim 5. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 5, as described above, is equally applicable to claim 12.
Therefore, claim 12 is ineligible.
Regarding Claim 13,
The claim recites similar limitations as corresponding claim 6. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 6, as described above, is equally applicable to claim 13.
Therefore, claim 13 is ineligible.
Regarding Claim 14,
The claim recites similar limitations as corresponding claim 7. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 7, as described above, is equally applicable to claim 14.
Therefore, claim 14 is ineligible.
Regarding Claim 15,
The claim recites similar limitations as corresponding claim 1. Therefore, the same analysis (subject matter eligibility analysis) that was utilized for claim 1, as described above, is equally applicable to claim 15. The only difference is that claim 1 is drawn to a system, and claim 15 is drawn to a non-transitory storage medium. The recitation of “non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations...” merely defines computer component and instructions to implement a judicial exception, and hence the claimed additional elements listed above are merely generic elements and the implementation of the elements merely amount to no more than instructions to apply the abstract idea using generic computer components. Therefore, the additional elements do not integrate the judicial exception into a practical application or amount to significantly more. See MPEP 2106.05(f).
Therefore, claim 15 is ineligible.
Regarding Claim 16,
The claim recites similar limitations as corresponding claim 2. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 2, as described above, is equally applicable to claim 12.
Therefore, claim 16 is ineligible.
Regarding Claim 17,
The claim recites similar limitations as corresponding claim 3. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 3, as described above, is equally applicable to claim 17.
Therefore, claim 17 is ineligible.
Regarding Claim 18,
The claim recites similar limitations as corresponding claim 4. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 4, as described above, is equally applicable to claim 18.
Therefore, claim 18 is ineligible.
Regarding Claim 19,
The claim recites similar limitations as corresponding claim 5. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 5, as described above, is equally applicable to claim 19.
Therefore, claim 19 is ineligible.
Regarding Claim 20,
The claim recites similar limitations as corresponding claim 6. Therefore, the same subject matter eligibility analysis (including the abstract idea) that was utilized for claim 6, as described above, is equally applicable to claim 20.
Therefore, claim 20 is ineligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-5, 7-9, 11-12, 14-16, and 18-19 are rejected under 35 U.S.C. 102(a)(1) and/or 102(a)(2) as being anticipated by Marcjan et al., (Pub. No.: US 20190130406 A1).
Regarding Claim 1,
Marcjan discloses the following:
A system, comprising: a non-transitory memory; and one or more hardware processors coupled with the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising: (Marcjan, [0023] “As illustrated in FIG. 1, in its most basic configuration, a computing system 100 typically includes at least one hardware processing unit 102 and memory 104. The memory 104 may be physical system memory, which may be volatile, non-volatile, or some combination of the two. The term “memory” may also be used herein to refer to non-volatile mass storage such as physical storage media. If the computing system is distributed, the processing, memory and/or storage capability may be distributed as well.”)
receiving a request for processing a transaction; (Marcjan, [0039] “As shown in FIG. 2, the computing system 200 may include a transaction entry module 210. In operation, the transaction module 210 may receive input from multiple users 201, 202, 203, 204, and any number of additional users as illustrated by the ellipses 205 to initiate a data transaction that is performed by the computing system 200.” [0066] “Once the training process is completed, the risk score module 220 may use the finally trained risk determination model to provide a risk score 221-224 for data transactions 211-214 received after or subsequent to the training process.”) [Examiner’s Note: The received input data transactions from users for processing corresponds to the received request for processing a transaction.]
determining, using a first machine learning model, a classification for the transaction based on data associated with the transaction, (Marcjan, [0044] “the determination of the risk scores may be based at least in part on one or more attributes that are associated with each of the data transactions 211-214.” [0063]-[0066] “Once the trained risk determination model 227 has been trained in part by the use of these labeled sets 253 and 254, the trained risk determination model 227 may then be used by the risk score module 220 to determine risk scores 221-224 for the data transactions 211-214. That is, the trained risk determination model 227 may be used by the risk score module 220 to determine the probability that a data transaction should be approved or rejected... Once the training process is completed, the risk score module 220 may use the finally trained risk determination model to provide a risk score 221-224 for data transactions 211-214 received after or subsequent to the training process.” [Examiner’s Note: the trained risk determination model (i.e., first machine learning model) determines a classification (i.e., risk score for the data transaction indicating whether it should be approved or rejected) based on attributes/features associated with the data transactions.]) wherein the first machine learning model is trained using first training data having verified labels and second training data having inferred labels, (Marcjan, [0061]-[0062] “The label module 275 may then update the labeled data transactions 250 by generating a labeled set 253 that includes those data transactions that are labeled as being approved. As shown in FIG. 2, the labeled set 253 may include those data transactions that were part of the labeled set 251 as well as any newly labeled data transactions. Likewise, the label module 275 may generate a labeled set 254 that includes those data transactions that are labeled as being rejected... The label module 275 may then use the labeled sets 253 and 254 as a training set in a training operation to train the risk determination model 227 to more correctly generate the risk scores 221-224 for the data transactions.” [Examiner’s Note: the risk determination model is trained using labeled sets 253 and 254, which includes the set of labeled data transactions 251/252 (i.e., first training data having verified labels) and the generated labeled data transactions added to the training sets e.g., 401, 402, 403, 404 (i.e., second training data having inferred labels). Further see paragraphs [0052] and [0070].]) and wherein the inferred labels of the second training data are generated based on a distribution of classifications associated with the first training data; (Marcjan, [0060]-[0061] “The training module 270 may include a risk score receiving module 276 that receives the risk scores 221-224 for each of the data transactions, a low threshold 277 that specifies a risk score that is very likely to be a valid, non-fraudulent data transaction that should be approved and a high threshold 278 that specifies a risk score that is very likely to be a non-valid, fraudulent transaction that should be rejected... The label module 275 may then apply the low and high thresholds 277 and 278 to the risk scores to determine which data transactions have a high likelihood of being approved or rejected. Those data transactions having a risk score that is below the low threshold 277 may be labeled as being approved and the data transactions having a risk score above the high threshold 278 may be labeled as being rejected.” [Examiner’s Note: the generated labels for the unlabeled data transactions using low/high thresholding of the risk scores determined by the risk determination model based on the approved set of labeled data transactions. It is noted that the first training data having verified labels are interpreted as the previously labeled data transactions (i.e., approved transactions) and the inferred labels of the second training data are interpreted as labeled data transactions based on the risk determination module (e.g., the data transactions are not completed or rejected). Further see [0078] and [0080].]) and
processing the request based on the classification. (Marcjan, [0066] “Once the training process is completed, the risk score module 220 may use the finally trained risk determination model to provide a risk score 221-224 for data transactions 211-214 received after or subsequent to the training process. Based on this risk score, the determination module 230 may determine if the subsequently received data transactions should be approved or rejected. Those that should be approved will be automatically approved and those that should be rejected will be automatically rejected by the computing system 200.” [0047] “The risk score module may further include a decision module 230 that in operation uses the risk scores 221-224 to determine if each data transaction should be approved or rejected based on the risk score generated by one or more of the risk determination models 225-229.” [Examiner’s Note: the system processes the received subsequent transaction to approve or reject the transaction based on the model outputs.])
Regarding Claim 2, Marcjan teaches the elements of claim 1 as outlined above, and further teaches:
wherein the classifications include a first classification and a second classification, wherein the first classification corresponds to non-fraudulent transactions, and the second classification corresponds to fraudulent transactions. (Marcjan, [0042] “In operation, the risk score module 220 may determine a risk score for each of the data transactions 211-215 based on one or more risk determination models 225, 226, 227, 228 or any number of additional risk score models as determined by the ellipses 229. The risk scores may be a probability that is indicative of whether a given data transaction is a good transaction that should be approved or is a fraudulent or bad transaction that should be rejected.” [0047]-[0050] “That is, the decision module 230 may, based on the operation of the risk determination models, determine which data transactions should be approved as denoted at 231 and which should be rejected based on the risk scores as denoted at 232. In one embodiment, the decision model 225-229 may set a boundary or demarcation for risk scores that will be approved and risk scores that will rejected. For example, FIG. 3 shows risk scores with probabilities from 0 to 1, with those data transactions having a lower probability of being a fraudulent data transaction being given a lower score and those data transactions having a higher probability of being a fraudulent activity being given a higher score.” Further see para. [0060].)
Regarding Claim 4, Marcjan teaches the elements of claim 1 as outlined above, and further teaches:
determining a first score threshold and a second score threshold based on the distribution of classifications; (Marcjan, [0060] “The training module 270 may include a risk score receiving module 276 that receives the risk scores 221-224 for each of the data transactions, a low threshold 277 that specifies a risk score that is very likely to be a valid, non-fraudulent data transaction that should be approved and a high threshold 278 that specifies a risk score that is very likely to be a non-valid, fraudulent transaction that should be rejected. For example, as shown in FIG. 3, the low threshold 277 may be set very close to 0 and is thus quite distant from the X1% demarcation line. Accordingly, any data transactions having a risk score below the low threshold 277 have a very high likelihood of being a valid, non-fraudulent data transaction that should be approved. Likewise, the high threshold 278 may be set very close to 1 and is also quite distant from the X1% demarcation line.” [Examiner’s Note: Marcjan identifies two threshold (low threshold and high threshold) that are determined relative to the risk score range (i.e., the distribution of classifications) generated by the risk determination model.]) and inferring, for each transaction record in the second training data, a label based on at least the first score threshold and the second score threshold. (Marcjan, [0061] “The label module 275 may then apply the low and high thresholds 277 and 278 to the risk scores to determine which data transactions have a high likelihood of being approved or rejected. Those data transactions having a risk score that is below the low threshold 277 may be labeled as being approved and the data transactions having a risk score above the high threshold 278 may be labeled as being rejected. The label module 275 may then update the labeled data transactions 250 by generating a labeled set 253 that includes those data transactions that are labeled as being approved.” [Examiner’s Note: the reference describes the process of generating labels for transactions in the set of unlabeled data transactions by applying the low and high threshold of the risk score. This corresponds to the broadest reasonable interpretation of “inferring, for each transaction record in the second training data, a label based on at least the first score threshold and the second score threshold.”])
Regarding Claim 5, Marcjan teaches the elements of claim 5 as outlined above, and further teaches:
wherein the inferring the label comprises: generating a first score using a second machine learning model based on a first transaction record in the second training data; (Marcjan, [0061] “In operation, the risk score receiving module 276 may receive the risk scores 221-224 for each of the data transactions that were determined using the trained risk determination model 226 that was trained using the labeled sets 251 and 252.” [0044] “the determination of the risk scores may be based at least in part on one or more attributes that are associated with each of the data transactions 211-214.”) and assigning a first label to the first transaction record when the first score falls below the first score threshold or assigning a second label to the first transaction record when the first score exceeds the second score threshold. (Marcjan, [0061] “Those data transactions having a risk score that is below the low threshold 277 may be labeled as being approved and the data transactions having a risk score above the high threshold 278 may be labeled as being rejected.” [0070]-[0071] “FIG. 4A shows data transactions 401 and 402. As illustrated in the figure, these two unlabeled data transactions are quite distant from the demarcation on the approved (left) side of the demarcation. In other words, they are below the low threshold 277. Accordingly, as shown by the lines 401 a and 402 a between FIGS. 4A and 4B, the data transactions 401 and 402 are labeled as being approved and are added to the approved labeled set 253. FIG. 4A also shows data transactions 403 and 404. As illustrated in the figure, these two unlabeled data transactions are quite distant from the demarcation on the reject (right) side of the demarcation. In other words, they are above the high threshold 278. Accordingly, as shown by the lines 403 a and 404 a between FIGS. 4A and 4B, the data transactions 403 and 404 are labeled as being rejected and are added to the rejected labeled set 254.”) [Examiner’s Note: the approved/rejected labels correspond to the claimed first and second labels assigned to the transactions based on the scores and the corresponding thresholds.]
Regarding Claim 7, Marcjan teaches the elements of claim 1 as outlined above, and further teaches:
wherein the first training data includes previous transactions that have been approved, and the second training data includes previous transactions that have been declined. (Marcjan, [0015] “To combat fraud, many fraud detection models may be devised that attempt to label data transactions as good transactions or fraudulent transactions. However, properly labeled data transactions may be hard to acquire, especially for data transactions that are labeled as having been properly rejected.” [0052] “The review module 240 may then label these transactions as being approved or rejected. For example, as illustrated the review module may determine a set of labeled data transactions 250, which may initially include a set 251 of data transactions that are labeled as being approved and a set 252 of data transactions that are labeled as being rejected.” [0054] “However, the data transactions that are rejected by decision module 230 based on the risk scores are not actually performed or completed by the computing system 200. Accordingly, there is no way to actually determine if these transactions were properly rejected or if some of them should have been approved.”)
Regarding Claim 8, Marcjan discloses the following:
A method, comprising: (Marcjan, [0082] “FIG. 5 illustrates a flow chart of an example method 500 for training one or more risk determination models based on a set of labeled data transactions. The method 500 will be described with respect to one or more of the FIGS. 2-4 discussed previously.”)
receiving a request for processing a transaction; (Marcjan, [0039] “As shown in FIG. 2, the computing system 200 may include a transaction entry module 210. In operation, the transaction module 210 may receive input from multiple users 201, 202, 203, 204, and any number of additional users as illustrated by the ellipses 205 to initiate a data transaction that is performed by the computing system 200.” [0066] “Once the training process is completed, the risk score module 220 may use the finally trained risk determination model to provide a risk score 221-224 for data transactions 211-214 received after or subsequent to the training process.” [Examiner’s Note: The received input data transactions from users for processing corresponds to the received request for processing a transaction.])
accessing a first machine learning model trained using first training data having verified labels and second training data having inferred labels, (Marcjan, [0044] “the determination of the risk scores may be based at least in part on one or more attributes that are associated with each of the data transactions 211-214.” [0062]-[0066] “The label module 275 may then use the labeled sets 253 and 254 as a training set in a training operation to train the risk determination model 227 to more correctly generate the risk scores 221-224 for the data transactions... Once the trained risk determination model 227 has been trained in part by the use of these labeled sets 253 and 254, the trained risk determination model 227 may then be used by the risk score module 220 to determine risk scores 221-224 for the data transactions 211-214.” [0073] “As previously discussed, the labeled set 253 includes the data transactions of the labeled set 251 as well as data transactions 401 and 402. The labeled set 254 includes the data transactions of the labeled set 252 as well as data transactions 403 and 404. As previously discussed, the label module 275 may use the labeled sets 253 and 254 to train the risk determination model 227.” Further see [0087]. [Examiner’s Note: the trained risk determination model 227 using set of data transactions 253 and 254 which consist of approved labeled transactions (e.g., 251/252) and newly labeled data transactions (e.g., 401, 402, 403, 404). The risk determination model (e.g., 226) trained using labeled transactions (e.g., 251/252) to generate risk scores that is used to labeled data transactions.])
wherein the inferred labels of the second training data were generated using a second machine learning model based on a distribution of classifications associated with the first training data, (Marcjan, [0060]-[0061] “The training module 270 may include a risk score receiving module 276 that receives the risk scores 221-224 for each of the data transactions, a low threshold 277 that specifies a risk score that is very likely to be a valid, non-fraudulent data transaction that should be approved and a high threshold 278 that specifies a risk score that is very likely to be a non-valid, fraudulent transaction that should be rejected... In operation, the risk score receiving module 276 may receive the risk scores 221-224 for each of the data transactions that were determined using the trained risk determination model 226 that was trained using the labeled sets 251 and 252. The label module 275 may then apply the low and high thresholds 277 and 278 to the risk scores to determine which data transactions have a high likelihood of being approved or rejected.” [0070] “FIG. 4A shows data transactions 401 and 402. As illustrated in the figure, these two unlabeled data transactions are quite distant from the demarcation on the approved (left) side of the demarcation. In other words, they are below the low threshold 277. Accordingly, as shown by the lines 401 a and 402 a between FIGS. 4A and 4B, the data transactions 401 and 402 are labeled as being approved and are added to the approved labeled set 253.” [0086] “The method 500 includes, based at least on the determined first risk score, newly labeling one or more of the data transactions in the set of unlabeled data transactions, the newly labeled one or more data transactions being added to a second set of labeled data transactions, the second set of labeled transactions including the first set of labeled transactions and the newly labeled data transactions (540).” [Examiner’s Note: The risk determination model (e.g., 226) trained using labeled transactions (e.g., 251/252) to generate risk scores that is used to labeled data transactions e.g. (unlabeled data transactions). The risk determination model (e.g., model 226) corresponds to the second machine learning model.]) and wherein the second machine learning model was trained using the first training data having the verified labels; (Marcjan, [0058] “As illustrated, the training module 270 may include a label module 275. In operation, the label module 275 may access the labeled sets 251 and 252 and may then use these labeled sets as a training set in a training operation to train the risk determination model 226 to more correctly generate the risk scores 221-224 for the data transactions. As shown in FIG. 2, the risk determination model 226 is shown as being associated with the labeled sets 251 and 252 and this is to illustrate that this model is trained in part by the use of these labeled sets.” [0084] “For example, as previously described the training module 270 may train the risk determination model 226 using the labeled sets 251 and 252.”)
determining, using the first machine learning model, a classification for the transaction based on data associated with the transaction; (Marcjan, [0044] “the determination of the risk scores may be based at least in part on one or more attributes that are associated with each of the data transactions 211-214.” [0063]-[0066] “Once the trained risk determination model 227 has been trained in part by the use of these labeled sets 253 and 254, the trained risk determination model 227 may then be used by the risk score module 220 to determine risk scores 221-224 for the data transactions 211-214. That is, the trained risk determination model 227 may be used by the risk score module 220 to determine the probability that a data transaction should be approved or rejected... Once the training process is completed, the risk score module 220 may use the finally trained risk determination model to provide a risk score 221-224 for data transactions 211-214 received after or subsequent to the training process.” [Examiner’s Note: the trained risk determination model (i.e., first machine learning model) determines a classification (i.e., risk score for the data transaction indicating whether it should be approved or rejected) based on attributes/features associated with the data transactions.]) and
processing the request based on the classification. (Marcjan, [0066] “Once the training process is completed, the risk score module 220 may use the finally trained risk determination model to provide a risk score 221-224 for data transactions 211-214 received after or subsequent to the training process. Based on this risk score, the determination module 230 may determine if the subsequently received data transactions should be approved or rejected. Those that should be approved will be automatically approved and those that should be rejected will be automatically rejected by the computing system 200.” [0047] “The risk score module may further include a decision module 230 that in operation uses the risk scores 221-224 to determine if each data transaction should be approved or rejected based on the risk score generated by one or more of the risk determination models 225-229.” [Examiner’s Note: the system processes the received subsequent transaction to approve or reject the transaction based on the model outputs.]).
Regarding Claim 9,
The claim recites substantially similar limitations as corresponding claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding Claim 11,
The claim recites substantially similar limitations as corresponding claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding Claim 12,
The claim recites substantially similar limitations as corresponding claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding Claim 14,
The claim recites substantially similar limitations as corresponding claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale.
Regarding Claim 15,
The claim recites substantially similar limitations as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale. Claim 1 is directed to a system, and claim 15 is directed to a non-transitory machine-readable medium.
Marcjan also discloses a non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations ... (Marcjan, [0033] “Further, upon reaching various computing system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to storage media (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computing system RAM and/or to less volatile storage media at a computing system.”)
Regarding Claim 16,
The claim recites substantially similar limitations as corresponding claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding Claim 18,
The claim recites substantially similar limitations as corresponding claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding Claim 19,
The claim recites substantially similar limitations as corresponding claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 3, 6, 10, 13, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Marcjan et al., (Pub. No.: US 20190130406 A1) in view of Liao et al., (IDS: “Data Augmentation Methods for Reject Inference in Credit Risk Models.” (2021)).
Regarding Claim 3, Marcjan teaches the elements of claim 1 as outlined above:
As outlined above, While Marcjan teaches the second machine learning model (i.e., risk determination model 226) trained using the first training data (i.e., 251/252 approved data transactions). See Marcjan paragraphs [0058] and [0084]. Marcjan does not appear to explicitly teach:
wherein the distribution of classifications comprises a first distribution of scores generated by a second machine learning model for first transactions associated with a first portion of the first training data classified as a first classification, wherein the distribution of classifications further comprises a second distribution of scores generated by the second machine learning model for second transactions associated with a second portion of the first training data classified as a second classification, and wherein the second machine learning model was trained based on the first training data, and not the second training data.
However, Marcjan in view of Liao teaches the limitations. Hereinafter, Liao, in combination with Marcjan, teaches:
wherein the distribution of classifications comprises a first distribution of scores generated by a second machine learning model for first transactions associated with a first portion of the first training data classified as a first classification, wherein the distribution of classifications further comprises a second distribution of scores generated by the second machine learning model for second transactions associated with a second portion of the first training data classified as a second classification, and wherein the second machine learning model was trained based on the first training data, and not the second training data. (Liao, [P. 2, Section: 2] “Consider a set of n loan applications x1,x2,··· ,xn ∈ Rk where k is the number of features. This set includes m accepted applications x1,x2,··· ,xm ∈ Xa with corresponding labels y1,y2,··· ,ym ∈ {Good, Bad} and con sists of xm+1,··· ,xn ∈ Xu whose labels are unknown. The credit scoring model trained with Xa only is denoted as Known Good/Bad (KGB) model. To mitigate sampling bias, reject inference techniques assign labels to unlabeled applications, and combine accepted data and pseudo-labeled data into inferred data sets to represent the whole application population and update credit scoring models. The scoring model with inferred data as training set is denoted as reject inference model (RI model).” [P. 2, Section: 2.1] “Figure 1 is an overview of our proposed self-training method pipeline. It starts with training an initial model (also the KGB model in the first iteration) on the accepted data Xa and uses it to predict all the unlabeled data. Then, a confidence model is introduced to filter the most confident predictions whose labels are either good or bad in unlabeled data Xu with a fine-tuned threshold... Our probability calibration confidence model adds isotonic probability calibration ... uses calibrated probabilities to filter confident predictions. Trust Score model, on the other hand, provides prediction accuracy from the nearest neighboring approach (Jiang et al. 2018). It pre-selects a high-density data range for each class. Then a trust score is defined as follows to evaluate the prediction: for a predicted test label, the trust score is the ratio between the distance from the test label to the nearest class different from the predicted label class and the distance to the predicted label class within the data range. In this work, the score is based on 5% of the instances from each class. A high score implies high prediction accuracy since the predicted case is close to labeled data with the same label class.” Further see [Section: 3.4].) [Examiner’ Note: The KGB model (i.e., the second machine learning mode) trained only with accepted dataset Xa (i.e., accepted applications/transactions). The subset of labeled data Xa “Good” represents the first portion of the first training data classified as a first classification. The subset of labeled data Xa “Bad” represents the second portion of the first training data classified as a second classification. The range of the prediction scores evaluated from 5% of the instances from each class Good and Bad corresponds to the first and second distribution of scores generated by the model.]
Marcjan and Liao are from the same field of endeavor and their disclosure generally relates to (training machine learning models for transaction classification).
Accordingly, at the effective filing date, it would have been prima facie obvious to one ordinarily skilled in the art to modify the combination of Marcjan and Liao to incorporate reject inference methods as taught by Liao. One would have been motivated to make such a combination in order improve the ability of credit score to differentiate good/bad loan applications and, more importantly, increased loan approval rate by 2.6% while keeping similar default rate comparing to the KGB model (Liao [Abstract]).
Regarding Claim 6,
Marcjan teaches the elements of claim 4 as outlined above:
While Marcjan teaches identifies two threshold (low threshold and high threshold) that are determined relative to the risk score range (i.e., the distribution of classifications) and generating labels for transactions in the set of unlabeled data transactions based on the identified low and high threshold relative to the risk score. Marcjan does not appear to explicitly teach:
... generating a duplicated second transaction record; assigning a first label and a first weight to the second transaction record; and assigning a second label and a second weight to the duplicated second transaction record.
However, Marcjan in view of Liao teaches the limitations:
generating a second score using a second machine learning model based on a second transaction record in the second training data; (Marcjan, [0044] “the determination of the risk scores may be based at least in part on one or more attributes that are associated with each of the data transactions 211-214.” [0061] “In operation, the risk score receiving module 276 may receive the risk scores 221-224 for each of the data transactions that were determined using the trained risk determination model 226 that was trained using the labeled sets 251 and 252. The label module 275 may then apply the low and high thresholds 277 and 278 to the risk scores to determine which data transactions have a high likelihood of being approved or rejected.”) in response to determining that the second score falls between the first score threshold and the second score threshold, (Marcjan, [0050] “However, as shown in FIG. 3 by the dashed lines on both sides of the line at X1%, there are some data transactions that are close to X1%. Since these transactions are close to X1%, it is likely that some of those to the left of the line are false negatives, that is bad or fraudulent data transactions that should have been rejected instead of being approved and some of those to the right of the line are false positives, that is good or non-fraudulent data transactions that should not have been rejected, but should have been approved. In some embodiments, the data transactions close to X1% may be marked as needing further review, as denoted at 330.” [0060]-[0061] “The training module 270 may include a risk score receiving module 276 that receives the risk scores 221-224 for each of the data transactions, a low threshold 277 that specifies a risk score that is very likely to be a valid, non-fraudulent data transaction that should be approved and a high threshold 278 that specifies a risk score that is very likely to be a non-valid, fraudulent transaction that should be rejected. For example, as shown in FIG. 3, the low threshold 277 may be set very close to 0 and is thus quite distant from the X1% demarcation line. Accordingly, any data transactions having a risk score below the low threshold 277 have a very high likelihood of being a valid, non-fraudulent data transaction that should be approved. Likewise, the high threshold 278 may be set very close to 1 and is also quite distant from the X1% demarcation line... Those data transactions having a risk score that is below the low threshold 277 may be labeled as being approved and the data transactions having a risk score above the high threshold 278 may be labeled as being rejected.”) [Examiner’s Note: the data transactions that fall between the low and high threshold are defined as the X1% 330 for further review (middle range).]
Hereinafter, Liao, in combination with Marcjan, teaches: generating a second score using a second machine learning model based on a second transaction record in the second training data; in response to determining that the second score falls between the first score threshold and the second score threshold, generating a duplicated second transaction record; assigning a first label and a first weight to the second transaction record; and assigning a second label and a second weight to the duplicated second transaction record. (Liao, [P. 1, Section: 1] “fuzzy augmentation, assigns labels to the rejects based on the scoring model trained by accepted applicants with adjustment made on sample weights, and then retrains the scoring model.” [P. 2, Section: 2 & 2.1] “Consider a set of n loan applications x1,x2,··· ,xn ∈ Rk where k is the number of features. This set includes m accepted applications x1,x2,··· ,xm ∈ Xa with corresponding labels y1,y2,··· ,ym ∈ {Good, Bad} and con sists of xm+1,··· ,xn ∈ Xu whose labels are unknown. The credit scoring model trained with Xa only is denoted as Known Good/Bad (KGB) model. To mitigate sampling bias, reject inference techniques assign labels to unlabeled applications, and combine accepted data and pseudo-labeled data into inferred data sets to represent the whole application population and update credit scoring models. The scoring model with inferred data as training set is denoted as reject inference model (RI model). Figure 1 is an overview of our proposed self-training method pipeline. It starts with training an initial model (also the KGB model in the first iteration) on the accepted data Xa and uses it to predict all the unlabeled data. Then, a confidence model is introduced to filter the most confident predictions whose labels are either good or bad in unlabeled data Xu with a fine-tuned threshold. The selected unlabeled data are labeled in accordance with the predictions, and the training set is augmented with new labeled data, denoted as X1 a. Then RI model is retrained with labeled data X1 a. This process is repeated, and RI model and confidence model update every iteration along with labeled data Xj a. It will stop until no confident applications are identified from confidence model or reaching a pre-defined number of rounds as stop ping criteria.” [P. 4, Section: 3.4] “We adopted two benchmark models in this experiment: a Known Good/Bad model that does not have any sampling correction, and a fuzzy argumentation method as representative of current reject inference techniques. The Known Good/bad XGBoost model is trained with only accepted and funded applicants. Fuzzy argumentation involves assigning labels to unlabeled data based on the KGB model and retrain to get RI model (Montrichard 2007). It assigns unknown data as being partial Good and partial Bad by labels and weights. Every application in Xu is duplicated as two records with two labels y: (1) y1 = Good with weight p(Good); and (2) y2 = Bad with weight p(Bad). The weights p(Good) and p(Bad) are predicted probabilities based on KGB model. The sum of two weights is equal to 1. And accepted applications are also weighted by 1.”) [Examiner’s Note: the fuzzy augmentation applied to unlabeled application, and the KGB model is trained on accepted-labeled data to generate a predicted probability scores (good/bad) for each unknown data. The first assigned label is defined as “Good” with corresponding weight p(Good) and the second assigned label is defined as “Bad” with corresponding weight “p(Bad).”]
The same motivation that was utilized for combining Marcjan and Liao as set forth in claim 3 is equally applicable to claim 6.
Regarding Claim 10,
The claim recites substantially similar limitations as corresponding claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding Claim 13,
The claim recites substantially similar limitations as corresponding claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Regarding Claim 17,
The claim recites substantially similar limitations as corresponding claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding Claim 20,
The claim recites substantially similar limitations as corresponding claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
(Pub. No.: US 20210390385 A1) – “Moein Saleh” relates to “Machine learning module training using input reconstruction techniques and unlabeled transactions.”
[0026] “Computing system 110, in the illustrated embodiment, inputs a first set 112 of transactions with labels and a second set 114 of transactions without labels into machine learning module 120. Labels may indicate, for example, designated classifications for various transactions, or other types of data labels in various other instances. Specifically, in the context of electronic monetary transactions, labels may indicate whether the transaction is considered to be fraudulent or not. In some embodiments, transactions included in the first set 112 and the second set 114 are electronic monetary transactions.”
(Pub. No.: US 20220092472 A1) – “Nitin S. Sharma” relates to “Meta-Learning and Auto-Labeling for Machine Learning.”
This disclosure relates generally to processing data, and, more specifically, to improved techniques for training machine learning models e.g., to classify transactions for transaction security.
(Pub. No.: US 20220164699 A1) – “Piyush Neupane” relates to “Training and Using a Machine Learning Model to Make Predictions.”
The disclosure pertains to using machine learning to make predictions of outcomes of online operations. A feature engineering process is performed on the historical data, so that a machine learning model can learn what outcomes (e.g., transactions that are successfully approved versus transactions that are declined) are associated with what types of features of the transactions. After the machine learning model has been trained, it may be used to predict whether a prospective transaction will be approved.
(Pub. No.: US 20240095742 A1) – “Caitlyn CHEN” relates to “Machine learning for fraud tolerance.”
NPL: Liu, Qiang, et al. "Rmt-net: Reject-aware multi-task network for modeling missing-not-at-random data in financial credit scoring." (2022).
See Section 4: RMT-NET Under Single Rejection/Approval Strategry (Pages 5-6).
NPL: Salazar, Addisson, Gonzalo Safont, and Luis Vergara. "Semi-supervised learning for imbalanced classification of credit card transaction." (2018). See Section II. The Method.
NPL: Guo, Zhiyu, Xiang Ao, and Qing He. "Transductive semi-supervised metric network for reject inference in credit scoring." (2023).
See Figures 1 and 2, and corresponding description.
PNG
media_image1.png
281
539
media_image1.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SADIK ALSHAHARI whose telephone number is (703)756-4749. The examiner can normally be reached Monday - Friday, 9 a.m. 6 p.m. ET.
Examiner interviews are available via telephone, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached on (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.A.A./Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights