DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The following is a Final Office Action in response to applicant’s response on March 17, 2026. Claims 1, 9, and 13 were amended. Claims 1-20 are pending, of which claims 1, 9, and 13 are in independent form.
Response to Amendment
Applicant’s amendments and arguments regarding claims 1, 9, and 13 obviate the claim rejection under 35 USC §101, therefore the claim rejection is withdrawn.
Response to Arguments
Applicant’s arguments with respect to claim(s) are rejected, under 35 USC 103(a), have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter.
On Pages 4-5 of remarks, Applicant argues that JALAL and Cao does not teach “training, by the computing system, based on the training data, a machine learning model to output a risk score for performing a future computing action, wherein the training comprises use of a loss function having input parameters comprising: a percentage risk that the future computing action will fail; and a quantitative measure of at least one computing resource requested for the failed future computing action; wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters;" as claimed in the amended independent claim 1 and similarly claimed in the amended independent claims 9 and 13”. The examiner disagrees. Jalal teaches training a machine learning model using training data to generate a risk score indicative of a probability of an operational loss event occurring in response to execution of a transaction , which constitutes a future computing action. Applicant’s argument that Jalal’s loss function is limited to internal model training and does not relate to predicting futures actions is not supported by the reference is not persuasive. Jalal explicitly discloses predictive risk scoring for future transaction. Further, Jalal teaches use of multiple input data sets in model training. Accordingly, Applicant’s argument is not persuasive.
Further, Applicant’s argument regarding the amended limitation “a quantitative measure of at least one computing resource requested for the failed future computing action;”. Applicant’s arguments, with respect to the rejection(s) of claim(s) 1, 9 and 13 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of JEFFORDS et al . (US 2022/0345457 A1).
Furthermore, regarding the amended limitation “wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters;”, the examiner is relying on Cao to teach said limitation. Cao discloses receiving an inquiry request message…perturbing a value of the transaction parameter and re-analyzing the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter (see, Abstract), which the transaction parameter corresponds to the weight parameter. Accordingly, Applicant’s argument is not persuasive.
As to dependent claims 2-8, 10-12 and 14-20, these claims are rejected by virtue of dependency to independent claims 1, 9, and 13.
Therefore, the examiner maintains the rejection under 35 USC § 103.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL. — The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters;”.
The non-provisional specification fails to provide written description support for the claim limitation of “at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function…”. Given that the limitation of claim 1, for adjusting a machine learning weight parameter, the specification does not provide sufficient detail to demonstrate possession of the claimed feature for adjusting at least one weight parameter to refine a value of a loss function. The disclosure lacks a description as to how such adjustment is performed and fails to describe any specific algorithm or update rules (i.e., the values of the parameter weights 408 may be adjusted, and the machine learning model 404 tested again with the training data 402. The values of the parameter weights 408 may be iteratively adjusted, and the model 404 tested, so as to minimize the loss function 410 until an acceptable loss function value is obtained. The trained model may then be deployed as described herein, in paragraph [0049]) and therefore fails to satisfy the written description the written description requirement.
As a result, the independent claims 9 and 13 are similarly rejected. As to dependent claims 2-8, 10-12 and 14-20, these claims are rejected by virtue of dependency to their independent claims 1, 9, and 13.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over JALAL (US 2022/0108241 A1), hereinafter JALAL in view of Cao et al. (US 11,922,424 B2), hereinafter Cao and further in view of JEFFORDS et al . (US 2022/0345457 A1), hereinafter JEFFORDS.
Regrading claim 1, JALAL discloses a computer-implemented method comprising:
receiving, by a computing system, training data respective of a plurality of previous computing actions by a plurality of entities (JALAL, Para. 0020, a system comprises a first database configured to receive and store a feed of a first dataset for a first entity; a second database configured to receive and store a feed of a second dataset for a second entity; a third database configured to receive and store a feed of a shared dataset accessible to the first entity and the second entity);
training, by the computing system, based on the training data, a machine learning model to output a risk score for performing a future computing action (JALAL, Para. 0009, applying a scoring dataset to a risk predictive model that is trained with a training dataset causing the risk predictive model to generate one or more risk scores based on the scoring dataset), wherein the training comprises use of a loss function having input parameters comprising (JALAL, Para. 0188, an ensemble model may be created by using the bagging method and/or the random forests method. In some embodiments, boosted decision trees use an efficient implementation of the Multiple Additive Regression Trees (MART) gradient boosting algorithm. Gradient boosting is a machine learning technique for regression problems. It builds each regression tree in a step-wise fashion, using a predefined loss function to measure the error in each step and correct for it in the next):
a percentage risk that the future computing action will fail (JALAL, Para. 0216, a predictive model predicted that there is a 9% chance of an operational loss event occurring at time ‘1,’ a 3% chance of an operational loss event occurring at time ‘2,’ a 5% chance of an operational loss event occurring at a time ‘3,’ a 1% chance of an operational loss event occurring at time ‘4,’ a 31.07% chance of an operational loss event occurring at time ‘4,’ and a 50% chance of an operational loss event occurring at time ‘5.’) and (JALAL, Para. 0202, a “third” criteria, in some embodiments, could indicate that a predictive model that has a Recall and/or Precision calculation that is equal to and/or lower than a predetermined threshold (e.g., 30%) would fail to satisfy the criteria, while a predictive model having a Recall and/or Precision calculation that is equal to and/or higher than the predetermined threshold (e.g., 30%) would successfully satisfy the criteria); and
JALAL does not explicitly disclose a quantitative measure of at least one computing resource requested for the failed future computing action;
However, JEFFORDS teaches a quantitative measure of at least one computing resource requested for the failed future computing action (JEFFORDS, Para. 0041, data is parsed, copied, or otherwise computationally extracted from the request 208, and sent to a risk scoring subsystem 214 which then utilizes one or more machine learning models 216 to compute an anomaly-based risk score 218. The risk score may quantify, or otherwise represent, risk arising from possible misuse of the secured item 204, e.g., a reputational, financial, legal, or other risk);
JALAL and JEFFORDS are both considered to be analogous to the claim invention because they are in the same field of training and using a machine learning model to make an accurate decision regarding the previous user actions to prevent any failure or loss of computer system resources in the future. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filing date of the claimed invention to have modified JALAL to incorporate the teachings of JEFFORDS to include a quantitative measure of at least one computing resource requested for the failed future computing action (JEFFORDS, Para. 0041). Doing so would aid to improve the effectiveness and usability of security controls to protect Microsoft computing environments against attacks coming from inside a security perimeter (JEFFORDS, Para. 0019).
JALAL and JEFFORDS do not explicitly disclose wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters;
receiving, by the computing system, from a user entity, a request for permission to engage in further computing actions;
applying, by the computing system, the trained machine learning model to data respective of the user entity to generate a risk score for the further computing actions respective of the user entity; and
rejecting, by the computing system, based on the risk score output by the trained machine learning model, the request for permission.
However, Cao teaches wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters (Cao, Abstract, receiving an inquiry request message…perturbing a value of the transaction parameter and re-analyzing the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter);
receiving, by the computing system, from a user entity, a request for permission to engage in further computing actions (Cao, Col. 5, Lines. 48-58, receiving, by the user device, user input indicating selection of the selectable element; and in response to selection of the selectable element, generating and transmitting, by the user device, the inquiry request message. Clause 14: A system comprising at least one processor programmed or configured to: receive an inquiry request message identifying a first payment transaction having a plurality of transaction parameters and a risk score, wherein the risk score is generated by a machine-learning model based on the plurality of transaction parameters);
applying, by the computing system, the trained machine learning model to data respective of the user entity to generate a risk score for the further computing actions respective of the user entity (Cao, Col. 5, Lines 4-10, wherein re-analyzing the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter is conducted with only a single transaction parameter of the plurality of transaction parameters perturbed at a time) and (Cao, Col. 18, Lines 15-30, the risk score may be a categorical value (e.g., authorized/declined; fraudulent/non-fraudulent) indicating a relative level of risk associated with the payment transaction. The risk score may have been generated by the black box model 114 based on the plurality of transaction parameters for the payment transaction); and
rejecting, by the computing system, based on the risk score output by the trained machine learning model, the request for permission (Cao, Col. 6, Lines 29-32, generate an inquiry response message based on the at least one impact parameter… the system of any of clauses 14-19, wherein the first payment transaction comprises an authorization decision, wherein the authorization decision is a decline authorization decision based on the risk score).
JALAL, JEFFORDS and Cao are all considered to be analogous to the claim invention because they are in the same field of training and using a machine learning model to make an accurate decision regarding the previous user actions to prevent any failure or loss of computer system resources in the future. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filing date of the claimed invention to have modified JALAL and JEFFORDS to incorporate the teachings of Cao to include wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters (Cao, Abstract);
receiving, by the computing system, from a user entity, a request for permission to engage in further computing actions (Cao, Col. 5, Lines. 48-58);
applying, by the computing system, the trained machine learning model to data respective of the user entity to generate a risk score for the further computing actions respective of the user entity (Cao, Col. 5, Lines 4-10) and (Cao, Col. 18, Lines 15-30); and
rejecting, by the computing system, based on the risk score output by the trained machine learning model, the request for permission (Cao, Col. 6, Lines 29-32). Doing so would aid the system, method, and computer program product described herein enhances transparency of the black box machine learning model, enables user monitoring of the model allowing for remediation of errors, enables understanding or learning to be gleaned from the model, enables interpretable justification of the model's decisions, and/or improves upon existing payment processes while doing so in an efficient way that reduces the computing resources necessary for generating model interpretations (Cao, Col. 12, Lines 53-61).
Regrading claim 2, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 1, wherein the further computing actions would use resources controlled by the computing system (JALAL, Para. 0081, different operations may be performed in the method depending on the particular arrangement. In some arrangements, some or all operations of method 600B may be performed by one or more processors (e.g., processor 203A in FIG. 2B) executing on one or more computing devices, systems, or servers).
Regrading claim 3, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 2, wherein the data respective of the user entity comprises data respective of computing actions that used resources controlled by the user entity (JALAL, Para. 0224, the analytics server 1110 a may generate and/or host a website accessible to users operating any of the electronic devices described herein (e.g., end-users or clients), where the content presented via the various webpages may be controlled based upon each particular user's role or viewing permissions).
Regrading claim 4, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 1, wherein the training data comprises: first data respective of computing actions engaged in by the plurality of entities and facilitated by the computing system (JALAL, Para. 0045, each of the predictive models using a training dataset that is specifically generated for a predictive model, based on the type of predictive model. The model management system generates each training dataset using data that the model management system acquires (e.g., receives, retrieves, gathers, etc.) from third-party financial data providers (e.g., exchanges, investment banks, brokers, etc.), third-party event data providers (e.g., national weather forecast, health organizations, news organizations, etc.), internal and/or external groups that are responsible for managing the security for the organization, and/or one or more client devices that are operated by the users associated with of the organization);
second data, from a third party, respective of computing actions by the plurality of entities tracked by the third party (JALAL, Para. 0045, each of the predictive models using a training dataset that is specifically generated for a predictive model, based on the type of predictive model. The model management system generates each training dataset using data that the model management system acquires (e.g., receives, retrieves, gathers, etc.) from third-party financial data providers (e.g., exchanges, investment banks, brokers, etc.), third-party event data providers (e.g., national weather forecast, health organizations, news organizations, etc.), internal and/or external groups that are responsible for managing the security for the organization, and/or one or more client devices that are operated by the users associated with of the organization); and
third data respective of bibliographic information of the plurality of entities (JALAL, Para. 0094, the user performance model is trained based on user attributes (e.g., demographic factors, such as age, sex, education level), work related attributes (e.g., number of paid time away from work taken, when the time off was taken by the employee, medical leave (if any)) and (JALAL, Para. 0179, for a predictive model (e.g., OL model 108a-c, process optimizer model 109, and user performance model 110 in FIG. 1)).
Regrading claim 5, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 1, wherein the data respective of the user entity comprises: first data respective of computing actions engaged in by the user entity and facilitated by the computing system (JALAL, Para. 0045, each of the predictive models using a training dataset that is specifically generated for a predictive model, based on the type of predictive model. The model management system generates each training dataset using data that the model management system acquires (e.g., receives, retrieves, gathers, etc.) from third-party financial data providers (e.g., exchanges, investment banks, brokers, etc.), third-party event data providers (e.g., national weather forecast, health organizations, news organizations, etc.), internal and/or external groups that are responsible for managing the security for the organization, and/or one or more client devices that are operated by the users associated with of the organization);
second data, from a third party, respective of computing actions by user entity tracked by the third party (JALAL, Para. 0045, each of the predictive models using a training dataset that is specifically generated for a predictive model, based on the type of predictive model. The model management system generates each training dataset using data that the model management system acquires (e.g., receives, retrieves, gathers, etc.) from third-party financial data providers (e.g., exchanges, investment banks, brokers, etc.), third-party event data providers (e.g., national weather forecast, health organizations, news organizations, etc.), internal and/or external groups that are responsible for managing the security for the organization, and/or one or more client devices that are operated by the users associated with of the organization); and third data respective of bibliographic information of the user entity (JALAL, Para. 0094, the user performance model is trained based on user attributes (e.g., demographic factors, such as age, sex, education level), work related attributes (e.g., number of paid time away from work taken, when the time off was taken by the employee, medical leave (if any)) and (JALAL, Para. 0179, for a predictive model (e.g., OL model 108a-c, process optimizer model 109, and user performance model 110 in FIG. 1)).
Regrading claim 6, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 1, wherein the loss function determines a logarithm of the quantitative measure (JALAL, Para. 0115, the economic data (sometimes referred to as, “economic statistics”) is data and/or quantitative measures describing an actual economy, past or present. These are typically found in time-series form, that is, covering more than one time period (e.g., the monthly unemployment rate for the last five years) or in cross-sectional data in one time period (e.g., for consumption and income levels for sample households) and (JALAL, Para. 0120, the model management system 104 includes an operational loss model 108 (shown in FIG. 1 as, “OL model 108) that includes an operational loss sub-model 108 a (shown in FIG. 1 as, “OL sub-model 108 a) that is trained to generate a “first” risk score based on the market data and the economic data. The model management system 104 includes an operational loss sub-model 108 b (shown in FIG. 1 as, “OL sub-model 108 b) that is trained to generate a “second” risk score based on the market data. The model management system 104 includes an operational loss sub-model 108 c (shown in FIG. 1 as, “OL sub-model 108 c) that is trained to generate a “third” risk score based on the market data and the security data associated with the organization).
Regrading claim 7, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 1, wherein the user entity is a first user entity, the request is a first request, and the risk score is a first risk score, the method further comprising (JALAL, Para. 0148, the model management process 220A may train the operational loss model 104 to generate a risk score for a client that is associated with the commodities trading sector) and (JALAL, Para. 0143, the model management processor 220A may be configured to extract and/or parse information (e.g., user identifier, a description of a predetermined temporal window, etc.) from a request in response to receiving a request. The model management processor 220A may be configured to determine which of the one or more predictive models the model management system 104 could use (or rely on) to respond to the request):
receiving, by the computing system, from a second user entity, a second request for permission to engage in further computing actions (JALAL, Para. 0148, if the model management process 220A discovers (determines) that a new client is also relying on the operational loss model 104 to generate risk scores, and that the new client is associated with an insurance trading sector, then the model management process 220A can re-train the operational loss model 104 using a new training dataset such that the “newly trained” operational loss model 104 can generate accurate risk scores for both clients) and (JALAL, Para. 0143, the model management processor 220A may be configured to extract and/or parse information (e.g., user identifier, a description of a predetermined temporal window, etc.) from a request in response to receiving a request. The model management processor 220A may be configured to determine which of the one or more predictive models the model management system 104 could use (or rely on) to respond to the request);
applying the trained machine learning model to data respective of the second user entity to generate a second risk score respective of the second user entity; and granting (JALAL, Para. 0148, if the model management process 220A discovers (determines) that a new client is also relying on the operational loss model 104 to generate risk scores, and that the new client is associated with an insurance trading sector, then the model management process 220A can re-train the operational loss model 104 using a new training dataset such that the “newly trained” operational loss model 104 can generate accurate risk scores for both clients), based on the second risk score, the second request for permission; wherein the first user entity is associated with a higher quantitative measure and a lower percentage risk relative to the second user (JALAL, Para. 0240, upon receiving a request to generate results for a first entity, the analytics server may retrieve data specific to that entity. For instance, the analytics server may receive electronic authorization to access one or more databases that include operation data associated and specific to the first entity. In a non-limiting example, the analytics server may use an API to directly communicate with an internal database of the first entity. The analytics server may retrieve data that is specific and sometimes proprietary and confidential to the first entity, such as operation loss, trader data, employee data, and any data that could be used to train the AI models described herein to provide accurate operational loss data).
Regrading claim 8, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 7, further comprising: defining, based on the second risk score, a scope of further computing actions available to the second user entity (JALAL, Para. 0176, the administrator device 103 may be configured to extract one or more risk scores and/or recommendations from the message and present the extracted scores and/or recommendations on a display (e.g., computer display 105) associated with the administrator device 103. The administrator device 103 may be configured to send (e.g., forward, redirect) the message to one or more client devices 102, causing the one or more client devices 102 to present one or more risk scores and/or recommendations on a display associated with the one or more client devices 102).
Regrading claim 9, JALAL discloses a computing system comprising: a processor; and a non-transitory, computer-readable medium storing instructions that, when executed by the processor, cause the computing system to perform operations comprising:
receiving training data respective of a plurality of previous computing actions by a plurality of entities (JALAL, Para. 0020, a system comprises a first database configured to receive and store a feed of a first dataset for a first entity; a second database configured to receive and store a feed of a second dataset for a second entity; a third database configured to receive and store a feed of a shared dataset accessible to the first entity and the second entity);
training, based on the training data, a machine learning model to output a risk score (JALAL, Para. 0009, applying a scoring dataset to a risk predictive model that is trained with a training dataset causing the risk predictive model to generate one or more risk scores based on the scoring dataset), wherein the training comprises use of a loss function having input parameters comprising (JALAL, Para. 0188, an ensemble model may be created by using the bagging method and/or the random forests method. In some embodiments, boosted decision trees use an efficient implementation of the Multiple Additive Regression Trees (MART) gradient boosting algorithm. Gradient boosting is a machine learning technique for regression problems. It builds each regression tree in a step-wise fashion, using a predefined loss function to measure the error in each step and correct for it in the next):
a percentage risk that a future computing action will fail (JALAL, Para. 0216, a predictive model predicted that there is a 9% chance of an operational loss event occurring at time ‘1,’ a 3% chance of an operational loss event occurring at time ‘2,’ a 5% chance of an operational loss event occurring at a time ‘3,’ a 1% chance of an operational loss event occurring at time ‘4,’ a 31.07% chance of an operational loss event occurring at time ‘4,’ and a 50% chance of an operational loss event occurring at time ‘5.’) and (JALAL, Para. 0202, a “third” criteria, in some embodiments, could indicate that a predictive model that has a Recall and/or Precision calculation that is equal to and/or lower than a predetermined threshold (e.g., 30%) would fail to satisfy the criteria, while a predictive model having a Recall and/or Precision calculation that is equal to and/or higher than the predetermined threshold (e.g., 30%) would successfully satisfy the criteria): and
JALAL does not explicitly disclose a quantitative measure of a failed computing action;
However, JEFFORDS teaches a quantitative measure of a failed computing action (JEFFORDS, Para. 0041, data is parsed, copied, or otherwise computationally extracted from the request 208, and sent to a risk scoring subsystem 214 which then utilizes one or more machine learning models 216 to compute an anomaly-based risk score 218. The risk score may quantify, or otherwise represent, risk arising from possible misuse of the secured item 204, e.g., a reputational, financial, legal, or other risk);
JALAL and JEFFORDS are both considered to be analogous to the claim invention because they are in the same field of training and using a machine learning model to make an accurate decision regarding the previous user actions to prevent any failure or loss of computer system resources in the future. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filing date of the claimed invention to have modified JALAL to incorporate the teachings of JEFFORDS to include a quantitative measure of a failed computing action (JEFFORDS, Para. 0041). Doing so would aid to improve the effectiveness and usability of security controls to protect Microsoft computing environments against attacks coming from inside a security perimeter (JEFFORDS, Para. 0019).
JALAL and JEFFORDS do not explicitly disclose wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters; and
causing the trained machine learning model to be deployed in order to grant or deny user requests for permission to engage in further computing actions based on a risk score output by the trained machine learning model using data respective of the users.
However, Cao teaches wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters (Cao, Abstract, receiving an inquiry request message…perturbing a value of the transaction parameter and re-analyzing the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter); and
causing the trained machine learning model to be deployed in order to grant or deny user requests for permission to engage in further computing actions based on a risk score output by the trained machine learning model using data respective of the users (Cao, Col. 4, Lines 43-53, for each transaction parameter of the plurality of transaction parameters, perturb a value of the transaction parameter and re-analyze the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter; determine at least one impact parameter from the plurality of transaction parameters by comparing the perturbed risk scores generated for each of the plurality of transaction parameters) and (Cao, Col. 5, Lines 4-10, wherein re-analyzing the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter is conducted with only a single transaction parameter of the plurality of transaction parameters perturbed at a time) and (Cao, Col. 6, Lines 29-32, generate an inquiry response message based on the at least one impact parameter… the system of any of clauses 14-19, wherein the first payment transaction comprises an authorization decision, wherein the authorization decision is a decline authorization decision based on the risk score).
JALAL, JEFFORDS and Cao are all considered to be analogous to the claim invention because they are in the same field of training and using a machine learning model to make an accurate decision regarding the previous user actions to prevent any failure or loss of computer system resources in the future. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filing date of the claimed invention to have modified JALAL and JEFFORDS to incorporate the teachings of Cao to include wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters (Cao, Abstract); and causing the trained machine learning model to be deployed in order to grant or deny user requests for permission to engage in further computing actions based on a risk score output by the trained machine learning model using data respective of the users (Cao, Col. 4, Lines 43-53) and (Cao, Col. 5, Lines 4-10) and (Cao, Col. 6, Lines 29-32). Doing so would aid the system, method, and computer program product described herein enhances transparency of the black box machine learning model, enables user monitoring of the model allowing for remediation of errors, enables understanding or learning to be gleaned from the model, enables interpretable justification of the model's decisions, and/or improves upon existing payment processes while doing so in an efficient way that reduces the computing resources necessary for generating model interpretations (Cao, Col. 12, Lines 53-61).
Regrading claim 10, the combination of JALAL and JEFFORDS in view of Cao teaches the computing system of claim 9, wherein the training data comprises two or more of: first data respective of computing actions facilitated by a computing service to which the trained model is deployed (JALAL, Para. 0045, each of the predictive models using a training dataset that is specifically generated for a predictive model, based on the type of predictive model. The model management system generates each training dataset using data that the model management system acquires (e.g., receives, retrieves, gathers, etc.) from third-party financial data providers (e.g., exchanges, investment banks, brokers, etc.), third-party event data providers (e.g., national weather forecast, health organizations, news organizations, etc.), internal and/or external groups that are responsible for managing the security for the organization, and/or one or more client devices that are operated by the users associated with of the organization);
second data respective of computing actions tracked by a third party (JALAL, Para. 0045, each of the predictive models using a training dataset that is specifically generated for a predictive model, based on the type of predictive model. The model management system generates each training dataset using data that the model management system acquires (e.g., receives, retrieves, gathers, etc.) from third-party financial data providers (e.g., exchanges, investment banks, brokers, etc.), third-party event data providers (e.g., national weather forecast, health organizations, news organizations, etc.), internal and/or external groups that are responsible for managing the security for the organization, and/or one or more client devices that are operated by the users associated with of the organization); or third data respective of user bibliographic information stored on the computing service (JALAL, Para. 0094, the user performance model is trained based on user attributes (e.g., demographic factors, such as age, sex, education level), work related attributes (e.g., number of paid time away from work taken, when the time off was taken by the employee, medical leave (if any)) and (JALAL, Para. 0179, for a predictive model (e.g., OL model 108a-c, process optimizer model 109, and user performance model 110 in FIG. 1)).
Regrading claim 11, the combination of JALAL and JEFFORDS in view of Cao teaches the computing system of claim 9, wherein the loss function determines a logarithm of the quantitative measure (JALAL, Para. 0115, the economic data (sometimes referred to as, “economic statistics”) is data and/or quantitative measures describing an actual economy, past or present. These are typically found in time-series form, that is, covering more than one time period (e.g., the monthly unemployment rate for the last five years) or in cross-sectional data in one time period (e.g., for consumption and income levels for sample households) and (JALAL, Para. 0120, the model management system 104 includes an operational loss model 108 (shown in FIG. 1 as, “OL model 108) that includes an operational loss sub-model 108 a (shown in FIG. 1 as, “OL sub-model 108 a) that is trained to generate a “first” risk score based on the market data and the economic data. The model management system 104 includes an operational loss sub-model 108 b (shown in FIG. 1 as, “OL sub-model 108 b) that is trained to generate a “second” risk score based on the market data. The model management system 104 includes an operational loss sub-model 108 c (shown in FIG. 1 as, “OL sub-model 108 c) that is trained to generate a “third” risk score based on the market data and the security data associated with the organization).
Regrading claim 12, the combination of JALAL and JEFFORDS in view of Cao teaches the computing system of claim 11, wherein the loss function applies a scaling factor to the logarithm of the quantitative measure (JALAL, Para. 0115, the economic data (sometimes referred to as, “economic statistics”) is data and/or quantitative measures describing an actual economy, past or present. These are typically found in time-series form, that is, covering more than one time period (e.g., the monthly unemployment rate for the last five years) or in cross-sectional data in one time period (e.g., for consumption and income levels for sample households) and (JALAL, Para. 0120, it builds each regression tree in a step-wise fashion, using a predefined loss function to measure the error in each step and correct for it in the next. Thus, the prediction model is an ensemble of weaker prediction models. In regression problems, boosting builds a series of trees in a step-wise fashion, and then selects the optimal tree using an arbitrary differentiable loss function) and (JALAL, Para. 0120, the model management system 104 includes an operational loss model 108 (shown in FIG. 1 as, “OL model 108) that includes an operational loss sub-model 108 a (shown in FIG. 1 as, “OL sub-model 108 a) that is trained to generate a “first” risk score based on the market data and the economic data. The model management system 104 includes an operational loss sub-model 108 b (shown in FIG. 1 as, “OL sub-model 108 b) that is trained to generate a “second” risk score based on the market data. The model management system 104 includes an operational loss sub-model 108 c (shown in FIG. 1 as, “OL sub-model 108 c) that is trained to generate a “third” risk score based on the market data and the security data associated with the organization).
Regrading claim 13, JALAL discloses a computer-implemented method comprising: receiving, by a first computing system, training data respective of a plurality of previous computing actions by a plurality of first users (JALAL, Para. 0020, a system comprises a first database configured to receive and store a feed of a first dataset for a first entity; a second database configured to receive and store a feed of a second dataset for a second entity; a third database configured to receive and store a feed of a shared dataset accessible to the first entity and the second entity);
training, by the first computing system, based on the training data (JALAL, Para. 0009, a machine learning model to output a risk score, wherein the training comprises use of a loss function having input parameters comprising (JALAL, Para. 0009, applying a scoring dataset to a risk predictive model that is trained with a training dataset causing the risk predictive model to generate one or more risk scores based on the scoring dataset):
a percentage risk that a future computing action will fail (JALAL, Para. 0147, if the likelihood is equal to and/or greater than a predetermined threshold (e.g., 80%, 90%, etc.), then the model management process 220A can re-train the predictive model using a second training dataset that is different from the training dataset that was used to deploy the predictive model into production); and
JALAL does not explicitly disclose a quantitative measure of a failed computing action,
However, JEFFORDS teaches a quantitative measure of a failed computing action (JEFFORDS, Para. 0041, data is parsed, copied, or otherwise computationally extracted from the request 208, and sent to a risk scoring subsystem 214 which then utilizes one or more machine learning models 216 to compute an anomaly-based risk score 218. The risk score may quantify, or otherwise represent, risk arising from possible misuse of the secured item 204, e.g., a reputational, financial, legal, or other risk);
JALAL and JEFFORDS are both considered to be analogous to the claim invention because they are in the same field of training and using a machine learning model to make an accurate decision regarding the previous user actions to prevent any failure or loss of computer system resources in the future. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filing date of the claimed invention to have modified JALAL to incorporate the teachings of JEFFORDS to include a quantitative measure of a failed computing action (JEFFORDS, Para. 0041). Doing so would aid to improve the effectiveness and usability of security controls to protect Microsoft computing environments against attacks coming from inside a security perimeter (JEFFORDS, Para. 0019).
JALAL and JEFFORDS do not explicitly disclose wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters;
receiving, by a second computing system, from a second user, a request for permission to engage in further computing actions;
applying, by the second computing system, the trained machine learning model to data respective of the second user to generate a risk score respective of the second user; and
granting, by the second computing system, based on the risk score output by the trained machine learning model, the request for permission.
However, Cao teaches wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters (Cao, Abstract, receiving an inquiry request message…perturbing a value of the transaction parameter and re-analyzing the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter);
receiving, by a second computing system, from a second user, a request for permission to engage in further computing actions (Cao, Col. 5, Lines. 48-58, receiving, by the user device, user input indicating selection of the selectable element; and in response to selection of the selectable element, generating and transmitting, by the user device, the inquiry request message. Clause 14: A system comprising at least one processor programmed or configured to: receive an inquiry request message identifying a first payment transaction having a plurality of transaction parameters and a risk score, wherein the risk score is generated by a machine-learning model based on the plurality of transaction parameters);
applying, by the second computing system, the trained machine learning model to data respective of the second user to generate a risk score respective of the second user (Cao, Col. 4, Lines 43-53, for each transaction parameter of the plurality of transaction parameters, perturb a value of the transaction parameter and re-analyze the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter; determine at least one impact parameter from the plurality of transaction parameters by comparing the perturbed risk scores generated for each of the plurality of transaction parameters) and (Cao, Col. 5, Lines 4-10, wherein re-analyzing the first payment transaction with the machine-learning model to generate a perturbed risk score based on the perturbed transaction parameter is conducted with only a single transaction parameter of the plurality of transaction parameters perturbed at a time); and
granting, by the second computing system, based on the risk score output by the trained machine learning model, the request for permission (Cao, Col. 21, Lines 24-37, the user interface 948 may display transaction history data 950 associated with payment transactions initiated by the user of the user device 102. The transaction history data 950 may include, for example, a transaction identifier 952, a merchant identifier 954, a transaction amount 956, and an authorization status 958. The transaction identifier 952 may comprise a unique identifier that identifies the subject transaction from any other transaction. The merchant identifier 954 may identify the merchant engaged in the subject transaction. The transaction amount 956 may specify the amount exchanged (or intended to be exchanged) between the user and merchant of the subject transaction. The authorization status 958 may identify whether the transaction was authorized or declined).
JALAL, JEFFORDS and Cao are all considered to be analogous to the claim invention because they are in the same field of training and using a machine learning model to make an accurate decision regarding the previous user actions to prevent any failure or loss of computer system resources in the future. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filing date of the claimed invention to have modified JALAL and JEFFORDS to incorporate the teachings of Cao to include wherein at least one weight parameter of the machine learning model is adjusted to refine a value of the loss function for a given request based on the training data and the input parameters (Cao, Abstract); receiving, by a second computing system, from a second user, a request for permission to engage in further computing actions (Cao, Col. 5, Lines. 48-58); applying, by the second computing system, the trained machine learning model to data respective of the second user to generate a risk score respective of the second user (Cao, Col. 4, Lines 43-53); and granting, by the second computing system, based on the risk score output by the trained machine learning model, the request for permission (Cao, Col. 21, Lines 24-37). Doing so would aid the system, method, and computer program product described herein enhances transparency of the black box machine learning model, enables user monitoring of the model allowing for remediation of errors, enables understanding or learning to be gleaned from the model, enables interpretable justification of the model's decisions, and/or improves upon existing payment processes while doing so in an efficient way that reduces the computing resources necessary for generating model interpretations (Cao, Col. 12, Lines 53-61).
Regrading claim 14, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 13, wherein the first computing system is the second computing system (JALAL, Para. 0008, a request for one or more risk scores associated with a plurality of transactions executed by an organization, the one or more risk scores indicating a probability of one or more users instructing execution of a transaction using an incorrect transaction attribute, the transaction causing an operational loss to the organization).
Regrading claim 15, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 13, further comprising: defining, by the second computing system, based on the risk score, a scope of further computing actions available to the second user (JALAL, Para. 0008, by one or more processors, a request for one or more risk scores associated with a plurality of transactions executed by an organization, the one or more risk scores indicating a probability of one or more users instructing execution of a transaction using an incorrect transaction attribute, the transaction causing an operational loss to the organization; applying, by the one or more processors, a scoring dataset to a risk predictive model that is trained with a training dataset causing the risk predictive model to generate one or more risk scores based on the scoring dataset; and sending, by the one or more processors, a message that includes the one or more risk scores to a client device).
Regrading claim 16, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 15, wherein the scope of further computing actions comprises a maximum quantitative measure of failure for a computing action in which the second user is permitted to engage (JALAL, Para. 0020, the “second criteria” could indicate that a predictive model that has an AUC that is equal to and/or lower than a predetermined threshold (e.g., 0.6) would fail to satisfy the criteria).
Regrading claim 17, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 13, wherein the training data comprises: first data respective of computing actions engaged in by the first users via the second computing system(JALAL, Para. 0020, a third database configured to receive and store a feed of a shared dataset accessible to the first entity and the second entity; a server in communication with the first database, the second database, and the third database, the server configured to: train an artificial intelligence model using the first dataset to generate at least one weight factor of the artificial intelligence model trained for the first entity and store the at least one weight factor on the first database; train the artificial intelligence model using the second dataset to generate at least one weight factor of the artificial intelligence model trained for the second entity and store the at least one weight factor on the second database); and
second data respective of bibliographic information of the first users available to the first computing system (JALAL, Para. 00119, The client data may include, for example, transaction data (e.g., transactions performed by the client device), voice and/or visual data associated with the user, data in the device storage, calendar and/or email data associated with the user, client device identifier, any data intercepted by an input/output processor (e.g., input/output processor 205B in FIG. 2B) of the client device 102).
Regrading claim 18, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 13, wherein the second user is one of the first users (JALAL, Para. 00111, a market data provider and an economic data provider, or both entities may be the same entity capable of providing both market data and economic data).
Regrading claim 19, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 13, wherein the further computing actions would use resources controlled by the second computing system (JALAL, Para. 0081, different operations may be performed in the method depending on the particular arrangement. In some arrangements, some or all operations of method 600B may be performed by one or more processors (e.g., processor 203A in FIG. 2B) executing on one or more computing devices, systems, or servers).
Regrading claim 20, the combination of JALAL and JEFFORDS in view of Cao teaches the computer-implemented method of claim 13, wherein the data respective of the second user comprises data respective of computing actions that used resources controlled by the second user (JALAL, Para. 0224, the analytics server 1110 a may generate and/or host a website accessible to users operating any of the electronic devices described herein (e.g., end-users or clients), where the content presented via the various webpages may be controlled based upon each particular user's role or viewing permissions).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GITA FARAMARZI whose telephone number is (571)272-0248. The examiner can normally be reached Monday- Friday 9:00 am- 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jorge L. Ortiz-Criado can be reached at (571)272-7624. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GITA FARAMARZI/Examiner, Art Unit 2496
/JORGE L ORTIZ CRIADO/Supervisory Patent Examiner, Art Unit 2496
Prosecution Insights