DETAILED ACTION
1. This communication is in response to the Application No. 18/430,818 filed on February 2, 2024 in which Claims 1-20 are presented for examination.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
3. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
4. Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
5. Claims 1, 13, and 17 recite “[…] a model output specifying propensity that the user will obtain the first item”. However, it is not clear nor defined by the claim, what exactly comprises such a propensity and/or how such a propensity is calculated simply ‘based on’ received user data and a determined current state of the user without significantly more. Further, Applicant’s specification merely mentions that the propensity is determined by a ‘propensity determination engine’ (See for example, specification Par. [0033]) based on the received data, without further explanation. This renders the claims indefinite and applies to Independent Claims 1, 13, 17 and their respective dependents 2-12, 14-16, and 18-20. Applicant is encouraged to amend the claims to replace the term ‘propensity’ and/or clarify what the propensity indicates and how it is calculated (e.g., computed likelihood, probabilities, etc.)
6. Claims 1, 3-4, 13, 15-17, and 19-20 recite the term “[…] impact […]” with respect to “determining […] an impact on an electronic platform […]” and […] detrimental impacts of changing conditions […]” and additionally “detrimental impact” in Claims 4, 16, and 20. However, it is not clear nor defined by the claim, what exactly comprises such an impact/detrimental impact and/or how such an impact/detrimental impact is calculated/determined simply “based on changing conditions” without significantly more. Further, Applicant’s specification merely mentions the term impact (See for example, specification Par. [0062-0063]) without further explanation. Additionally, “detrimental impact” is a relative term – the term “detrimental” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one or ordinary skill in the art would not be reasonably apprised of the scope of the invention. This renders the claims indefinite and applies to Independent Claims 1, 13, 17 and their respective dependents 2-12, 14-16, and 18-20. Applicant is encouraged to amend the claims to replace the term ‘impact’ and/or clarify how the impact is determined/calculated.
7. The term “unstable” in Claims 5 and 7 is a relative term which renders the claim indefinite. The term “unstable” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Applicant’s specification merely recites the term without providing a standard for ascertaining the requisite degree, as shown by specification Par. [0008-0010] and Par. [0094]. Further, the claims do not define a threshold/degree/requisite to which a feature is considered “unstable” or “stable”, thus rendering the claims indefinite.
8. The term “highest likelihood” in Claims 10-11 is a relative term which renders the claim indefinite. The term “highest likelihood” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The claim states “[…] predicting the next hyperparameter set to test that has a highest likelihood of improving the performance of the machine learning model […]” – however, the claims do not define a threshold/degree/requisite as to what constitutes having a ‘highest likelihood’ of improving the performance of the machine learning model. Applicant’s specification merely recites the limitation without providing a standard for ascertaining the requisite degree, as shown by specification Par. [0012] – thus, rendering the claims indefinite.
9. The terms “most predictive” and “least predictive” in Claim 5 is a relative term which renders the claim indefinite. The terms “most/least predictive” are not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The claim states “[…] ranking the features in the condensed subset of features from most predictive to least predictive” – however, the claims do not define a threshold/degree/requisite as to what is considered ‘most predictive’ as opposed to ‘least predictive’. Applicant’s specification merely recites the limitation without providing a standard for ascertaining the requisite degree, as shown by specification Par. [0079] – thus, rendering the claim indefinite.
Claim Rejections - 35 USC § 101
10. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
11. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Step 1: Claim 1 is a method type claim. Therefore, Claims 1-12 are directed to either a process, machine, manufacture, or composition of matter.
2A Prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation by mathematical calculation but for the recitation of generic computer components, then it falls within the “Mathematical Concepts” grouping of abstract ideas.
determining a current state of the user in the processing pipeline (mental process – determining a current state of a user may be performed manually by a user observing/analyzing received data related to a user and pipeline and accordingly using judgement/evaluation to determine a current state of the user in the processing pipeline based on said analysis. Per Applicant’s specification Par. [0054], the received user data may include where in the processing pipeline the user is with respect to obtaining the item/service, the status of the user in the pipeline, etc. For example, the particular item/service may be a mortgage application with a particular interest rate commitment, hence the current state may be determined manually by a user to indicate the point in the processing pipeline (user makes an initial request for a particular item/service, user submitted an official application requesting the particular item/service, etc.) that the user is at during a particular time)
[…] generate an output specifying a propensity that the particular user will obtain a particular item (mental process – other than reciting “machine learning model”, generating an output specifying a propensity that the particular user will obtain a particular item may be performed manually by a user observing/analyzing the received data and current state and accordingly using judgement/evaluation to cast a prediction specifying a propensity/likelihood that the particular user will obtain a particular item based on said analysis)
performing, based on the propensity that the user will obtain the first item, a corrective action that mitigates for risks of changing conditions and corresponding impact on an electronic platform when the user obtains the first item (mental process – performing a corrective action that mitigates for risks of changing conditions and corresponding impact when the user obtains the first item may be performed manually by a user observing/analyzing the propensity that the user will obtain the first item and accordingly using judgement/evaluation to perform a corrective action that mitigates for risks of changing conditions and corresponding impact when the user obtains the first item. For example, per Applicant’s specification Par. [0063], performing a corrective action may comprise adjusting pricing or availability related to an item on a platform, adjusting hedging ratios to account for mortgage conditions, etc.)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
receiving, via a network interface, data relating to a user and a processing pipeline relating to obtaining a first item (Adding insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g))
inputting the received data and the current state into a machine learning model that is trained to receive such inputs for a particular user […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a trained machine learning model with previously determined data without significantly more)
in response to inputting the received data and the current state, obtaining, from the machine learning model, a model output specifying a propensity that the user will obtain the first item (Adding insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g))
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
receiving, via a network interface, data relating to a user and a processing pipeline relating to obtaining a first item (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
inputting the received data and the current state into a machine learning model that is trained to receive such inputs for a particular user […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a trained machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
in response to inputting the received data and the current state, obtaining, from the machine learning model, a model output specifying a propensity that the user will obtain the first item (MPEP 2106.05(d)(II) indicates that merely “Presenting offers and gathering statistics” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
For the reasons above, Claim 1 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 2-12. The additional limitations of the dependent claims are addressed below.
Regarding Claim 2:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 2 depends on.
determining, from external data, one or more changing conditions, wherein the changing conditions are interest rates regarding the particular item (mental process – determining one or more changing conditions from external data, wherein the changing conditions are interest rates regarding the particular item may be performed manually by a user observing/analyzing the changing conditions/interest rates and accordingly using judgement/evaluation to determine the changing conditions/interest rates based on said analysis)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 3:
Step 2A Prong 1: See the rejection of Claim 2 above, which Claim 3 depends on.
determining, based on the changing conditions, an impact on the electronic platform from the user obtaining the first item at a later point in time (mental process – determining an impact on the platform from the user obtaining the first item at a later point in time may be performed manually by a user observing/analyzing the changing conditions and accordingly using judgement/evaluation to determine an impact on the platform based on said analysis)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 4:
Step 2A Prong 1: See the rejection of Claim 2 above, which Claim 4 depends on.
wherein the corrective action comprises implementing a platform response strategy that offsets detrimental impacts of changing conditions on an electronic platform (mental process – implementing a platform response strategy that offsets detrimental impacts of changing conditions on a platform may be performed manually by a user observing/analyzing the propensity that the user will obtain the first item and accordingly using judgement/evaluation to implement a platform response strategy that offsets detrimental impacts of changing conditions, such as adjusting item pricing/availability and/or adjusting hedging ratios to account for mortgage commitments, as supported by Applicant’s specification Par. [0063])
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 5:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 5 depends on.
generating a condensed subset of features from the set of features (mental process – generating a condensed subset of features from the set of features may be performed manually by a user observing/analyzing the set of features and accordingly using judgement/evaluation to generate a condensed subset of features (with the aid of pen and paper) based on said analysis)
ranking the features in the condensed subset of features from most predictive to least predictive (mental process – ranking the features in the condensed subset of features from most predictive to least predictive may be performed manually by a user observing/analyzing the condensed subset of features and accordingly using judgement/evaluation to rank the features (with the aid of pen and paper) in the condensed subset from most to least predictive based on some predetermined criteria)
removing corelated features from the condensed set of features to obtain an updated set of features (mental process – removing correlated features from the condensed set of features to obtain an updated set of features may be performed manually by a user observing/analyzing the condensed set of features and accordingly using judgement/evaluation to remove correlated features (with the aid of pen and paper) to obtain an updated set of features)
removing unstable features from the updated set of features to obtain a finalized set of features for training the model (mental process – removing unstable features from the updated set of features to obtain a finalized set of features for training the model may be performed manually by a user observing/analyzing the updated set of features and accordingly using judgement/evaluation to remove unstable features (with the aid of pen and paper) from the updated set of features based on some predetermined criteria to obtain a finalized set of features for training the model)
Step 2A Prong 2 & Step 2B:
training the machine learning model, wherein during the training of the model, multiple sets of data are selected using a feature selection process (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner' s note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
receiving a set of features (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
training the model by recursively selecting features from the condensed subset of features until the performance of the model reaches a predetermined threshold (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner' s note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 6:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 6 depends on.
Step 2A Prong 2 & Step 2B:
wherein training data for the predictive machine learning model includes multiple sets of data relating to multiple users and items that users have requested to obtain, and labels in the corresponding set of labels specifying whether the items were obtained along with where the respective user was in in the processing pipeline (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying the training data and labels does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 7:
Step 2A Prong 1: See the rejection of Claim 5 above, which Claim 7 depends on.
computing model contributions for each feature in the condensed set of features using shapley values (mathematical process – computing model contributions for each feature in the condensed set using shapley values may be performed by mathematical process utilizing algorithms/equations for SHAP analysis to compute such shapley values, as supported by Applicant’s specification Par. [0058])
ranking the top features using the model contributions (mental process – ranking the top features may be performed manually by a user observing/analyzing the computed model contributions and accordingly using judgement/evaluation to rank the top features (with the aid of pen and paper) based on said analysis)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 8:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 8 depends on.
determining accuracy of the machine learning model at predetermined time intervals (mathematical process – determining accuracy of the machine learning model at predetermined time intervals may be performed by mathematical process utilizing algorithms/equations for computing a Brier score (see instant claim 9 which supports this interpretation))
Step 2A Prong 2 & Step 2B:
triggering a re-training of the model in response to determining that the accuracy does not satisfy a predetermined threshold (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner' s note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 9:
Step 2A Prong 1: See the rejection of Claim 8 above, which Claim 9 depends on.
determining accuracy of the predictive machine learning model at predetermined time intervals comprises computing a Brier score at predetermined time intervals (mathematical process – determining accuracy of the machine learning model at predetermined time intervals may be performed by mathematical process utilizing algorithms/equations for computing a Brier score)
[…] comparing the Brier score to a predetermined threshold (mental process – comparing the Brier score to a predetermined threshold may be performed manually by a user observing/analyzing the Brier score and the threshold and accordingly using judgement/evaluation to compare the Brier score to the predetermined threshold)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 10:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 10 depends on.
Step 2A Prong 2 & Step 2B:
training the machine learning model, wherein during the training of the model, hyperparameters for the predictive machine learning model are determined using a hyperparameter tuning process comprising: iteratively predicting the next hyperparameter set to test that has a highest likelihood of improving the performance of the machine learning model (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner' s note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 11:
Step 2A Prong 1:
See the rejection of Claim 10 above, which Claim 11 depends on.
Step 2A Prong 2 & Step 2B:
wherein iteratively predicting the next hyperparameter set to test that has the highest likelihood of improving the performance of the machine learning model comprises performing a Bayesian optimization process (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that the iteratively predicting comprises a Bayesian optimization process does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 12:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 12 depends on.
Step 2A Prong 2 & Step 2B:
wherein the machine learning model implements a gradient boosting algorithm (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that the machine learning model implements a gradient boosting algorithm does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Independent Claim 13 recites substantially the same limitations as Claim 1, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
For the reasons above, Claim 13 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 14-16. The additional limitations of the dependent claims are addressed below.
Claim 14 recites substantially the same limitations as Claim 2, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 15 recites substantially the same limitations as Claim 3, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 16 recites substantially the same limitations as Claim 4, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Independent Claim 17 recites substantially the same limitations as Claim 1, in the form of a non-transitory computer-readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
For the reasons above, Claim 17 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 18-20. The additional limitations of the dependent claims are addressed below.
Claim 18 recites substantially the same limitations as Claim 2, in the form of a non-transitory computer-readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 19 recites substantially the same limitations as Claim 3, in the form of a non-transitory computer-readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 20 recites substantially the same limitations as Claim 4, in the form of a non-transitory computer-readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim Rejections - 35 USC § 103
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. Claims 1-4, 6, and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Negi et al. (hereinafter Negi) (US PG-PUB 20200327444), in view of Kallimani et al. (hereinafter Kallimani) (US PG-PUB 20140279404).
Regarding Claim 1, Negi teaches a computer implemented method (Negi, Abstract, “A system and method are presented for customer journey event representation learning and outcome prediction using neural sequence models.”, thus, a computer implemented method is disclosed – see Figure 1 & Par. [0033] which discloses the use of a computer system) comprising:
receiving, via a network interface, data relating to a user (Negi, Par. [0057], “The communicated captured interaction data is received by the server application at the server and analyzed in step 312 using various statistical predictive analytics techniques such as Bayesian inferencing or regression models that are known in the art. The user's interaction pattern is determined by the server application and an interaction prediction is output, such as when a customer will initiate an interaction or may cease to peruse the webpage or website”, therefore, data related to a user (user interaction data) is received via a network interface. This user data is fully detailed by preceding Par. [0056]) and a processing pipeline relating to obtaining a first item (Negi, Par. [0078], “As previously mentioned, customer journey data may be captured in units referred to as events. Each event comprises information such as a unique ID and a timestamp, along with attributes that capture information, including but not limited to: customer profiles, interaction of the customer with the business, online browsing behavior on the website, etc. Customer journey data may further comprise information such as: who the customer is, where the customer is located, what hardware and/or software the customer is using, what items did the customer select on the website (or which buttons did they select), what was filled in on a web form by the customer, keywords used by the customer to perform searches, whether the customer has interacted with an agent, and other details around any such interaction (e.g., customer rating of the interaction, transcripts, outcome, etc.).”, thus, customer journey data, including event data, is also received, where the customer journey data is analogous to a processing pipeline relating to obtaining a first item – this is further outlined by Par. [0076], which states that the predictive outcome may relate to the status of a mortgage application (hence, the customer journey data relates to the process to obtain an item such as a mortgage application, as supported by Applicant’s specification Par. [0054]));
determining a current state of the user in the processing pipeline (Negi, Par. [0100], “Event2vec algorithms are applied to the customer journey data to obtain event vectors 620 for each event (620 a, 620 b, and 620 c). The input layer ingests the corresponding event vector 620 a, 620 b, and 620 c in a timestamp order. Vectors are maintained and updated to summarize the state of the customer journey.”, therefore, a current state of the user in the processing pipeline (event vector for the current time stamp) may be determined);
inputting the received data and the current state into a machine learning model that is trained to receive such inputs for a particular user and generate an output specifying a propensity that the particular user will obtain a particular item (Negi, Par. [0092], “Once the input 501 comprising categorical attributes are converted in an embedding layer 502, they may be combined with the numeric attributes input 503 in the hidden layer 504A. The neural network is then optimized through additional hidden layers 504B to create an output layer 506 whose outputs are used to predict the class of the event 507. After the network is trained, the dense event vector representations 505 are able to be extracted (using standard algorithms) from the hidden layers 504B. The dense event vectors 505 may be used for outcome prediction (described in greater detail below).”, thus, the received data and current state may be inputted into a machine learning model which is trained to receive such inputs for a particular user and generate an output specifying a propensity that the user will obtain a particular item – See Par. [0069-0070] which further details the outcome comprising a propensity for a particular user to obtain an item);
in response to inputting the received data and the current state, obtaining, from the machine learning model, a model output specifying a propensity that the user will obtain the first item (Negi, Par. [0070], “The analytics module 407 takes the inputs and processes the data using the analytics data. The outputs can be machine learned “smart attributes” ranking customers or actions against outcomes defined by the business and within persona clusters assigned by the machine learning or specified by the business associated with the website. Outputs may be interaction recommendations. The routing module 408 may be configured to make interaction recommendations to agents providing the agent with contextual data learned about the users (e.g., likelihood of accepting an interaction offer, likelihood of purchasing a product) based on inputs provided. For example, the analytics module 407 may identify the users that the agent should offer chat interactions to in order to increase product sales.”, therefore, the received data and current state may be inputted into a machine learning model which generates an output specifying a propensity that the user will/is likely to obtain a particular item – this is further supported by preceding Par. [0069] which states “[…] analytics data in various forms (e.g., smart attributes (need for support, propensity to complete an outcome) derived based on the analysis of raw data using machine learning algorithms […]”); and
Negi does not explicitly disclose performing, based on the propensity that the user will obtain the first item, a corrective action that mitigates for risks of changing conditions and corresponding impact on an electronic platform when the user obtains the first item.
However, Kallimani teaches performing, based on the propensity that the user will obtain the first item, a corrective action that mitigates for risks of changing conditions and corresponding impact on an electronic platform when the user obtains the first item (Kallimani, Abstract, “The system obtains input from the property owner, mortgage investor, mortgage servicer, mortgage sub servicer, or other stakeholder adjusting at least one aspect of information about the note used in the automatic valuation and scoring of the note. The system then reports to the owner, agent, investor, servicer, sub servicer, or other stakeholder a refined current valuation and/or scoring of the note (the note's so-called, “Propensity to be Assumed”) that is based on the adjustment of the obtained input.” & Par. [0204], “In step 2108 the system moves on to calculate discounted cash flow for the next period. Step 2109 yields the Propensity adjusted MSR value as the summation of all results from 2107.”, therefore, based on the propensity that the user will obtain the first item (See Kallimani Par. [0220] which details that a neural network is used for propensity scoring), a corrective action that mitigates for risks of changing conditions and corresponding impact on an electronic platform when the user obtains the first item is performed. Applicant’s specification Par. [0030] explicitly discloses examples of such a “corrective action” as “adjusting attributes related to an item on a platform (e.g., pricing, availability, etc.” – hence, adjusting the MSR (mortgage servicing right) value as a result of the propensity, comprises such a corrective action).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the computer implemented method of Claim 1, as disclosed by Negi to include performing, based on the propensity that the user will obtain the first item, a corrective action that mitigates for risks of changing conditions and corresponding impact on an electronic platform when the user obtains the first item, as disclosed by Kallimani. One of ordinary skill in the art would have been motivated to make this modification to enable performing a corrective action, such as the adjustment of attributes related to obtaining a first item, which may mitigate risk and provide more efficient and accurate assumable notes based on propensity (Kallimani, Par. [0090], “In many instances, it may be useful to be able to automatically and accurately determine the value of an assumable mortgage or “note” and to automatically and accurately determine the propensity of an assumable note to be assumed. By being able to accurately determine the value of an existing or outstanding assumable note, this allows for several comparisons, which may create or expose value to various stakeholders in transactions or assessments involving assumable notes. The value of an assumable note and the ability for various stakeholders in the note to be able to recognize or identify value is dependent on both objective and subjective inputs. Hence, a system that may quantify these inputs may add significant opportunity for facilitating and enhancing financial transactions and benefit real estate markets.”)
Regarding Claim 2, Negi in view of Kallimani teaches the computer-implemented method of claim 1, further comprising:
determining, from external data, one or more changing conditions, wherein the changing conditions are interest rates regarding the particular item (Kallimani, Par. [0092], “In one embodiment, an investor in the note may consider factors such as terms and conditions of the note, the interest rate the note was written at, current market interests rates, market conditions, suitability, worthiness of the borrower, the expected lifetime or duration of the note (which for mortgages are typically less than the note's term), convexity, extension risk and various other factors.”, therefore, one or more changing conditions, such as interest rates regarding an item, are determined based on external data (current market conditions, suitability, etc.)).
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 3, Negi in view of Kallimani teaches the computer-implemented method of claim 2, further comprising determining, based on the changing conditions, an impact on the electronic platform from the user obtaining the first item at a later point in time (Kallimani, Par. [0104], “FIG. 1D is a diagram showing how an investor in mortgage servicing rights (MSRs) may utilize the system to identify the propensity of assumption of each loan in the MSR's underlying pool by determining how assumption would affect future servicing cash flows to the investor. The investor in MSRs could use this analysis to more accurately value an MSR and from that decide on whether to purchase or sell an MSR.”, thus, an impact on the electronic platform from the user obtaining the first item in the future is determined based on the changing conditions).
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 4, Negi in view of Kallimani teaches the computer implemented method of claim 2, wherein the corrective action comprises implementing a platform response strategy that offsets detrimental impacts of changing conditions on an electronic platform (Kallimani, Par. [0134], “If the mortgage rate on the assumable mortgage is significantly below current market rates and/or credit conditions are currently constrained in the lending market, by choosing to not invoke or enforce the due-on-sale clause the lender has the potential to create value in the property and at the same time avoid costly foreclosure or workout expenses in some cases. The owner of the non-performing notes or their representative may seek to identify situations, utilizing the system, whereby allowing a note/property to be assumed can offset, to certain degrees, losses being created by non-performance of the mortgage.”, thus, the corrective action may comprise implementing a platform response strategy to offset detrimental impacts (losses) of changing conditions on an electronic platform).
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 6, Negi in view of Kallimani teaches the computer-implemented method of claim 1, wherein training data for the predictive machine learning model includes multiple sets of data relating to multiple users and items that users have requested to obtain, and labels in the corresponding set of labels specifying whether the items were obtained along with where the respective user was in in the processing pipeline (Negi, Par. [0079], “In an embodiment, the use of supervised machine learning methods for predictive analytics requires examples drawn from existing customer journeys comprising the information to be predicted (e.g., whether a business outcome was achieved). The examples are used to train a model capable of making predictions for new customer journeys. The examples comprise raw data logs of the customer journey which are required to be heavily processed before being input into machine learning algorithm(s) for model training and prediction. In addition, the processing steps may vary depending on the raw data format.”, thus, training data for the predictive machine learning model may include multiple sets of data related to multiple users and relevant items and labels specifying where the respective user was in the processing pipeline (supervised learning methods using customer journey data)).
Regarding Claim 13, Negi in view of Kallimani teaches a system comprising: at least one memory storing instructions; a network interface; and at least one hardware processor interoperably coupled with the at least one memory and the network interface, wherein execution of the instructions by the at least one hardware processor causes performance of operations (Negi, Par. [0035], “FIGS. 2A and 2B are diagrams illustrating an embodiment of a computing device as may be employed in an embodiment of the invention, indicated generally at 200. Each computing device 200 includes a CPU 205 and a main memory unit 210. As illustrated in FIG. 2A, the computing device 200 may also include a storage device 215, a removable media interface 220, a network interface 225, an input/output (I/O) controller 230, one or more display devices 235A, a keyboard 235B and a pointing device 235C (e.g., a mouse).”, therefore, a system comprising at least one memory storing instructions, a network interface, and at least one hardware processor coupled with at least one memory and the network interface is disclosed) comprising: […]
The rest of the claim language in Claim 13 recites substantially the same limitations as Claim 1, in the form of a system, therefore it is rejected under the same rationale.
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Claim 14 recites substantially the same limitations as Claim 2 in the form of a system, therefore it is rejected under the same rationale.
Claim 15 recites substantially the same limitations as Claim 3 in the form of a system, therefore it is rejected under the same rationale.
Claim 16 recites substantially the same limitations as Claim 4 in the form of a system, therefore it is rejected under the same rationale.
Regarding Claim 17, Negi in view of Kallimani teaches a non-transitory, computer-readable medium storing computer-readable instructions, that upon execution by at least one hardware processor, cause performance of operations (Negi, Par. [0033], “The computer program instructions are stored in a memory which may be implemented in a computing device using a standard memory device, such as, for example, a RAM. The computer program instructions may also be stored in other non-transitory computer readable media such as, for example, a CD-ROM, a flash drive, etc.”, thus, a non-transitory computer-readable medium storing instructions to be executed by a processor is disclosed), comprising: […]
The rest of the claim language in Claim 17 recites substantially the same limitations as Claim 1, in the form of a system, therefore it is rejected under the same rationale.
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Claim 18 recites substantially the same limitations as Claim 2 in the form of a system, therefore it is rejected under the same rationale.
Claim 19 recites substantially the same limitations as Claim 3 in the form of a system, therefore it is rejected under the same rationale.
Claim 20 recites substantially the same limitations as Claim 4 in the form of a system, therefore it is rejected under the same rationale.
14. Claims 5, 7, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Negi et al. (hereinafter Negi) (US PG-PUB 20200327444), in view of Kallimani et al. (hereinafter Kallimani) (US PG-PUB 20140279404), further in view of Gill et al. (hereinafter Gill) (“A Responsible Machine Learning Workflow with Focus on Interpretable Models, Post-hoc Explanation, and Discrimination Testing”).
Regarding Claim 5, Negi in view of Kallimani teaches the computer-implemented method of claim 1, further comprising training the machine learning model (Negi, Par. [0079], “In an embodiment, the use of supervised machine learning methods for predictive analytics requires examples drawn from existing customer journeys comprising the information to be predicted (e.g., whether a business outcome was achieved). The examples are used to train a model capable of making predictions for new customer journeys. The examples comprise raw data logs of the customer journey which are required to be heavily processed before being input into machine learning algorithm(s) for model training and prediction. In addition, the processing steps may vary depending on the raw data format.”, thus, the machine learning model is trained),
Negi in view of Kallimani does not explicitly disclose wherein during the training of the model, multiple sets of data are selected using a feature selection process comprising:
receiving a set of features;
generating a condensed subset of features from the set of features;
ranking the features in the condensed subset of features from most predictive to least predictive;
training the model by recursively selecting features from the condensed subset of features until the performance of the model reaches a predetermined threshold;
removing corelated features from the condensed set of features to obtain an updated set of features; and
removing unstable features from the updated set of features to obtain a finalized set of features for training the model.
However, Gill teaches wherein during the training of the model, multiple sets of data are selected using a feature selection process comprising:
receiving a set of features (Gill, Pg. 3, Section 2.2 Mortgage Data, “The mortgage dataset analyzed here is a random sample of consumer-anonymized loans from the HDMA database. These loans are a subset of all originated mortgage loans in the 2018 HMDA data that were chosen to represent a relatively comparable group of consumer mortgages. A selection of features is used to predict whether a loan is high-priced, i.e., the annual percentage rate (APR) charged was 150 basis points (1.5%) or more above a survey-based estimate of other similar loans offered around the time of the given loan.”, thus, a set of features are received);
generating a condensed subset of features from the set of features (Gill, Pg. 3, Section 2.2 Mortgage Data, “After data cleaning and preprocessing to encode categorical features and create missing markers, the mortgage data contain ten input features and the binary outcome, high-priced. The data are split into a training set with 160,338 loans and a marker for 5-fold cross-validation and a test set containing 39,662 loans. While lenders would almost certainly use more information than the selected features to determine whether to offer and originate a high-priced loan, the selected input features (loan to value (LTV) ratio, debt to income (DTI) ratio, property value, loan amount, introductory interest rate, customer income, etc.) are likely to be some of the most influential factors that a lender would consider.”, thus, a condensed subset of features is generated from the set of features by preprocessing);
ranking the features in the condensed subset of features from most predictive to least predictive (Gill, Pg. 22, Appendix B.5 Shapley Value Details, “Local, per-instance explanations using Shapley values tend to involve ranking xj ∈ x by φj values or delineating a set of the Xj names associated with the k-largest φj values for some x, where k is some small positive integer, say five. Global explanations are typically the absolute mean of the φj associated with a given Xj across all of the instances in some set X”, therefore, the features may be ranked from most to least predictive – this is similarly supported by Pgs. 25-26 which describe the ranking in terms of importance and also depicts the ranking in Figure 2 on Pg. 10);
training the model by recursively selecting features from the condensed subset of features until the performance of the model reaches a predetermined threshold (Gill, Pg. 3, “The simulated data are then split into a training and test set, with 80,000 and 20,000 instances, respectively. Within the training set, a five-fold cross-validation indicator is used for training all models. For an exact specification of the simulated data, see the software resources referenced in Section 2.8.” & Pg. 3, “The data are split into a training set with 160,338 loans and a marker for 5-fold cross-validation and a test set containing 39,662 loans.”, thus, the model is trained recursively using feature importance/shapley until respective fit measures are fulfilled (predetermined threshold), as shown by Table 1 on Pg. 7) ;
removing corelated features from the condensed set of features to obtain an updated set of features (Gill, Pg. 17, Section 4.3 Discrimination Testing and Remediation in Practice, “Methods can be broadly placed into two groups: more traditional methods that mitigate discrimination by searching across possible algorithmic and feature specifications, and many approaches that have been developed in the last 5–7 years that alter the training algorithm, preprocess training data, or post-process predictions in order to diminish class-control correlations or dependencies.”, thus, corelated features may be removed to obtain an updated set of features used to train the model); and
removing unstable features from the updated set of features to obtain a finalized set of features for training the model (Gill, Pg. 12, “Figure 5 displays global feature importance for gXNN(X) on the mortgage test data. Deep SHAP values are reported in the probability space, after the application of the logit link function. They are also calculated from the projection layer of gXNN. Thus, the Deep SHAP values in Figure 5 are the estimated average absolute impact of each input, Xj, in the projection layer and probability space of gXNN(X) for the mortgage test data. gXNN distributes importance more evenly across business drivers and puts stronger emphasis on the no introductory rate period flag feature than does gMGBM. Like gMGBM, gXNN puts little emphasis on the other flag features. Unlike gMGBM, gXNN assigned higher importance to property value, loan amount, and income, and lower importance on LTV ratio and DTI ratio.”, therefore, unstable features are removed to obtain a finalized set of features for training the model – See Figure 5 which shows the feature ‘no debt to income flag’ which is not considered/removed as it provides no feature importance according to the calculated SHAP value).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of claim 1, as disclosed by Negi in view of Kallimani to include wherein during the training of the model, multiple sets of data are selected using a feature selection process comprising: receiving a set of features; generating a condensed subset of features from the set of features; ranking the features in the condensed subset of features from most predictive to least predictive; training the model by recursively selecting features from the condensed subset of features until the performance of the model reaches a predetermined threshold; removing corelated features from the condensed set of features to obtain an updated set of features; and removing unstable features from the updated set of features to obtain a finalized set of features for training the model.as disclosed by Gill. One of ordinary skill in the art would have been motivated to make this modification to enable feature engineering and training based on feature importance, which may mitigate risks of discrimination and provide a more accurate model (Gill, Pg. 1, Abstract, “For maximum transparency and the potential generation of personalized adverse action notices, the constrained models are analyzed using post-hoc explanation techniques including plots of partial dependence and individual conditional expectation and with global and local Shapley feature importance. The constrained model predictions are also tested for disparate impact and other types of discrimination using measures with long-standing legal precedents, adverse impact ratio, marginal effect, and standardized mean difference, along with straightforward group fairness measures. By combining interpretable models, post-hoc explanations, and discrimination testing with accessible software tools, this text aims to provide a template workflow for machine learning applications that require high accuracy and interpretability and that mitigate risks of discrimination.”).
Regarding Claim 7, Negi in view of Kallimani in view of Gill teaches the computer implemented method of claim 5, wherein removing unstable features from the updated set of features comprises, for each batch in a plurality of batches:
computing model contributions for each feature in the condensed set of features using shapley values (Gill, Pg. 10, “Figure 2. Global mean absolute Tree Shapley Additive Explanations (SHAP) feature importance for gMGBM(X) on the mortgage test data.”, thus, model contributions for each feature are computed using shapley values); and
ranking the top features using the model contributions (Gill, Pg. 10, Figure 2, which depicts the features ranked based on the model contributions/shap values).
The reasons of obviousness have been noted in the rejection of Claim 5 above and applicable herein.
Regarding Claim 12, Negi in view of Kallimani teaches the computer-implemented method of claim 1.
Negi in view of Kallimani does not explicitly disclose wherein the machine learning model implements a gradient boosting algorithm.
However, Gill teaches wherein the machine learning model implements a gradient boosting algorithm (Gill, Pg. 1, Abstract, “The accuracy and intrinsic interpretability of two types of constrained models, monotonic gradient boosting machines and explainable neural networks, a deep learning architecture well-suited for structured data, are assessed on simulated data and publicly available mortgage data.”, thus, the model may implement a gradient boosting algorithm).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of claim 1, as disclosed by Negi in view of Kallimani to include wherein the machine learning model implements a gradient boosting algorithm, as disclosed by Gill. One of ordinary skill in the art would have been motivated to make this modification to enable the use of a gradient boosting algorithm which may provide enhanced predictive accuracy across diverse data and features (Gill, Pg. 1, Abstract, “The accuracy and intrinsic interpretability of two types of constrained models, monotonic gradient boosting machines and explainable neural networks, a deep learning architecture well-suited for structured data, are assessed on simulated data and publicly available mortgage data. For maximum transparency and the potential generation of personalized adverse action notices, the constrained models are analyzed using post-hoc explanation techniques including plots of partial dependence and individual conditional expectation and with global and local Shapley feature importance. The constrained model predictions are also tested for disparate impact and other types of discrimination using measures with long-standing legal precedents, adverse impact ratio, marginal effect, and standardized mean difference, along with straightforward group fairness measures. By combining interpretable models, post-hoc explanations, and discrimination testing with accessible software tools, this text aims to provide a template workflow for machine learning applications that require high accuracy and interpretability and that mitigate risks of discrimination.”)
15. Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Negi et al. (hereinafter Negi) (US PG-PUB 20200327444), in view of Kallimani et al. (hereinafter Kallimani) (US PG-PUB 20140279404), further in view of Vieira et al. (hereinafter Vieira) (“Machine learning models for credit analysis improvements: Predicting low-income families’ default”).
Regarding Claim 8, Negi in view of Kallimani teaches the computer-implemented method of claim 1.
Negi in view of Kallimani do not explicitly disclose:
determining accuracy of the machine learning model at predetermined time intervals; and
triggering a re-training of the model in response to determining that the accuracy does not satisfy a predetermined threshold.
However, Vieira teaches:
determining accuracy of the machine learning model at predetermined time intervals (Vieira, Pg. 6, Section 3.3 Model evaluation and validation procedures, “Thus, the models are evaluated using the following metrics: mean accuracy (MA — total number of hits per total observations), type I error (defaulters classified as non-defaulters), type II error (non-defaulters predicted to be defaulters), AUC, KS test, and Brier Score.”, thus, accuracy of the machine learning model is determined at predetermined time intervals – see Table 3 which depicts the mean accuracy of models for different time thresholds & Pg. 5 Section 3.1.1. which discloses that different time intervals are considered); and
triggering a re-training of the model in response to determining that the accuracy does not satisfy a predetermined threshold (Vieira, Pg. 6, Section 3.2 Techniques involved in the study, “The main Boosting algorithm [85], AdaBoost, takes into account the set of weights over the training examples. Each iteration of the learning algorithm is intended to minimize the weighted error on the training set and return a hypothesis, hl. The weighted error of hl is computed and ap plied to update the weights in the training observations.”, thus, re-training/iterative training of the model is triggered in response to determining that the accuracy does not satisfy a predetermined threshold (calculated error is not minimized)).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of claim 1, as disclosed by Negi in view of Kallimani to include determining accuracy of the machine learning model at predetermined time intervals and triggering a re-training of the model in response to determining that the accuracy does not satisfy a predetermined threshold, as disclosed by Vieira. One of ordinary skill in the art would have been motivated to make this modification to enable model/data drift detection over time which may trigger re-training/iterative training, hence improving model accuracy and performance (Vieira, Pg. 6, Section 3.2 Techniques involved in the study, “The main Boosting algorithm [85], AdaBoost, takes into account the set of weights over the training examples. Each iteration of the learning algorithm is intended to minimize the weighted error on the training set and return a hypothesis, hl. The weighted error of hl is computed and ap plied to update the weights in the training observations. The change in weights places more relevance on training observations that were misclassified by hl and less relevance for examples that were correctly classified. In subsequent iterations, therefore, AdaBoost constructs a model that tackles complex problems by progressively focusing on more difficult observations to classify them.”)
Regarding Claim 9, Negi in view of Kallimani in view of Vieira teaches the computer-implemented method of claim 8, wherein determining accuracy of the predictive machine learning model at predetermined time intervals comprises computing a Brier score at predetermined time intervals and comparing the Brier score to a predetermined threshold (Vieira, Pg. 6, Section 3.3 Model evaluation and validation procedures, “Thus, the models are evaluated using the following metrics: mean accuracy (MA — total number of hits per total observations), type I error (defaulters classified as non-defaulters), type II error (non-defaulters predicted to be defaulters), AUC, KS test, and Brier Score. […] To avoid this problem, we used other validation instruments, such as the KS test and the Brier Score, for all the models. The KS test is used in financial markets as one of the efficiency indicators for credit scoring models, and it is associated with the maximum difference between two accumulated distributions [9]. The Brier score is a function that measures the accuracy of probabilistic predictions. The lower the Brier score for a set of predictions, the better the predictions.”, therefore, determining accuracy comprises computing a Brier score at predetermined time intervals (See Table 3 on Pg. 6) and comparing the Brier score to a predetermined threshold (accuracy/error threshold)).
The reasons of obviousness have been noted in the rejection of Claim 8 above and applicable herein.
16. Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Negi et al. (hereinafter Negi) (US PG-PUB 20200327444), in view of Kallimani et al. (hereinafter Kallimani) (US PG-PUB 20140279404), further in view of Xia et al. (hereinafter Xia) (“A Boosted Decision Tree Approach using Bayesian Hyper-parameter Optimization for Credit Scoring”).
Regarding Claim 10, Negi in view of Kallimani teaches the computer-implemented method of claim 1 further comprising training the machine learning model (Negi, Par. [0079], “In an embodiment, the use of supervised machine learning methods for predictive analytics requires examples drawn from existing customer journeys comprising the information to be predicted (e.g., whether a business outcome was achieved). The examples are used to train a model capable of making predictions for new customer journeys. The examples comprise raw data logs of the customer journey which are required to be heavily processed before being input into machine learning algorithm(s) for model training and prediction. In addition, the processing steps may vary depending on the raw data format.”, thus, the machine learning model is trained),
Negi in view of Kallimani do not explicitly disclose wherein during the training of the model, hyperparameters for the predictive machine learning model are determined using a hyperparameter tuning process comprising: iteratively predicting the next hyperparameter set to test that has a highest likelihood of improving the performance of the machine learning model.
However, Xia teaches wherein during the training of the model, hyperparameters for the predictive machine learning model are determined using a hyperparameter tuning process comprising: iteratively predicting the next hyperparameter set to test that has a highest likelihood of improving the performance of the machine learning model (Xia, Pg. 5, Section 2.3.2 Sequential model-based global optimization (SMBO), “Bayesian hyper-parameter optimization assumes the presence of a noisy black-box function mapping from hyper-parameters to a specific objective (e.g., model performances). The methodology gathers hyper-parameters (input) and the corresponding model performances (observations) to infer information on the unknown function. Particularly, an SMBO algorithm is proposed, as shown in Fig. 2 ( Hutter, Hoos, & Leyton-Brown, 2011 ). SMBO first builds a model S to map the hyper-parameter settings λ to the loss function L (line 1 in Fig. 2 ), and an H (or trails) to record the settings and the corresponding loss. The loss function L is used to evaluate a specific hyper-parameter setting. The trails assist in the inspection inside the Bayesian hyper-parameter optimization, and display the parameter settings and the corresponding evaluations. Then, SMBO iterates the following steps: find the local optimal hyper-parameter settings λ∗ according to the current model S t−1 ; calculate the loss c under the settings λ∗; store λ∗ and the corresponding loss c in H ; and fit a new model S t according to the updated record H . T is the pre-defined maximum iteration. When the loop ends, the SMBO outputs the global optimal hyper-parameter settings with the min- imum c”, thus, a Bayesian optimization process which iteratively predicts the next hyperparameter set to test that has a highest likelihood of improving performance is disclosed).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the training of the machine learning model per claim 10, as disclosed by Negi in view of Kallimani to include wherein during the training of the model, hyperparameters for the predictive machine learning model are determined using a hyperparameter tuning process comprising: iteratively predicting the next hyperparameter set to test that has a highest likelihood of improving the performance of the machine learning model, as disclosed by Xia. One of ordinary skill in the art would have been motivated to make this modification to enable the use of a Bayesian optimization process, which may enhance the hyperparameter tuning process to facilitate more efficient and accurate model performance (Xia, Pg. 1, Abstract, “Third, the hyper-parameters of XGBoost are adaptively tuned with Bayesian hyper-parameter optimization and used to train the model with selected feature subset. Several hyper-parameter optimization methods and baseline classifiers are considered as reference points in the experiment. Results demonstrate that Bayesian hyper-parameter optimization performs better than random search, grid search, and manual search. Moreover, the proposed model outperforms baseline models on average over four evaluation measures: accuracy, error rate, the area under the curve (AUC) H measure (AUC-H measure), and Brier score. The proposed model also provides feature importance scores and decision chart, which enhance the interpretability of credit scoring model.”)
Regarding Claim 11, Negi in view of Kallimani in view of Xia teaches the computer-implemented method of claim 10, wherein iteratively predicting the next hyperparameter set to test that has the highest likelihood of improving the performance of the machine learning model comprises performing a Bayesian optimization process (Xia, Pg. 1, Abstract, “Third, the hyper-parameters of XGBoost are adaptively tuned with Bayesian hyper-parameter optimization and used to train the model with selected feature subset. Several hyper-parameter optimization methods and baseline classifiers are considered as reference points in the experiment. Results demonstrate that Bayesian hyper-parameter optimization performs better than random search, grid search, and manual search.”, thus, a Bayesian optimization process is disclosed, as also outlined by the rejection of Claim 10 above).
The reasons of obviousness have been noted in the rejection of Claim 10 above and applicable herein.
Conclusion
17. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Sirignano et al. (“Deep Learning for Mortgage Risk”) discloses a deep learning model for mortgage credit and prepayment risk, including evaluating borrower propensity for prepayment.
Dorai et al. (US PG-PUB 20110078071) discloses data processing techniques for mortgage applications, including performing corrective actions, such as adjusting mortgage product attributes, based on a calculated likelihood of closing.
18. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Devika S Maharaj whose telephone number is (571)272-0829. The examiner can normally be reached Monday - Thursday 8:30am - 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEVIKA S MAHARAJ/Examiner, Art Unit 2123
Prosecution Insights