Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 7/29/2024, 8/18/2024, and 7/29/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 1-11 objected to because of the following informalities: “the plurality of deep learning/machine learning models” lacks antecedent basis. It is recommended that claim 1 should disclose “a plurality of deep learning/machine learning models” similarly to claim 12. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 12, 13 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1).
Regarding claim 1, Wu discloses “a renewals optimization subsystem identifying variables that impact a target column value;” (See [0026]; Wu uses numeric data of non-target columns as input variables before training)
“the renewals optimization subsystem training each of the plurality of deep learning/machine learning models using a first subset of historical records, wherein training each of the plurality of deep learning/machine learning models comprises each of the plurality of deep learning/machine learning models respectively analyzing each of the first subset of the historical records to identify a relationship between each target column value of the respective historical records in the first subset of historical records and respective variables of the respective historical records in the first subset of historical records;” (See [0026], [0047]; Wu teaches training a machine learning model on records to analyze the relationship between target column values and non-target column values (variables), and also discusses in further detail that a plurality of models are trained during the process 600 of training the models)
“the renewals optimization subsystem implementing each of the plurality of deep learning/machine learning models, as trained, to predict a value of respective known target columns in each of a second subset of historical records based on respective variables in each of the second subset of historical records;” (See [0032]; Wu teaches using a machine learning model to generate predicted values for specific columns using a newly generated set (second subset) of variables)
“a renewals prediction subsystem implementing… deep learning/machine learning models to respectively decide a target column value of a received renewal request based on variables in the received renewal request;” (See [0026]; Wu discloses deciding a target column value)
Wu fails to explicitly disclose, “a renewals selection subsystem identifying respective success rates for each of the plurality of deep learning/machine learning models at predicting the respective known target columns for each of the second subset of historical records,”
“the renewals selection subsystem selecting a subset plurality of the plurality of deep learning/machine learning models based on the success rates;”
“a renewals prediction subsystem implementing each of the subset plurality of the plurality of deep learning/machine learning models”.
Mandal teaches “a renewals selection subsystem identifying respective success rates for each of the plurality of deep learning/machine learning models at predicting the respective known target columns for each of the second subset of historical records,” (See [0081], [0082]; Mandal determines using a model selector 530 that certain models are more suitable than other models in the plurality based on prediction accuracy rates)
“the renewals selection subsystem selecting a subset plurality of the plurality of deep learning/machine learning models based on the success rates;” (See [0081], [0082]; Mandal selects the best models to use by using a model selector 530 to select models based on prediction accuracy rates)
“a renewals prediction subsystem implementing each of the subset plurality of the plurality of deep learning/machine learning models” (See [0086], [0087]; Mandal discloses using the subset plurality of the plurality models to decide a new target column value prediction using a prediction model (Model-2) that uses the subset plurality of plurality models).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu and Mandal before them to modify Wu to identify success rates of models, select subsets of models based on the success rates, and decide target column values based on the selected subset of models. One would be motivated to identify success rates, select model subsets, and decide target column values based on the model subsets to improve target column value prediction, see e.g., Mandal [0082], where Mandal teaches using a model selector to determine a subset with the best models for predicting a target column and Mandal [0086] where Mandal uses a subset with the best models to decide a target column value.
Wu-Mandal fails to explicitly disclose, “the renewals prediction subsystem gathering each respective target column decision from each of the subset plurality of the plurality of deep learning/machine learning models and identifying a majority of the target column value through a democratic process”
“the renewals prediction subsystem deciding the target column value based on the majority”.
Griffin teaches “the renewals prediction subsystem gathering each respective target column decision from each of the subset plurality of the plurality of deep learning/machine learning models and identifying a majority of the target column value through a democratic process” (See [0058]; Griffin discloses using majority voting among a plurality of models to generate a prediction (target column value decision) based on a plurality of predictions from the plurality of models)
“the renewals prediction subsystem deciding the target column value based on the majority” (See [0058]; Griffin discloses deciding predictions based on a majority and could be used to decide a target column value).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal, and Griffin before them to modify Wu-Mandal to gather votes from a plurality of models to identify a target column value majority and decide a target column value based on the majority. One would be motivated to gather votes from a plurality of models and use the majority vote to decide a target column value as another method of predicting the target column value, see e.g., Griffin [0058], where Griffin teaches obtaining votes from a plurality of machine learning models to generate a prediction based on the multiple predictions from the plurality to determine the best prediction.
Regarding claim 13, Wu discloses “wherein the subsystem at least includes a processor.” (See [0022], [0026]; Wu discloses using a processor 410 for a data structure detector 415 and 605, which is used in the subsystem to train models to determine the relationship between target column values and their respective variables)
Regarding claim 12, this claim is similar in scope to claim 1.
Claim Rejections - 35 USC § 103
Claims 2, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Evangelou (US 11227690 B1).
Regarding claim 2, Wu fails to explicitly disclose, “wherein each success rate comprises an accuracy rate… of each of the plurality of deep learning/machine learning models at predicting the respective known target columns for each of the second subset of historical records”.
Mandal teaches “wherein each success rate comprises an accuracy rate… of each of the plurality of deep learning/machine learning models at predicting the respective known target columns for each of the second subset of historical records” (See [0081], [0082]; Mandal discloses determining a success rate using an accuracy rate).
Wu-Mandal-Griffin fails to explicitly disclose, “wherein each success rate comprises an accuracy rate and a confusion metric”.
Evangelou teaches “wherein each success rate comprises an accuracy rate and a confusion metric” (See [Col 11, Lines 60-67]; Evangelou discloses using false positives (confusion metric) and true positives (accuracy rate of correct predictions) to determine a success rate).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Evangelou before them to modify Wu-Mandal-Griffin to include a confusion metric with the success rate. One would be motivated to use the confusion metric to determine the rate of inaccurate predictions while calculating accuracy decide the reliability of a model’s success rate, see e.g., [Col 11, Lines 60-67], where Evangelou considers false positives and true positives when calculating a success rate to determine performance based on its AUC value.
Regarding claim 20, this claim is similar in scope to claim 2.
Claim Rejections - 35 USC § 103
Claims 3, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Ju (US 20240070450 A1).
Regarding claim 3, Wu-Mandal-Griffin fails to explicitly disclose, “wherein the plurality of deep learning/machine learning models comprises logistic regression, decision trees, support vector machines, artificial neural networks, and gradient boosting machines”.
Ju teaches “wherein the plurality of deep learning/machine learning models comprises logistic regression, decision trees, support vector machines, artificial neural networks, and gradient boosting machines” (See [0559]; Ju discloses using machine learning models that comprise logistic regression, decision trees, support vector machines, artificial neural networks, and gradient boosting machines).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Ju before them to modify Wu-Mandal-Griffin to use a ML model with logistic regression, decision trees, support vector machines, artificial neural networks, and gradient boosting machines. One would be motivated to have a variety of different ML model algorithms to use to obtain a plurality of models to select from, see e.g., [0558], where Ju uses multiple machine learning models so that there is no need to retrain a single model.
Regarding claim 14, this claim is similar in scope to claim 3.
Claim Rejections - 35 USC § 103
Claims 4, 5, 15, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Fischer (US 20170344550 A1).
Regarding claim 4, Wu-Mandal-Griffin fails to explicitly disclose, “wherein the target column is a renewal status related to a prescription drug prescribed to a patient”.
Fischer teaches “wherein the target column is a renewal status related to a prescription drug prescribed to a patient” (See [0042]; Fischer discloses storing prescription drug renewals in a target first column).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Fischer before them to modify Wu-Mandal-Griffin to store renewal statuses in the target column. One would be motivated to group the renewal statuses in a single column to be able to access the statuses in a more convenient manner while comparing them to other variables, see e.g., [0042], where Fischer teaches storing patient attributes in a column for comparison and selection.
Regarding claim 5, Wu-Mandal-Griffin fails to explicitly disclose, “wherein the variables comprise the patient’s demographics, the patient’s medical history, a treatment plan, prior authorization history, healthcare provider information, payer information, claims data, utilization data, pharmacy data, lab data, and patient-reported outcomes”.
Fischer teaches “wherein the variables comprise the patient’s demographics, the patient’s medical history, a treatment plan, prior authorization history, healthcare provider information, payer information, claims data, utilization data, pharmacy data, lab data, and patient-reported outcomes” (See [0042]; Fischer discloses storing attributes relating to a patient’s medical condition and other health-related data as data to be accessed as variables).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Fischer before them to modify Wu-Mandal-Griffin to store patient attributes as variables. One would be motivated to store patient attributes as variables to use them when comparing them to other data fields is necessary, see e.g., [0042], where Fischer teaches using patient attributes for comparison with other attributes.
Regarding claim 15, this claim is similar in scope to claim 4.
Regarding claim 16, this claim is similar in scope to claim 5.
Claim Rejections - 35 USC § 103
Claims 6, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Vadakattu (US 20170004513 A1).
Regarding claim 6, Wu-Mandal-Griffin fails to explicitly disclose, “addressing class imbalances among the first subset of historical records by employing synthetic minority oversampling technique to create synthetic datapoints corresponding to a minority class”.
Vadakattu teaches “addressing class imbalances among the first subset of historical records by employing synthetic minority oversampling technique to create synthetic datapoints corresponding to a minority class” (See [0062]; Vadakattu discloses addressing class imbalance among historical data by using Synthetic Minority Oversampling Technique (SMOTE) to create synthetic data that can be used with the historical data to make up for the class imbalance in the historical data).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Vadakattu before them to modify Wu-Mandal-Griffin to create synthetic data to address an imbalance in the dataset. One would be motivated to add more data to the minority classes to decrease the amount of bias that the majority classes could receive, see e.g., [0062], where Vadakattu creates synthetic data to add to the minority class to address the data imbalance between minority and majority classes.
Regarding claim 17, this claim is similar in scope to claim 6.
Claim Rejections - 35 USC § 103
Claims 7, 18 rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Lent (US 20250328972 A1).
Regarding claim 7, Wu-Mandal-Griffin fails to explicitly disclose, “optimizing each of the plurality of deep learning/machine learning models using categorical embedding for any categorical predictors among the variables”.
Lent teaches “optimizing each of the plurality of deep learning/machine learning models using categorical embedding for any categorical predictors among the variables” (See [0040]; Categorical predictors are things like a member’s state or drug name, so use that as a predictor for this and say that they get turned into categorical embeddings (which are like vectors)).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Lent before them to modify Wu-Mandal-Griffin to optimize the models by performing categorical embedding on the variables. One would be motivated to use categorical embedding to convert variables into values that can be calculated or processed, see e.g., [0040], where Lent performs categorical embedding on processed image data to convert it into numerical vector data that can be processed on a numeric scale.
Regarding claim 18, this claim is similar in scope to claim 7.
Claim Rejections - 35 USC § 103
Claims 8, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Lent (US 20250328972 A1), and further in view of Wang (US 11367141 B1).
Regarding claim 8, Wu-Mandal-Griffin fails to explicitly disclose, “optimizing each of a plurality of deep learning/machine learning models further comprises identifying layers of each of a plurality of deep learning/machine learning models, editing error functions of each of a plurality of deep learning/machine learning models, editing epochs of each of a plurality of deep learning/machine learning models, editing average, or maximum pooling, depth of the network or adjusting other hyper parameters of each of a plurality of deep learning/machine learning models”.
Wang teaches “optimizing each of a plurality of deep learning/machine learning models further comprises identifying layers of each of a plurality of deep learning/machine learning models, editing error functions of each of a plurality of deep learning/machine learning models, editing epochs of each of a plurality of deep learning/machine learning models, editing average, or maximum pooling, depth of the network or adjusting other hyper parameters of each of a plurality of deep learning/machine learning models” (See [Col 13, Lines 33-36]; Wang discloses establishing the number of layers, epochs, error functions, and other hyperparameters as things to optimize for a model).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Wang before them to modify Wu-Mandal-Griffin to optimize a model’s hyperparameters, epochs, and layers. One would be motivated to optimize a model’s hyperparameters to improve model performance by reducing the amount of redundant hyperparameters that lengthen runtimes.
Regarding claim 19, this claim is similar in scope to claim 8.
Claim Rejections - 35 USC § 103
Claims 9, 10 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Schuster (US 20200363794 A1).
Regarding claim 9, Wu-Mandal-Griffin fails to explicitly disclose, “communicating the decided target column of the received renewal request to one or more stakeholders, wherein communicating the decided target column further comprises communicating a rationale statement explaining the factors influencing the decided target column value”.
Schuster teaches “communicating the decided target column of the received renewal request to one or more stakeholders, wherein communicating the decided target column further comprises communicating a rationale statement explaining the factors influencing the decided target column value” (See [0087]; Schuster discloses giving an explanation for a prediction and communicates the reasoning as well as factors that disclose what data caused the prediction).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Schuster before them to modify Wu-Mandal-Griffin to show stakeholders the explanations for predictions made by the models as well as the factors that influenced the prediction. One would be motivated to explain to users why a model generated a prediction in a certain outcome as well as showing the factors that caused the generation, see e.g., [0087], where Schuster teaches generating explanations for a performance prediction as well as generating a prediction interpretation that includes what vibration data was considered as the factor for causing the performance prediction.
Regarding claim 10, Wu-Mandal-Griffin fails to explicitly disclose, “the renewals prediction subsystem receiving user feedback regarding the decided target column of the received renewal request, the user feedback indicating whether the decided target column of the received renewal request was correctly or incorrectly decided”.
Schuster teaches “the renewals prediction subsystem receiving user feedback regarding the decided target column of the received renewal request, the user feedback indicating whether the decided target column of the received renewal request was correctly or incorrectly decided” (See [0093]; Schuster discloses receiving feedback from a user that a prediction is incorrect).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Schuster before them to modify Wu-Mandal-Griffin to obtain user feedback on whether a prediction was correct or incorrect. One would be motivated to adjust the model’s training data to further improve the model’s prediction capabilities, see e.g., [0093], where Schuster teaches obtaining user feedback to incorporate it into the model’s training data to improve the model.
Claim Rejections - 35 USC § 103
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 20220382906 A1), in view of Mandal (US 20220374810 A1) and Griffin (US 20230178253 A1), and further in view of Sughrue (US 20210358592 A1).
Regarding claim 11, Wu-Mandal-Griffin fails to explicitly disclose, “wherein the renewals selection subsystem identifies a subset of the plurality of deep learning/machine learning models on a drug-by-drug or procedure-by-procedure basis”.
Sughrue teaches “wherein the renewals selection subsystem identifies a subset of the plurality of deep learning/machine learning models on a drug-by-drug or procedure-by-procedure basis” (See [0084], [0085]; Sughrue discloses selecting a machine learning model subset of one model from a list of selected models based on a patient's response to a particular drug or treatment procedure).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Wu-Mandal-Griffin and Sughrue before them to modify Wu-Mandal-Griffin to select a machine learning model subset from the plurality of models based off of the type of drug or procedure. One would be motivated to determine which drug or procedure works best with a certain subset of models, see e.g., [0085], where Sughrue teaches selecting a certain model based on a patient’s response to a certain drug or medical procedure.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID KIM whose telephone number is (571)272-4331. The examiner can normally be reached 7:30 AM - 4:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.K./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
