DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to the claimed invention filed on March 13, 2024, in which claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement filed on March 13, 2024, complies with the provisions of 37 CFR 1.97, 1.98 and MPEP § 609. It has been placed in the application file. The information referred to therein has been considered as to the merits.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sinay et al., (hereinafter “Sinay”) US 11,537,934, Barnhill et al., (hereinafter “Barnhill” US 6,789,069 in view Gebremariam et al., (hereinafter “Gebremariam”) US 10,824,694.
As to claim 1, Sinay discloses a computer-based method of generating interpretability data for a target machine learning model by determining sensitive ranges of continuous variables for post-modeling analysis, the method comprising:
receiving, for the target machine learning model, historical data including a series of relevant continuous variables (see col.7, lines 35-67, determine the interpretable model starting with a machine learning algorithm such as XGBoost. As part of this process, the model generator trains the machine learning algorithm using a series of records as training data, each record including values for the input variables as well as the target variable or output for that record. In addition, the monotonic relationships assigned to the input variables are also incorporated into the training data, for example, as meta-data. both classification for a categorical model or target variable or regression for a continuous model or target variable and once the machine learning algorithm is trained, new sets of records can be analyzed using that trained machine learning algorithm. The monotonic relationships assigned to the input variables are also applied during this process, so that the resulting model continues to be an interpretable model).
Barnhill, also discloses the claimed “receiving, for the target machine learning model, historical data including a series of relevant continuous variables (see col.5, 28-54, transformation may comprise optimally categorizing the continuous variable of the training data set, where the training output comprises a continuous variable, the training output may be post-processed by optimally categorizing the training output to derive categorizations from the continuous variable).
However, the combination of Sinay and Barnhill fails to explicitly generate bins for each relevant continuous variable in the series of relevant continuous variables.
Meanwhile, Gebremariam discloses the claimed:
generating bins for each relevant continuous variable in the series of relevant continuous variables (see col.1, lines 40-54 and col.11, line 35-col.12, line 15, a plurality of bins is defined for each variable of the plurality of variables based on the data description determined for each variable of the plurality of variables and based on the number of rare-levels determined for any variable of the plurality of variables identified as the nominal variable type. The defined plurality of bins describes a cumulative distribution function for a respective variable. A transformed value is determined for each variable of the plurality of variables and for each observation vector of the plurality of observation vectors using the plurality of bins defined for a respective variable of the plurality of variables. Each determined transformed value is written to a transformed dataset with a respective observation vector of the plurality of observation vectors);
calculating overall sensitivity values for pairs of neighbor bins (see col.3, lines 36-54, col.11, lines 35-50, computation of the transformed variable values as described herein addresses the critical scalability issues for distributed data. The standard generation of transformed variable values involves distributed sorting, which is known to be one of the most computationally expensive tasks for distributed platforms. However, for both relevant feature selection and model interpretability, the only requirement is that the transformed variable values have approximately the same empirical frequency/cumulative distribution. Computation of the transformed variable values as described herein relies on this insight to solve the problem by computing variable data descriptions that are small packets of information that describe the input features and that are subsequently used to generate the transformed variable values, a plurality of bins is defined for each variable having the interval variable type from the cumulative quantile data, wherein for each bin, a minimum value, a maximum value, a mean value, and a standard deviation value associated with the variable values in each bin are defined for each variable having the interval variable type, wherein a mean value and a standard deviation for each bin of the plurality of bins defined for an interval variable using the values that are included within the boundary of each bin. For example, table 1100, includes a variable name column 1102, a bin identifier column 1104, a lower bin boundary column 1106, an upper bin boundary column 1108, a bin width column 1110, a bin count column 1112, a mean value column 1114, a standard deviation value column 1116, a minimum value column 1118, and a maximum value column);
in response to the calculated overall sensitivity value of respective pairs of neighbor bins being below a predetermined threshold value, merging the respective pairs of neighbor bins (see col.11, lines 53-67 and col.23, lines 30-55, the plurality of bins may be defined for each variable having the interval variable type using the plurality of data descriptions DD.sub.j, j=1, 2, . . . , d determined in operation 216. For example, interval cut point values are determined for each bin with a lower bin boundary for a first bin starting at an overall minimum value, and an upper bin boundary for a last bin ending at an overall maximum value. A first cut point value is an average of a maximum value of the first bin and a minimum value of a second bin, second cut point value is an average of a maximum value of the second bin and a minimum value of a third bin, and so on for each successive bin to define Bin 1 as (overall minimum value, first cut point value), Bin 2 as (first cut point value, second cut point value), . . . , Bin N.sub.b as (N.sub.b−1 cut point value, overall maximum value), where N.sub.b is a number of bins determined by a number of centroids of the cumulative quantile data and the selected variable data description DD.sub.j,w,t is merged with a controller variable data description DD.sub.j, and processing continues in operation 820. For example, a quantile merge as described in in Algorithm 2 on page 12 of Ted Dunning and Otmar Ertl, Computing Extremely Accurate Quantiles Using t-Digests, (2091) published online at https://arxiv.org/pdf/1902.04023.pdf is performed to merge DD.sub.j,w,t into DD.sub.j.); and
generating interpretability data for the target machine learning model based on the merged respective pairs of neighbor bins (see col.11, line 35-col.12, line 30).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the combined system of Sinay and Barnhill to define multiple transformed variable values for training a machine learning model, as disclosed by Gebremariam, in order to reduce many sensitivity numerical value to a smaller variations or noise in the continuous data.
As to claim 2, the combination of Sinay, Barnhill and Gebremariam discloses the invention as claimed. In addition, Tirupathi discloses the claimed wherein the received historical data for the target machine learning model comprises previously processed data, training datasets, or a combination thereof (see par. [0038]).
As to claim 3, the combination of Sinay, Barnhill and Gebremariam discloses the invention as claimed. In addition, Gebremariam discloses the claimed wherein the generated bins for each of the relevant continuous variables in the series of relevant continuous variables comprise equal-width bins, wherein each of the equal-width bins corresponds to a given range of values of an associated continuous variable (see col.1, lines 40-54 and col.11, line 35-col.12, line 15).
As to claim 4, the combination of Sinay, Barnhill and Gebremariam discloses the invention as claimed. In addition, Gebremariam discloses the claimed wherein the calculated overall sensitivity values are derived from calculated component variables related to sensitivity of specific ranges of values of respective continuous variables within the series of relevant continuous variables with respect to a target value to be predicted or output by the target machine learning model (see col.1, lines 40-54 and col.11, line 35-col.12, line 15).
As to claim 5, the combination of Sinay, Barnhill and Gebremariam discloses the invention as claimed. In addition, Gebremariam discloses the claimed wherein the calculated overall sensitivity values are derived from calculated component variables including, at least, target variance values, range correlation values, and set correlation values for the pairs of neighbor bins (col.11, line 35-col.12, line 30).
As to claim 6, the combination of Sinay, Barnhill and Gebremariam discloses the invention as claimed. In addition, Gebremariam discloses the claimed wherein the generated interpretability data depicts sensitive ranges of selected continuous variables from the series of continuous variables with respect to a given target to be predicted or output by the target machine learning model (col.11, line 35-col.12, line 30).
As to claim 7, the combination of Sinay, Barnhill and Gebremariam discloses the invention as claimed. In addition, Gebremariam discloses the claimed wherein the generated interpretability data comprises a bar chart having a line appended thereto, the line corresponding to the calculated overall sensitivity values for different values of respective selected continuous variables from the series of relevant continuous variables with respect to the given target to be predicted or output by the target machine learning model (see, col.3, lines 30-54, machine learning interpretability and performance is also well established. In these approaches, local and global explanations are produced by systematically generating and consuming transformed variable values of a predefined number of records along with the rest of the variables in an approach-specific way and computation of the transformed variable values as described herein addresses the critical scalability issues for distributed data. The standard generation of transformed variable values involves distributed sorting, which is known to be one of the most computationally expensive tasks for distributed platforms. However, for both relevant feature selection and model interpretability, the only requirement is that the transformed variable values have approximately the same empirical frequency/cumulative distribution. Computation of the transformed variable values as described herein relies on this insight to solve the problem by computing variable data descriptions that are small packets of information that describe the input features and that are subsequently used to generate the transformed variable values. This enables computation of the transformed variable values as described herein to be scalable and distributed unlike existing methods).
As to claims 8-14, claims 8-14 are system for performing for the method of claims 1-7. They are rejected under the same rationale.
As to claims 15-20, claims 15-20 are computer program product having program instructions the method of claims 1-7. They are rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 2025022372 (involved in monitoring and retraining techniques for automatically triggering model retraining based on evaluation scores).
US 20230342671 (involved in executing the plurality of models on test data and determining accuracy values and interpretability toughness values for the plurality models, selecting a most accurate model as a core model based on the accuracy values and select a most interpretable model as a surrogate model from among other models remaining in the plurality of models based on the interpretability toughness values, building a composite model comprising the core model, the surrogate model, and instructions for generating a debrief model for debriefing the core model based on a combination of the core model and the surrogate model).
US 20200193313 (involved in interpretability-based machine learning adjustment during production).
US20200097439 (involved in generating models that are more interpretable and transparent than existing machine learning approaches).
US 20150254568 (involved in optimizing the bin weights such that the prediction error for the model is minimized, predict both continuous and binary targets. Scorecards are generally characterized as being much more powerful than regression models, and still simple enough to be interpretable).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEAN M CORRIELUS whose telephone number is (571)272-4032. The examiner can normally be reached Monday-Friday 6:30a-10p(Midflex).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J Lo can be reached at (571)272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEAN M CORRIELUS/Primary Examiner, Art Unit 2159 October 6, 2026
Prosecution Insights