DETAILED ACTION
1. This communication is in response to the Application No. 18/397,662 filed on December 27, 2023 in which Claims 1-20 are presented for examination.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
3. The information disclosure statement submitted on 04/25/2024 and 10/10/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 101
4. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Step 1: Claim 1 is a method type claim. Therefore, Claims 1-7 are directed to either a process, machine, manufacture, or composition of matter.
2A Prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation by mathematical calculation but for the recitation of generic computer components, then it falls within the “Mathematical Concepts” grouping of abstract ideas.
[…] predicting actions and behaviors […] (mental process – other than reciting “computer-implemented”, predicting actions and behaviors may be performed manually by a user observing/analyzing historical actions and behaviors and accordingly using judgement/evaluation to cast a prediction regarding actions and behavior based on said analysis)
generating, based on the plurality of prediction goal criteria, a prediction goal (mental process – generating a prediction goal may be performed manually by a user observing/analyzing the received prediction goal criteria and accordingly using judgement/evaluation to generate a prediction goal based on said analysis)
generating a plurality of machine learning features based on data included in a historical data set (mental process – generating a plurality of machine learning features may be performed manually by a user observing/analyzing the data included in a historical data set and accordingly using judgement/evaluation to generate a plurality of machine learning features (i.e., numerical features, categorical features, influential features, directional features, etc.), with the aid of pen and paper, based on said analysis of the historical data)
predicting, via the trained machine learning model, a probability that the prediction goal will be satisfied within a specified time frame (mental process/mathematical process – other than reciting “via the trained machine learning model”, predicting a probability that the prediction goal will be satisfied within a specified time frame may be performed manually by a user observing/analyzing the prediction goal and time frame and accordingly using judgement/evaluation to cast a prediction regarding a probability that the prediction goal will be satisfied within a specified time frame. Alternatively, the predicting a probability may be performed by mathematical process utilizing a mathematical equation/algorithm for determining a probability)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
receiving, from a user via a graphical user interface, a plurality of prediction goal criteria for a machine learning model (Adding insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g))
generating, based on the plurality of machine learning features and the historical data set, a trained machine learning model (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data without significantly more)
[…] via the trained machine learning model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a machine learning model with previously determined data without significantly more)
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
receiving, from a user via a graphical user interface, a plurality of prediction goal criteria for a machine learning model (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
generating, based on the plurality of machine learning features and the historical data set, a trained machine learning model (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
[…] via the trained machine learning model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
For the reasons above, Claim 1 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 2-7. The additional limitations of the dependent claims are addressed below.
Regarding Claim 2:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 2 depends on.
Step 2A Prong 2 & Step 2B:
wherein each of the plurality of prediction goal criteria includes at least one of an event or a property to be predicted (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that each of the plurality of prediction goal criteria includes at least one of an event or a property to be predicted does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 3:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 3 depends on.
Step 2A Prong 2 & Step 2B:
wherein two or more of the plurality of prediction goal criteria are included as operands in a logical operation (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that two or more of the plurality of prediction goal criteria are included as operands in a logical operation does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 4:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 4 depends on.
partitioning, via a training engine, the historical data set into data associated with a training time period and data associated with a testing time period (mental process – other than reciting “via a training engine”, partitioning the historical data set into data associated with a training time period and data associated with a testing time period may be performed manually by a user observing/analyzing the historical data set and accordingly using judgement/evaluation to partition the historical data set (with the aid of pen and paper) into data associated with a training time period and a testing time period)
determining, via the training engine, that the training time period and the testing time period do not overlap in time (mental process – other than reciting “via the training engine”, determining that the training time period and the testing time period do not overlap in time may be performed manually by a user observing/analyzing the training and testing time periods and accordingly using judgement/evaluation to determine that the two time periods do not overlap and/or contain the same time period data)
determining, via the training engine and based on the specified time frame, that the data associated with the testing time period is not right-censored (mental process – other than reciting “via the training engine”, determining that the data associated with the testing time period is not right-censored may be performed manually by a user observing/analyzing the specified time frame and the testing time period data and accordingly using judgement/evaluation to determine that the data associated with the testing time period, based on the specified time frame, is not right-censored (i.e., having a complete data set with all observations belonging to a specific time/time period))
determining, via the training engine and based on the specified time frame, that a duration associated with the training time period is longer than a first predetermined threshold (mental process – other than reciting “via the training engine”, determining that a duration associated with the training time period is longer than a first predetermined threshold may be performed manually by a user observing/analyzing the training time period, specified time frame, and predetermined threshold and accordingly using judgement/evaluation to determine that a duration associated with the training time period is longer than a first predetermined threshold)
determining, via the training engine and based on the specified time frame, that a duration associated with the testing time period is longer than a second predetermined threshold (mental process – other than reciting “via the training engine”, determining that a duration associated with the testing time period is longer than a second predetermined threshold may be performed manually by a user observing/analyzing the testing time period, specified time frame, and predetermined threshold and accordingly using judgement/evaluation to determine that a duration associated with the testing time period is longer than a second predetermined threshold)
Step 2A Prong 2 & Step 2B:
[…] via a training engine […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner's note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 5:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 5 depends on.
calculating, via a training engine, a predictive strength for each of one or more machine learning features (mental process/mathematical process – other than reciting “via a training engine”, calculating a predictive strength for each of one or more machine learning features may be performed manually by a user observing/analyzing each of the one or more machine learning features and accordingly using judgement/evaluation to calculate a predictive strength for each of the one or more machine learning features, based on said analysis. Alternatively, this may be performed by mathematical process utilizing a mathematical equation/algorithm for calculating predictive strength – See Applicant’s specification Par. [0065])
selecting, via the training engine, at least one of the one or more machine learning features for inclusion in the plurality of machine learning features based on the predictive strength (mental process – other than reciting “via the training engine”, selecting at least one of the one or more machine learning features for inclusion in the plurality of machine learning features based on predictive strength may be performed manually by a user observing/analyzing the one or more machine learning features and their respective predictive strengths and accordingly using judgement/evaluation to select at least one of the one or more machine learning features to be included based on said analysis (i.e., features with increased predictive strength may be selected over features with decreased predictive strength))
Step 2A Prong 2 & Step 2B:
[…] via a training engine […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner's note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 6:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 6 depends on.
calculating, via a training engine, a correlation value between two or more machine learning features (mental process/mathematical process – other than reciting “via a training engine”, calculating a correlation value between two or more machine learning features may be performed manually by a user observing/analyzing each of the two or more machine learning features and accordingly using judgement/evaluation to calculate a correlation value between the two or more machine learning features, based on said analysis. Alternatively, this may be performed by mathematical process utilizing a mathematical equation/algorithm for calculating correlation value – See Applicant’s specification Par. [0049])
selecting, via the training engine, at least one of the one or more machine learning features for inclusion in the plurality of machine learning features based on the correlation value (mental process – other than reciting “via the training engine”, selecting at least one of the one or more machine learning features for inclusion in the plurality of machine learning features based on correlation value may be performed manually by a user observing/analyzing the one or more machine learning features and their respective correlation values and accordingly using judgement/evaluation to select at least one of the one or more machine learning features to be included based on said analysis (i.e., features with increased correlation may be selected over features with decreased correlation))
Step 2A Prong 2 & Step 2B:
[…] via a training engine […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner's note: high level recitation of training a machine learning model with previously determined data without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 7:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 7 depends on.
Step 2A Prong 2 & Step 2B:
wherein the historical data set includes at least one of customer data, transaction data, and organizational data (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying the historical data set includes at least one of customer data, transaction data, and organizational data does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Independent Claim 8 recites substantially the same limitations as Claim 1, in the form of a non-transitory computer-readable media, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
For the reasons above, Claim 8 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 9-14. The additional limitations of the dependent claims are addressed below.
Claim 9 recites substantially the same limitations as Claim 2, in the form of a non-transitory computer-readable media, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 10 recites substantially the same limitations as Claim 3, in the form of a non-transitory computer-readable media, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 11 recites substantially the same limitations as Claim 4, in the form of a non-transitory computer-readable media, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 12 recites substantially the same limitations as Claim 5, in the form of a non-transitory computer-readable media, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 13 recites substantially the same limitations as Claim 6, in the form of a non-transitory computer-readable media, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 14 recites substantially the same limitations as Claim 7, in the form of a non-transitory computer-readable media, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Independent Claim 15 recites substantially the same limitations as Claim 1, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
For the reasons above, Claim 15 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 16-20. The additional limitations of the dependent claims are addressed below.
Claim 16 recites substantially the same limitations as Claim 2, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 17 recites substantially the same limitations as Claim 3, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 18 recites substantially the same limitations as Claim 4, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 19 recites substantially the same limitations as Claim 5, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 20 recites substantially the same limitations as Claim 6, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim Rejections - 35 USC § 103
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Achin et al. (hereinafter Achin) (US PG-PUB 20180046926), in view of Duncan et al. (hereinafter Duncan) (US PG-PUB 20170220943).
Regarding Claim 1, Achin teaches a computer-implemented method for predicting actions and behaviors (Achin, Par. [0035], “In some embodiments, the actions of the method further include: determining suitabilities of a plurality of predictive modeling procedures for the prediction problem based, at least in part, on characteristics of the prediction problem and/or on attributes of the respective predictive modeling procedures; selecting one or more predictive modeling procedures from the plurality of predictive modeling procedures based on the determined suitabilities of the selected modeling procedures for the prediction problem; and performing the one or more predictive modeling procedures.”, therefore, a computer-implemented method for predicting actions and behaviors is disclosed – see Achin Claim 1 for further details on the predictive modeling method), the method comprising:
receiving, from a user via a graphical user interface, a plurality of prediction goal criteria for a machine learning model (See introduction of Duncan reference below);
generating, based on the plurality of prediction goal criteria, a prediction goal (Achin, Par. [0198], “Without a well-defined process for consistently applying nested cross-validation, even the most experienced users can omit steps or implement them incorrectly. Thus, the application of a double loop of k-fold cross validation may allow predictive modeling system 100 to simultaneously achieve five important goals: (1) tuning complex models with many hyper-parameters, (2) developing informative derived features, (3) tuning a blend of two or more models, (4) calibrating the predictions of single and/or blended models, and (5) maintaining a pure untouched test set that allows an accurate comparison of different models.” & Par. [0203], “As the exploration engine 110 calculates model performance and eliminates modeling techniques from consideration, predictive modeling system 100 may present the progress of the search space evaluation to the user through the user interface 120 (step 442). In some embodiments, at step 444, exploration engine 110 permits the user to modify the process of evaluating the search space based on the progress of the search space evaluation, the user's expert knowledge, and/or other suitable information.”, thus, a prediction goal (related to accuracy/performance) may be generated based on a plurality of prediction goal criteria, received from a user);
generating a plurality of machine learning features based on data included in a historical data set (Achin, Par. [0054], “In some embodiments, performing feature engineering includes: generating a derived feature based on two or more particular features of the initial dataset having high model-specific predictive values; and adding the derived feature to the initial dataset, thereby generating a second initial dataset. In some embodiments, the actions of the method further include determining that the model-specific predictive values of the particular features are high based on the model-specific predictive values of the particular features being higher than a threshold value and/or based on the model-specific predictive values of the particular features being in a specified percentile of the particular model-specific predictive values for the first and second features of the initial dataset.” & Par. [0394], “Performing each modeling procedure includes fitting the associated predictive model to at least a portion of the initial dataset representing the initial prediction problem. The initial dataset includes prior observations, and each observation generally includes values of at least some of the features of the initial dataset.”, therefore, a plurality of machine learning features are generated based on data included in a historical data set (initial data set comprising prior observations));
generating, based on the plurality of machine learning features and the historical data set, a trained machine learning model (Achin, Claim 1, “(c) identifying one or more of the variables as targets, and identifying zero or more other variables as features; (d) determining a forecast range and a skip range associated with a prediction problem represented by the time-series data, wherein the forecast range indicates a duration of a period for which values of the targets are to be predicted, and wherein the skip range indicates a temporal lag between a time associated with an earliest prediction in the forecast range and a time associated with a latest observation upon which predictions in the forecast range are to be based; […] (g) fitting a predictive model to the training data”, thus, based on a plurality of machine learning features and historical data set (prior observations which are obtained in step (a) of Achin claim 1), a trained machine learning model is generated); and
predicting, via the trained machine learning model, a probability that the prediction goal will be satisfied within a specified time frame (Achin, Par. [0355], “In step 940, a “forecast range” and a “skip range” associated with a prediction problem represented by the time-series data are determined. The forecast range may indicate a duration of a time period for which values of the targets are to be predicted. The skip range may indicate a temporal lag between a time associated with an earliest prediction in the forecast range and a time associated with a latest observation upon which predictions in the forecast range are to be based.” & Par. [0395], “In step 1020, the system 100 determines a first accuracy score of each of the fitted predictive models. The first accuracy score of a fitted model represents the accuracy with which the fitted model predicts one or more outcomes of the initial prediction problem. Any suitable metric and/or technique for determining the accuracy of the model may be used, including, without limitation, testing the model on a holdout portion of the initial dataset.”, thus, a probability that the prediction goal will be satisfied (accuracy score) within a specified time frame (forecast range/skip range) is predicted via the trained/fitted machine learning model).
While Achin teaches the use of a user interface for specifying metrics used to evaluate and compare modeling solutions by specifying criteria for recognizing a suitable modeling solution (See Achin Par. [0203] & Par. [0211]), Achin does not explicitly disclose receiving, from a user via a graphical user interface, a plurality of prediction goal criteria for a machine learning model;
However, Duncan teaches receiving, from a user via a graphical user interface, a plurality of prediction goal criteria for a machine learning model (Duncan, Par. [0081], “The back end components of system 10 collectively define an expert system 11 via which the user 32 can provide input regarding analysis goals (for example, analyses required to fulfil business objectives), receive recommendations as to appropriate data sources and analysis tools for meeting those goals, accept one or more of the recommendations, and execute, via controls provided in user interface 22, one or more of the recommended analysis tools in order to produce summaries, predictions, and/or visualisations based on the data from the recommended data sources and/or from user-defined data sources.”, thus, a plurality of prediction goal criteria for a machine learning model (analysis goals) may be received from a user via a graphical user interface (See Duncan Claim 1 for explicit recitation of the user interface));
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the computer-implemented method for predicting actions and behaviors, as disclosed by Achin to include receiving, from a user via a graphical user interface, a plurality of prediction goal criteria for a machine learning model, as disclosed by Duncan. One of ordinary skill in the art would have been motivated to make this modification to improve accuracy and performance of the machine learning model through the consideration of insights and goals received from the user, which may instead reduce computational burden and fulfill business objectives (Duncan, Par. [0019], “In view of the above, there remains a need for systems and methods which can provide users (such as retailers) with the ability to access, analyse and derive insights from large data sets, without requiring great computational or statistical sophistication of the user. There is also a need in the retail environment for real-time access to customer, sales and inventory data.”).
Regarding Claim 2, Achin in view of Duncan teaches the computer-implemented method of claim 1, wherein each of the plurality of prediction goal criteria includes at least one of an event or a property to be predicted (Achin, Par. [0103], “The user interface 120 provides tools for monitoring and/or guiding the search of the predictive modeling space. These tools may provide insight into a prediction problem's dataset (e.g., by highlighting problematic variables in the dataset, identifying relationships between variables in the dataset, etc.), and/or insight into the results of the search. In some embodiments, data analysts may use the interface to guide the search, e.g., by specifying the metrics to be used to evaluate and compare modeling solutions, by specifying the criteria for recognizing a suitable modeling solution, etc. Thus, the user interface may be used by analysts to improve their own productivity, and/or to improve the performance of the exploration engine 110. In some embodiments, user interface 120 presents the results of the search in real-time, and permits users to guide the search (e.g., to adjust the scope of the search or the allocation of resources among the evaluations of different modeling solutions) in real-time.”, thus, each of the plurality of prediction goal criteria may include at least one of an event or a property to be predicted. Duncan similarly discloses the same in Duncan Par. [0081] which recites that the analysis goals include analyses required to fulfill business objectives).
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 3, Achin in view of Duncan teaches the computer-implemented method of claim 1, wherein two or more of the plurality of prediction goal criteria are included as operands in a logical operation (Achin, Par. [0219], “Users acting as developers may access the builder areas of the interface to create and modify modeling methodologies, techniques, and tasks. As discussed previously, each builder may present one or more tools with different types of user interfaces that perform the corresponding logical operations. In some embodiments, the user interface 120 may permit developers to use a “Properties” sheet to edit the metadata attached to a technique. A technique may also have tuning parameters corresponding to variables for particular tasks. A developer may publish these tuning parameters to the technique-level Properties sheet, specifying default values and whether or not model builders may override these defaults.”, thus, the prediction goal criteria, received by the user interface, may be included as operands in a logical operation).
Regarding Claim 4, Achin in view of Duncan teaches the computer-implemented method of claim 1, further comprising:
partitioning, via a training engine, the historical data set into data associated with a training time period and data associated with a testing time period (Achin, Par. [0028], “In some embodiments, the first subset of observations corresponds to a sliding training window covering a first range of training times and each observation included in the first subset is associated with a time within the first range of training times, […] In some embodiments, the second subset of observations corresponds to a sliding testing window covering a first range of testing times and each observation included in the second subset is associated with a time within the first range of testing times, […]” & Par. [0194], “To facilitate rigorous testing of the predictive models, predictive modeling system 100 may partition the dataset (or suggest a partitioning of the dataset) into a training set and a “holdout” test set.”, thus, a training engine (included within predictive modeling system 100 depicted by Figure 1) partitions a historical data set (prior observations) into data associated with a training time period and data associated with a testing time period)
determining, via the training engine, that the training time period and the testing time period do not overlap in time (Achin, Par. [0029], “In some embodiments, the second testing time range does not overlap any portion of the first training time range, and does not overlap any portion of the second training time range”, thus, the training engine determines that the training time period and testing time period do not overlap in time);
determining, via the training engine and based on the specified time frame, that the data associated with the testing time period is not right-censored (Achin, Par. [0341], “The user may indicate a “skip range” in the data, which is a gap between the end of a training window (e.g., a time range of data used for training) and the start of a validation window (e.g., a time range of data used for validation) or a holdout window (e.g., a time range of data used for holdout testing). In some cases, there is an operational or logistical reason for a delay between the last historical observation and the first forecast.”, thus, based on the specified time frame (validation window), the training engine (of system 100 in Figure 1) may determine that the data associated with the testing time period is not right-censored (holdout testing data with observed target values used to evaluate accuracy – event of interest has occurred by end of operation/testing, hence not right-censored));
determining, via the training engine and based on the specified time frame, that a duration associated with the training time period is longer than a first predetermined threshold (Achin, Par. [0028], “In some embodiments, the first subset of observations corresponds to a sliding training window covering a first range of training times and each observation included in the first subset is associated with a time within the first range of training times, the third subset of observations corresponds to the sliding training window covering a second range of training times and each observation included in the third subset is associated with a time within the second range of training times, and an earliest time in the first range of training times is earlier than an earliest time in the second range of training times.” & Par. [0346], “Depending on the size of the data, the frequency of the data, the length of the skip range, and/or the forecast range, the engine 110 may select lengths for training ranges. […] However, if the amount of variation over longer time periods is low, the engine 110 may shorten the time windows. Or, if there is annual seasonality in the data, the engine 110 use only 3 windows, thereby placing several years of data into each range. Or, if there are a few specific periods within the dataset that exhibit high variation, the engine 110 may divide the data such that each window includes one of these periods.”, therefore, the system may determine that a duration associated with the training time period is longer than a first predetermined threshold and accordingly select lengths for training ranges); and
determining, via the training engine and based on the specified time frame, that a duration associated with the testing time period is longer than a second predetermined threshold (Achin, Par. [0028], “In some embodiments, the second subset of observations corresponds to a sliding testing window covering a first range of testing times and each observation included in the second subset is associated with a time within the first range of testing times, the fourth subset of observations corresponds to the sliding testing window covering a second range of testing times and each observation included in the fourth subset is associated with a time within the second range of testing times, and an earliest time in the first range of testing times is earlier than an earliest time in the second range of testing times. In some embodiments, the first testing time range partially overlaps the second training time range.” & Par. [0373], “In some embodiments, the duration of the testing-input time range is determined based on the total number of observations in the time-series data, the amount of variation over time in values of at least one of the variables, the amount of seasonal variation in values of at least one of the variables, the consistency of variation in values of at least one of the variables over a plurality of time periods, and/or a duration of the forecast range”, therefore, the system may determine that a duration associated with the testing time period is longer than a second predetermined threshold).
Regarding Claim 5, Achin in view of Duncan teaches the computer-implemented method of claim 1, wherein generating the plurality of machine learning features further comprises:
calculating, via a training engine, a predictive strength for each of one or more machine learning features (Achin, Par. [0393], “FIG. 10 shows a method 1000 for determining the predictive value (e.g., “importance”) of one or more features of an initial dataset representing an initial prediction problem. […] In some embodiments, the method 1000 can be used to determine the predictive value of any dataset feature to any predictive model or predictive modeling technique.”, thus, a training engine (See Achin Figure 1 which discloses a system comprising a modeling space exploration engine and a model deployment engine used to perform the methods of Figure 10) calculates a predictive strength value for each of one or more machine learning features); and
selecting, via the training engine, at least one of the one or more machine learning features for inclusion in the plurality of machine learning features based on the predictive strength (Achin, Par. [0399], “In step 1060, the system 100 determines whether to analyze the predictive value of another feature. […] In some embodiments, the system analyzes only a subset of the features in the dataset. Such features can be selected based on any suitable criteria. If the system determines in step 1060 that there is an another feature to analyze, the system then repeats steps 1030-1050 for that feature.”, thus, at least one of the one or more machine learning features may be selected for inclusion in the plurality of features based on the predictive strength value).
Regarding Claim 6, Achin in view of Duncan teaches the computer-implemented method of claim 1, wherein generating the plurality of machine learning features further comprises:
calculating, via a training engine, a correlation value between two or more machine learning features (Achin, Par. [0438], “In general, detecting interactions involves performing a method to detect a statistically significant interaction between two or more features. In some embodiments, the engine 110 may detect interactions upon evaluating the dataset (e.g., at step 408 of the method 400).”, thus, a training engine (See Achin Figure 1 which discloses a system comprising a modeling space exploration engine and a model deployment engine used to perform the method 400) calculates a correlation value (interaction strength) between two or more machine learning features); and
selecting, via the training engine, at least one of the one or more machine learning features for inclusion in the plurality of machine learning features based on the correlation value (Achin, Par. [0040], “In some embodiments, the actions of the method further include: determining a value of a metric that indicates an interaction strength of two or more of the features included in the time-series data; and if the value of the metric exceeds a threshold value, generating time-series values of a new feature based on the values of the two or more features and adding the new feature to the time-series data.”, thus, at least one of the one or more machine learning features may be selected for inclusion in the plurality of features based on the correlation value (interaction strength)).
Regarding Claim 7, Achin in view of Duncan teaches the computer-implemented method of claim 1, wherein the historical data set includes at least one of customer data, transaction data, and organizational data (Achin, Par. [0213], “User interface 120 may include a variety of interface components that allow users to manage multiple modeling projects within an organization, create and modify elements of the modeling methodology hierarchy, conduct comprehensive searches for accurate predictive models, gain insights into the dataset and model results, and/or deploy completed models to produce predictions on new data.”, thus, the historical data (prior observations disclosed by Par. [0394]) may include at least one of organizational data).
Regarding Claim 8, Achin in view of Duncan teaches one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps (Achin, Par. [0477], “In this respect, some embodiments may be embodied as a computer readable medium (or multiple computer readable media) (e.g., a computer memory, one or more floppy discs, compact discs, optical discs, magnetic tapes, flash memories, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments discussed above. The computer readable medium or media may be non-transitory.”, thus, one or more non-transitory computer-readable media storing instructions to be executed by one or more processors is disclosed) of: […]
The rest of the claim language in Claim 8 recites substantially the same limitations as Claim 1, in the form of a non-transitory computer-readable media, therefore it is rejected under the same rationale.
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Claim 9 recites substantially the same limitations as Claim 2 in the form of a non-transitory computer-readable media, therefore it is rejected under the same rationale.
Claim 10 recites substantially the same limitations as Claim 3 in the form of a non-transitory computer-readable media, therefore it is rejected under the same rationale.
Claim 11 recites substantially the same limitations as Claim 4 in the form of a non-transitory computer-readable media, therefore it is rejected under the same rationale.
Claim 12 recites substantially the same limitations as Claim 5 in the form of a non-transitory computer-readable media, therefore it is rejected under the same rationale.
Claim 13 recites substantially the same limitations as Claim 6 in the form of a non-transitory computer-readable media, therefore it is rejected under the same rationale.
Claim 14 recites substantially the same limitations as Claim 7 in the form of a non-transitory computer-readable media, therefore it is rejected under the same rationale.
Regarding Claim 15, Achin in view of Duncan teaches a system comprising: one or more memories storing instructions; and one or more processors for executing the instructions (Achin, Par. [0060], “Other embodiments of this aspect include a predictive modeling apparatus, including a memory configured to store processor-executable instructions; and a processor configured to execute the processor-executable instructions, wherein executing the processor-executable instructions causes the apparatus to perform steps”, thus, a system comprising one or more memories storing instructions and one or more processors for executing the instructions is disclosed) to: […]
The rest of the claim language in Claim 15 recites substantially the same limitations as Claim 1, in the form of a system, therefore it is rejected under the same rationale.
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Claim 16 recites substantially the same limitations as Claim 2 in the form of a system, therefore it is rejected under the same rationale.
Claim 17 recites substantially the same limitations as Claim 3 in the form of a system, therefore it is rejected under the same rationale.
Claim 18 recites substantially the same limitations as Claim 4 in the form of a system, therefore it is rejected under the same rationale.
Claim 19 recites substantially the same limitations as Claim 5 in the form of a system, therefore it is rejected under the same rationale.
Claim 20 recites substantially the same limitations as Claim 6 in the form of a system, therefore it is rejected under the same rationale.
Conclusion
8. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Devika S Maharaj whose telephone number is (571)272-0829. The examiner can normally be reached Monday - Thursday 8:30am - 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEVIKA S MAHARAJ/Examiner, Art Unit 2123
Prosecution Insights