DETAILED ACTION
Acknowledgement
This final office action is in response to the amendment filed on 02/17/2026.
Status of Claims
Claims 2,11, and 20 have been canceled.
Claims 1, 3, 10, 12, and 19 have been amended.
Claims 21-23 have been added.
Claims 1, 3-10, 12-19, and 21-23 are now pending.
Response to Arguments
The 35 U.S.C. 101 “signal per se” rejection of claims 19-20 is withdrawn in light of amendments.
Applicant's arguments filed on 02/17/2026 regarding the 35 U.S.C. 101, 102, and 103 rejections of the claims have been fully considered. The Applicant argues the following:
(1) As per the 101 rejection, the Applicant argues, in summary, that (i) claim 1, as amended, does not recite an abstract idea because a human mind is not equipped to train and execute such models across the volume of interactions and features contemplated, nor to compute PDP-based feature-change quantities, as a practical matter; (ii) the claims integrate the alleged judicial exception into a practical application because the claimed features incorporate specific configurations and operations that apply, rely on, or use any alleged judicial exception with meaningful limitation; and (iii) the claims recite an inventive concept that is significantly more than any alleged abstract idea because they require a specific, non-generic implementation that constrains how key performance metrics, methods and systems of the application are evaluated and implemented for predicting KPIs from customer-agent interactions.
The Examiner respectfully disagrees with all arguments. As per argument (i), the Examiner submits that the claims are directed to Mental Processes based on the abstract limitations of claims 1, 10, and 19 as listed in Step 2A(1). These limitations describe a process of receiving and analyzing KPI metric, KPI target and customer-agent interaction data via modelling in order to predict a value for the KPI metric and determine features of the customer-agent interaction that would improve and/or meet the KPI target and output results, which can practically be performed in the human mind with pen and paper. Per MPEP 2106.04(a), a claim recites a judicial exception when the judicial exception is “set forth” or “described” in the claim. The claims do not a specific type of interaction data, the volume of data, and a specific way of training the classifier model that would preclude being performed in the mind with pen and paper. Predicting KPI values from recorded customer-agent interactions and determining feature-level improvement amounts can be performed mentally via observation (listening/reading/looking) and evaluation (measuring, valuating, rating/scoring, assessing). A human can train a mathematical model via tuning and adjustment of parameters to fit a data set. The training is just being performed in the computer environment. The Examiner is not stating that the human mind can perform machine learning, as machine learning is listed as an additional element in Step 2A(2) and 2B.
As per arguments (ii) and (iii), the Examiner submits that the additional elements recited in the claims and listed in Steps 2A(2) and 2B do not integrate the abstract idea into a practical application or amount to significantly more because the additional elements are viewed as mere instructions to apply or implement the abstract idea on a computer. Applying an abstract idea on a computer does not integrate a judicial exception into a practical application or provide an inventive concept (see MPEP 2106.05(f)). The claimed features highlighted in the Applicant’s arguments are abstract features which cannot furnish the technological improvement or inventive concept. The claims reflect using computer-implemented technology (e.g. machine learning) to perform an abstract process (e.g. predicting a KPI metric). KPI represents a data element. The claimed improvement is in the KPI determination and optimization, which reflects an improvement in the abstract idea itself. As per MPEP 2106.05, an improvement in the abstract idea is not an improvement in technology. Therefore, the 35 U.S.C. 101 rejection is maintained.
(2) As per the 102 and 103 rejections, the Applicant argues that Minter and Basu, whether individually or in combination, does not disclose each and every element recited in amended independent claim 1 and similarly 10 and 19.
The Examiner somewhat disagrees. The Examiner submits that Minter alone does not teach every element in amended claims 1, 10, 19. Therefore, the previous 35 U.S.C. 102 rejection is withdrawn. However, because the amended limitations include limitations from dependent claims 2 and 4, the Examiner submits that the combination of Minter and Basu teach every element of amended claims 1, 10, and 19. Therefore, claims 1, 10, and 19 are now rejected under 35 U.S.C. 103. See details below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-10, 12-19, and 21-23 are rejected under 35 U.S.C. 101 because the claimed invention, “Systems and Methods for Key Performance Index Prediction and Improvement Through Feature Analysis”, is directed to an abstract idea, specifically Mental Processes, without significantly more. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements individually or in combination provide mere instructions to implement the abstract idea on a computer.
Step 1: Claims 1, 3-10, 12-19, and 21-23 are directed to a statutory category, namely a process (claims 1, 3-9, and 21), a machine (claims 10, 12-18, and 22), and a manufacture (claims 19 and 23).
Step 2A (1): Independent claims 1, 10, and 19 are directed to an abstract idea of Mental Processes based on the following claim limitations: “a method for providing a predicted key performance indicator (KPI) value, comprising: receiving/receive a KPI metric and a target value for the KPI metric; receiving/receive one or more sets of predefined features that are determined to be controllable by an agent when engaged in a customer-agent interaction data; implementing/implement a model corresponding to the KPI metric from a plurality of models, wherein the model is trained to predict a value for the KPI metric; predicting/predict, with the model, the value for the KPI metric based on a plurality of features that the model identifies and measures from a plurality of suggested agent- controllable features of the customer-agent interaction data; determining/determine, from the plurality of features, one or more features having a potential for improvement based on measured values of the plurality of features determined by the model; identifying a first subset of features from the one or more features that are not controllable by the agent when engaged in the customer-agent interaction; generating a second set of features by filtering out the first subset of features from the one or more features based on the one or more sets of predefined features determined to be in control of the agent; determining/determine that improvement in a measured value of the one or more features maintains or increases the value of the KPI metric; outputting/output a first indication of whether the value predicted for the KPI metric meets the target value and a second indication of the one or more features that maintains or increases the value of the KPI metric when the one or more features are improved; and outputting a report indicating the value predicted for the KPI metric, the one or more features having the potential for improvement, and an amount that each of the one or more features is to improve such that the value predicted for the KPI metric meets the target value. ”. These claim limitations describe a process of receiving and analyzing KPI metric, KPI target and customer-agent interaction data via modelling in order to predict a value for the KPI metric and determine features of the customer-agent interaction that would improve and/or meet the KPI target and output results, which can practically be performed in the human mind with pen and paper. Dependent claims 3-9, 12-18, and 21-23 further describe the process of determining features, predicting the KPI metric value via modelling, identifying feature improvements, and outputting results. Therefore, these limitations, under the broadest reasonable interpretation, fall within the abstract groupings of Mental Processes which include concepts performed in the human mind such as observations, evaluations, judgments, and opinions. Mental Processes include claims directed to collecting information, analyzing it, and displaying certain results of the collection and analysis even if they are claimed as being performed on a computer. The courts have found claims requiring a generic computer or nominally reciting a generic computer may still recite a mental process even though the claim limitations are not performed entirely in the human mind. Therefore, claims 1, 3-10, 12-19, and 21-23 are directed to an abstract idea and are not patent eligible.
Step 2A (2): The claims as a whole do not integrate this abstract idea into a practical application. In particular, claims 10, 19, and 21-23 recite additional elements of “An apparatus configured …, comprising: one or more memories comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions and cause the apparatus to (claim 10); a computer program product …, the computer program product comprising instructions, which when executed by a computer, cause the computer to carry out steps (claim 19); and the classifier model is a machine learning-based model (claims 21-23) ”. These additional elements do not integrate the abstract idea into a practical application because the claims do not recite (a) an improvement to another technology or technical field and (b) an improvement to the functioning of the computer itself and (c) implementing the abstract idea with or by use of a particular machine, (d) effecting a particular transformation or reduction of an article, or (e) applying the judicial exception in some other meaningful way beyond generally linking the use of an abstract idea to a particular technological environment. These additional elements evaluated individually and in combination are viewed as computing devices that are used to perform the abstract idea identified in Step 2A(1). The use of machine learning and trained models are considered instructions to apply or implement a model on a computer. Limitations that recite mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform an abstract idea are not indicative of integration into a practical application (see MPEP 2106.05(f)). Therefore, claims 1, 3-10, 12-19, and 21-23 as a whole do not include individual or a combination of additional elements that integrate the abstract idea into a practical application and thus are not patent eligible.
Step 2B: The claims as a whole do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Claims 10, 19, and 21-23 recite additional elements of “An apparatus configured …, comprising: one or more memories comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions and cause the apparatus to (claim 10); a computer program product …, the computer program product comprising instructions, which when executed by a computer, cause the computer to carry out steps (claim 19); and the classifier model is a machine learning-based model (claims 21-23) ”. These additional elements evaluated individually and in combination are viewed as mere instructions to apply or implement the abstract idea on a computer. Applying an abstract idea on a computer does not integrate a judicial exception into a practical application or provide an inventive concept (see MPEP 2106.05(f)). Therefore, claims 1, 3-10, 12-19, and 21-23 as a whole do not include individual or a combination of additional elements that are sufficient to amount to significantly more than the abstract idea and thus are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-7, 9-10, 12-16, 18-19, and 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over Minter (US 2022/0245557 A1) in view of Basu et al. (US 11,087,261 B1).
As per claims 1, 10, and 19 (Currently Amended), Minter teaches a method for providing a predicted key performance indicator (KPI) value, comprising; an apparatus configured for providing a predicted key performance indicator (KPI) value, comprising: one or more memories comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions and cause the apparatus to; a computer program product for providing a predicted key performance indicator (KPI) value, the computer program product comprising instructions, which when executed by a computer, cause the computer to carry out steps comprising (Minter e.g. The systems and processes described herein enable detection of metrics, actions or both and automatically deliver actions, such as coaching and automated learning, to target result effective variables to change future outcomes [0025]. The computing system may have one or more non-transitory computer-readable storage media storing instructions executable by the one or more processors to perform operations that determine that a performance of a set of employees is to be improved with respect to at least one metric based in part on the data gathered in a first time period [0029].):
Minter teaches receiving/receive a KPI metric and a target value for the KPI metric; (Minter e.g. The monitoring servers may operate one or more monitoring services that monitor the agent computing devices and agent displays to collect metrics (Abstract). A method according to an implementation of the invention may include receiving metrics data and storing the metrics data in a database [0034]. FIG. 1 depicts a computing system 110 that may form the back end of a corporate human resources management department. The computing system 110 collects data from multiple sources that may characterize the company workforce and provide metrics of performance [0044]. The corporate work force may comprise multiple levels in a hierarchy with, for example, agents on the front line, and managers, senior managers, and executives above the agents in that order in the hierarchy. Each level may have its own respective metrics of performance [0044]. These systems and processes may be powered by multiple types of machine learning to identify the result effective variables, generate target metrics, and automate actions to be taken based on a variance of actual metrics from the target metrics [0025].)
Minter teaches receiving/receive one or more sets of predefined features that are determined to be controllable by an agent when engaged in a customer-agent interaction-data; (Minter e.g. One or more monitoring servers, connected via a data link or API, may deliver monitored data to a server system [0005]. A method according to an implementation of the invention may include receiving metrics data and storing the metrics data in a database. The system may then receive new metrics data, via a data link from one or more user devices, and evaluating the new metrics data relative to the one or more persona models [0034]. The system monitors employee activities for a predetermined period of time (e.g. N days, N>0) and groups employees with same (or similar) job functions. Within these role groups or corporate hierarchy levels, the system may generate upper percentile model employees and identify actual employees that can be improved (e.g., below average to average, average to above average) based on their metrics [0036]. Examples of data sources for the data link 112, the API 114, file upload 116, or the hardware inputs 118 include, but are not limited to email logs, activity monitors, phone call recordings, phone call transcripts, and/or call logs. An agent's desktop computer, for example, may be monitored for CPU usage, mouse movement, words per minute, application logs, and other indicators of activity [0052]. Likewise, customer relations software may monitor the contacts the agent has with customers ( e.g. the phone call characteristics, outcome, sales, etc.) and compile that information into a structured table form [0052]. The data model database 151 may store agent metrics including call logs, voice analytics, CRM outcomes, survey results, resolution times, and other efficiency metrics and quality metrics [0067]. The Examiner submits that phone call characteristics, outcomes, sales, resolution times, etc. are deemed controllable features.)
Minter teaches implementing/implement a model corresponding to the KPI metric from a plurality of models, wherein the model is trained to predict a value for the KPI metric; (Minter e.g. The method may then generate, via the first computer instructions executed on the one or more processors of the computing device, one or more distributions of performers based on key performance indicators in the metrics data [0034]. The method may then generate, via the first computer instructions executed on the one or more processors of the computing device, one or more persona models characterizing variables in an upper percentile of the one or more distributions [0034]. The method may then train one or more machine learning models based on the one or more persona models to identify one or more result effective variables associated with the key performance indicators, the one or more machine learning models including computer-executable instructions [0034]. The method may also train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0034]. From one or more select metrics for the employee, the system generates distributions of performance and uses the upper percentile employees to model balanced outcomes [0044].)
Minter teaches predicting/predict, with the model, the value for the KPI metric based on a plurality of features that the model identifies and measures from a plurality of suggested agent-controllable features of the customer-agent interaction-data; (Minter e.g. The computing system may train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0027]. The system may then receive new metrics data, via a data link from one or more user devices, and evaluating the new metrics data relative to the one or more persona models. The method may also train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0034]. The data model database 151 may store agent metrics including call logs, voice analytics, CRM outcomes, survey results, resolution times, and other efficiency metrics and quality metrics (i.e. controllable features) [0067]. The KPI prediction component 363 may generate one or more predictions of the change of a KPI in one or more future periods (Fig. 3 and [0089]).)
Minter teaches determining/determine, from the plurality of features, one or more features having a potential for improvement based on measured values of the plurality of features determined by the model; (Minter e.g. The system may then determine areas that need improvement and prioritization (e.g., based on company goals, based on department goals, general improvement, etc.) [0036]. Based on highest priority areas, the system may select, for employees identified as needing improvement, a technique (e.g., training) to improve the employees' performance [0036]. The system then applies trained machine learning logic to produce recommendations based on the result effective variables and the balanced outcome models [0044]. The recommendations identified by the machine learning may focus on driving improvement in the result effective variables that have been determined (during training) to be associated with the metrics in the balanced outcome models [0045]. One or more of the ML classifiers or models may be configured to identify recommendations based on the best path or sequence of actions to arrive at one or more target metrics. These target metrics may be persona-based goals (e.g. increase performance of top 10 percentile or persona model by 2%) or distribution-based goals (e.g. shift median performance) [0087].)
Minter teaches determining/determine that improvement in a measured value of the one or more features maintains or increases the value of the KPI metric; (Minter e.g. The selection of the technique is based on what has been proven to have highest effectiveness in the company or for that employee personality/learning profile, or other personal characteristics. The system may then deliver the selected technique (e.g., training) to the employees [0036]. Thus, the computing system 110 automates the identification of methods for improvement and automatically trains average employees towards higher percentile outcomes and lower percentile employees towards average performance shifting the performance distribution upwards [0045]. One or more of the ML classifiers or models may be configured to identify recommendations based on the best path or sequence of actions to arrive at one or more target metrics [0087]. These target metrics may be persona-based goals (e.g. increase performance of top 10 percentile or persona model by 2%) or distribution-based goals (e.g. shift median performance) [0087].)
Minter teaches outputting/output a first indication of whether the value predicted for the KPI metric meets the target value and a second indication of the one or more features that maintains or increases the value of the KPI metric when the one or more features are improved; and (Minter e.g. These recommendations may include actions such as training that may be customized and delivered automatically to the employee. After one or more rounds of actions and detected improvements ( or lack thereof), machine learning logic may be trained to provide customized actions to each agent based on learned connections between past performance and employee characteristics (e.g. psychological profile, experience, tenure, etc.) [0045]. The KPI prediction component 363 may generate one or more predictions of the change of a KPI in one or more future periods [0089]. The predicted change may be a vector to be used for comparison against vectors in historical data from prior periods in the data model database 151.The predicted change(s) may include a confidence interval or predict a range of possible changes ( e.g. 5% improvement+/-0.5% ) [0089]. FIG. 16 illustrates a task manager specific view 1600 of a GUI provided by content delivery network 540 to an agent terminal 530 according to an implementation of the system. As illustrated, the agent may view their performance on multiple metrics and may view various tasks related to skill improvement (e.g. training) [0126].)
Minter teaches outputting a report indicating the value predicted for the KPI metric, the one or more features having the potential for improvement, and an amount that each of the one or more features is to improve such that the value predicted for the KPI metric meets the target value. (Minter e.g. The method may also train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0034]. The data model database 151 may also store predicted distributions developed by the recommendation component 160 or the actions delivery component 170. The projected or predicted distributions or metrics may be directed to a best case, worst case, trendline extrapolation, or may be predicted based on an automatically delivered action, a recommended coaching, or other action taken or recommended by the system [0070]. The KPI prediction component 363 may generate one or more predictions of the change of a KPI in one or more future periods [0089]. The predicted change may be a vector to be used for comparison against vectors in historical data from prior periods in the data model database 151.The predicted change(s) may include a confidence interval or predict a range of possible changes ( e.g. 5% improvement+/-0.5% ) [0089]. FIG. 16 illustrates a task manager specific view 1600 of a GUI provided by content delivery network 540 to an agent terminal 530 according to an implementation of the system. As illustrated, the agent may view their performance on multiple metrics and may view various tasks related to skill improvement (e.g. training) [0126].)
Minter does not explicitly teach, however, Basu teaches identifying a first subset of features from the one or more features that are not controllable by the agent when engaged in the customer-agent interaction; generating a second set of features by filtering out the first subset of features from the one or more features based on the one or more sets of predefined features determined to be in control of the agent; (Basu e.g. A method of determining a set of future actions includes providing an interface to a user, the interface including an adjustable element associated with a future value of an influencer. The influencer is associated with an action (Abstract). Evaluation of the business process can be correlated with performance indicators (PIs). For example, a call center can quantify performance using performance indicators, such as customer satisfaction, problem resolution, productivity indicators, cost indicators, or any combination thereof (col. 3 lines 45-52). The performance indicators (PIs) are influenced by other factors associated with performing the business process. In particular, such factors are referred to as influencers and influencers correlate with the performance indicators (col 3 lines 64-67). An influencer associated with call center performance can include the number of contacts made with a customer to resolve an issue, the type of issue, hold time, shipping delays, or any combination thereof, among others (col. 4 lines 1-4). It is recognized that some influencers may be at least partially controllable by the management of the organization whose performance is being measured by the KPIs, while other influencers may be "external" influencers derived from external factors over which the organization his little or no control (col. 22 lines 36-42). The following influencers are designated as “external” brand description, product description, reason code, and age of the system (col. 22 lines 56-64). Controllable influencers can be identified in the system as part of a set of business rules. "Controllable influencers" are also referred to herein as "actionable influencers." (col. 22 lines 42-45). The user may wish to focus upon more "controllable" influencers in order to be able to determine how behavior modification by the organization is predicted to affect the future outcome (i.e. as indicated by a KPI value(s)) of one or more aspects of a business process (cols. 22-23 lines 65-2). In the non-limiting example of FIGS. 5A-5C and 7A-7B, the only influencers that are presented to the user are those defined as 'controllable.' (col. 23 lines 5-7). Displayed identifiers may be ordered both according to magnitudes of deviations as well as whether or not the influencer is designated as an 'external influencer' or a 'controllable influencer.' (col. 49 lines 60-64).)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Minter to include identifying features that are not controllable by an agent when engaged in a customer-agent interaction as taught by Basu in order to assist with determining a desirable set of future actions to maintain a business process in compliance with business criteria (Basu e.g. col. 3 lines 23-25).
As per claims 3 and 12 (Currently Amended), Minter in view of Basu teach the method of claim 1, further comprising and the apparatus of claim 10, wherein the one or more processors are configured to execute the processor-executable instructions and further cause the apparatus to: Minter teaches determining/determine that the value predicted for the KPI metric by the model does not meet the target value (Minter e.g. The system may then deliver the selected technique (e.g., training) to the employees. In subsequent periods, the system may measure the results of the training for the employees in the improvement areas. If desired result was not achieved, the machine learning may alter its logic and perform further analysis. The system may then repeat one or more of the pieces of the process, such that additional training is selected, or other improvement areas are selected to meet the goals [0036]. The data collection systems 550 may also connect to the user terminals 530 and may monitor one or more user actions, tasks, or workspaces [0105]. The employee activities that are monitored may be sales revenue, customer contacts, time logged in or available, and other computer or phone monitored statistics [0106]. In subsequent monitoring periods, the process may measure the result of the action delivered to each agent. If the desired or predicted result is not achieved, further analysis may be performed to update and optimize the machine learning logic of the various components being executed [0106].); and determining, from the second set of features, a second subset of features that when improved causes the value predicted for the KPI metric to meet or exceed the target value for the KPI metric. (Minter e.g. The method may then train one or more machine learning models based on the one or more persona models to identify one or more result effective variables associated with the key performance indicators, the one or more machine learning models including computer-executable instructions [0034]. A result effective variable may be characterized as a variable with a causal link to other variables or a variable that precipitates changes in other variables, metrics, or outcomes [0025]. The computing system 110 utilizes the data collected from the multiple sources and applies machine learning logic to separate the result effective variables from the other metrics [0044]. The result effective variable, generally, may be determined by a correlation between variables where causation may be limited to a single direction. That is, the result effective variable may represent a driving factor in other metrics or outcomes of more importance to the company [0067]. Advantageously, machine learning logic can separate result effective variables from non-causal variables when the separation would not be apparent to a manager [0067]. In the case of process 700, the result effective variables may be targeted by the machine learning logic to be those that drive the high performance of the persona models [0110]. The selection of the technique is based on what has been proven to have highest effectiveness in the company or for that employee personality/learning profile, or other personal characteristics. The system may then deliver the selected technique ( e.g., training) to the employees [0036]. In subsequent periods, the system may measure the results of the training for the employees in the improvement areas. If desired result was not achieved, the machine learning may alter its logic and perform further analysis [0036].)
As per claims 4 and 13 (Original), Minter in view of Basu teach the method of claim 3, further comprising and the apparatus of claim 12, Minter teaches wherein the one or more processors are configured to execute the processor-executable instructions and cause the apparatus to outputting/output a report indicating the second subset of features that when improved causes the value predicted for the KPI metric to meet or exceed the target value for the KPI metric. (Minter e.g. The method may also train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0034]. The data model database 151 may also store predicted distributions developed by the recommendation component 160 or the actions delivery component 170. The projected or predicted distributions or metrics may be directed to a best case, worst case, trendline extrapolation, or may be predicted based on an automatically delivered action, a recommended coaching, or other action taken or recommended by the system [0070]. The KPI prediction component 363 may generate one or more predictions of the change of a KPI in one or more future periods [0089]. The predicted change may be a vector to be used for comparison against vectors in historical data from prior periods in the data model database 151.The predicted change(s) may include a confidence interval or predict a range of possible changes ( e.g. 5% improvement+/-0.5% ) [0089]. FIG. 16 illustrates a task manager specific view 1600 of a GUI provided by content delivery network 540 to an agent terminal 530 according to an implementation of the system. As illustrated, the agent may view their performance on multiple metrics and may view various tasks related to skill improvement (e.g. training) [0126].)
As per claims 5 and 14 (Original), Minter in view of Basu teach the method of claim 3 further comprising and the apparatus of claim 12, Minter teaches wherein the one or more processors are configured to execute the processor-executable instructions and further cause the apparatus to: determining/determine an amount of change for each feature of the second subset of features to cause the value predicted for the KPI metric to meet or exceed the target value for the KPI metric; and outputting a report indicating the second subset of features and the amount of change for each feature of the second subset of features. (Minter e.g. These systems and processes may be powered by multiple types of machine learning to identify the result effective variables, generate target metrics, and automate actions to be taken based on a variance of actual metrics from the target metrics [0025]. Based on highest priority areas, the system may select, for employees identified as needing improvement, a technique (e.g., training) to improve the employees' performance [0036]. In subsequent periods, the system may measure the results of the training for the employees in the improvement areas. If desired result was not achieved, the machine learning may alter its logic and perform further analysis [0036]. The one or more ML classifiers or models may be configured to identify recommendations based on the goals assigned to an agent, such that based on variance from the goals one or more actions are recommended [0087]. The KPI prediction component 363 may generate one or more predictions of the change of a KPI in one or more future periods [0089]. The predicted change may be a vector to be used for comparison against vectors in historical data from prior periods in the data model database 151.The predicted change(s) may include a confidence interval or predict a range of possible changes (e.g. 5% improvement+/-0.5% ) [0089]. FIG. 16 illustrates a task manager specific view 1600 of a GUI provided by content delivery network 540 to an agent terminal 530 according to an implementation of the system. As illustrated, the agent may view their performance on multiple metrics and may view various tasks related to skill improvement (e.g. training) [0126].)
As per claims 6 and 15 (Original), Minter in view of Basu teach the method of claim 5 further comprising and the apparatus of claim 14, Minter teaches wherein: the one or more processors are configured to execute the processor-executable instructions and further cause the apparatus to ranking/rank the second subset of features into a ranked list based on the amount of change for each feature, wherein the report provides an indication of the second subset of features in the ranked list. (Minter e.g. The recommendations component 160 may include sub-components including a prioritization component 361, an action-to-agent component 362, a KPI prediction component 363, a variance analysis component 364, a root cause analysis component 365, and a machine learning (ML) update or re-training component 366 (Fig. 3 and [0081]). The prioritization component 361 may provide a GUI for selection or configuration of one or more KPIs as a priority, one or more optimization routines, or one or more outcomes [0082]. A priority may be a ranking or focus of a feature above the ranking or importance of another feature. A priority may also be defined by or based on one or more goals or targets. The priority may be to advance or increase production, quality, sales, or customer experiences, such that the priority allows the prioritization component 361 to select one or more machine learning routines that address at least one root cause of the priority, one or more rules that address at least one root cause of the priority, or one or more result effective variables that form [0082]. The recommendations generated by the recommendation component 160 may comprise information displayed to a GUI for an agent's supervisor or an office trainer [0085]. One or more of the ML classifiers or models may be configured to identify recommendations based on variance to persona. For example, an agent may vary from the persona metrics or a range defined by the persona model in one or more metrics, or one or more skills [0086]. The prioritization component 361 may call or execute the root cause analysis component 365 to determine which metrics to prioritize or to determine how to balance the prioritization of metrics and/or actions [0092].)
As per claims 7 and 16 (Original), Minter in view of Basu teach the method of claim 5 further comprising: and the apparatus of claim 14, wherein the one or more processors are configured to execute the processor-executable instructions and further cause the apparatus to: determining/determine a feature from the second subset of features having a highest positive impact to the value of the KPI metric (Minter e.g. One or more of the ML classifiers or models may be configured to identify recommendations based on the best path or sequence of actions to arrive at one or more target metrics [0087]. These target metrics may be persona-based goals (e.g. increase performance of top 10 percentile or persona model by 2%) or distribution-based goals (e.g. shift median performance) [0087]. The one or more ML classifiers or models may be configured to identify recommendations based on the goals assigned to an agent, such that based on variance from the goals one or more actions are recommended [0087]. FIG. 6 illustrates a process 600 for determining recommendations and delivering actions according to an implementation of the invention [0106]. The process 600 is focused on driving or improving the appropriate result effective variables that have been identified by the machine learning as likely to result or cause corresponding improvement in the KPIs [0110]. At 608 in FIG. 6, the system may correlate result effective variables to KPIs and prioritize a particular decision matrix [0108]. The correlation by the machine learning logic may identify causal relationships between the result effective variables and the KPIs such that improvements in the result effective variables will drive improvement in the KPIs [0108]. The system may generate recommendations based on the result effective variables (e.g. at the recommendations component 160) and KPI variance. The recommendations may be directed at improving one or more result effective variables so that further or other metrics are improved ( e.g. KPIs) or KPI variance from the persona model is reduced or minimized [0112]. FIG. 8 depicts various optimizer and decision matrix options of the recommendations component 160 that may be selected or determined to form the basis for ML logic validation, evaluation, and ranking by a voting classifier. These optimizer and decision matrix options may be a part of prioritization component 361 [0114]. The metric optimizer 850 may target action delivery that will yield largest improvements in one or more metrics. The KPI optimizer 860 may leverage enterprise-wide performance indicators ( e.g. revenue, sales, etc.) to identify result effective variables and shift performance on numerous metrics to meet corporate goals [0117]. FIG. 14 illustrates a process 1400 of metric optimizer 850 which may target action delivery that will yield largest improvements in one or more metrics [0124].) Minter does not explicitly teach, however, Basu teaches with a lowest amount of change to the measured value; and outputting an indication of the feature that has the highest positive impact to the value of the KPI metric with the lowest amount of change to the measured value. (Basu e.g. In addition, embodiments of the system can assist with ranking a set of influencers based on their contribution to a particular performance indicator. A small change in a high ranking influencer may have a greater effect on a performance indicator than a large change in a low ranking influencer. Such a ranking can be used to perform root cause analysis (col. 8 lines 15-21). A user 3612 can influence the relationships established between constraints (R) and influencers (I). For example, a user can select parameters, a type of model, or other factors that influence how a relationship (r) is established between the influencers 3604, the constraints 3608, and the KPI 3602. Such a relationship (r) permits the determination of the KPI 3602 at one or more future time periods based on present and future values of influencers 3604 subject to constraints 3608. In addition, such a relationship (r) is useful for determining the influence of small changes in the influencers 3604 on the KPI 3602 at a selected future time (col. 9 lines 11-22 and Fig. 22). In one embodiment, root cause analysis module 2342 executing on root cause analysis computer 2340 may determine a ranked list of influencers (for example, explanatory variables) associated with the calculation of a value for a performance metric (for example, output variables), where the list may be ranked by size of contribution (for example, to the value of the performance metric) or according to some other ordering criteria (col. 36 lines 49-57). In addition, in certain embodiments, the ranked list of influencers may exclude certain influencers over which an entity may have little or no control (col. 36 lines 57-59).)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Minter to include determining features having a highest positive impact to the value of the KPI metric with the lowest amount of change as taught by Basu in order to assist with determining a desirable set of future actions to maintain a business process in compliance with business criteria (Basu e.g. col. 3 lines 23-25).
As per claims 9 and 18 (Original), Minter in view of Basu teach the method of claim 1 and the apparatus of claim 10, Minter also teaches wherein the model is a classifier model selected from a plurality of trained classifier models based on the KPI metric. (Minter e.g. The computing system may train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0027]. The models stored in the agent model database 255 and the action model database 254 may be classifiers, neural network layers, or other machine learning mappings that may map general recommendations to customized actions [0077].)
As per claim 21 (New), Minter in view of Basu teach the method of claim 9, Minter teaches further comprising training the classifier model, wherein the classifier model is specific to the KPI metric and wherein the classifier model is a machine learning-based model. (Minter e.g. The method may also train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0034]. The persona models may be formed from one or more high performing employees that have correctly balanced priorities or succeeded in one or more metrics [0072]. The achievable metrics of the personas are used to train the machine learning logic and to serve as corporate goals where quality, effectiveness, and efficiency need not be sacrificed [0072]. The method may then train one or more machine learning models based on the one or more persona models to identify one or more result effective variables associated with the key performance indicators, the one or more machine learning models including computer-executable instructions [0034]. The specific KPIs for a distribution that forms the personas may be configured or selected via a GUI, by automatic prioritization based on the business sector, or based on CRM data identifying immediate or seasonal needs of customers [0075]. One or more of the ML classifiers or models may be configured to identify recommendations based on the best path or sequence of actions to arrive at one or more target metrics [0087]. These target metrics may be persona-based goals (e.g. increase performance of top 10 percentile or persona model by 2%) or distribution-based goals (e.g. shift median performance) [0087].)
As per claim 22 (New), Minter in view of Basu teach the apparatus of claim 18, Minter teaches wherein the one or more processors are configured to cause the apparatus to train the classifier model, wherein the classifier model is specific to the KPI metric and wherein the classifier model is a machine learning-based model. (See claim 21 response.)
As per claim 23 (New), Minter in view of Basu teach the computer-program product of claim 19, Minter teaches wherein the instructions cause the computer to train the model, wherein the model is specific to the KPI metric and wherein the model is a machine learning-based model. (See claim 21 response.)
Claims 8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Minter (US 2022/0245557 A1) in view of Basu et al. (US 11,087,261 B1) and in further view of Bly et al. (US 2023/0289698 A1).
As per claims 8 and 17, Minter in view of Basu teach the method of claim 3 and the apparatus of claim 12, Minter teaches wherein determining/determine the second subset of features that when improved causes the value predicted for the KPI metric to meet or exceed the target value for the KPI metric for each feature of the second subset of features, (Minter e.g. The method may also train one or more classifiers of the one or more machine learning models to identify and predict performance based on the one or more persona models [0034]. The data model database 151 may also store predicted distributions developed by the recommendation component 160 or the actions delivery component 170. The projected or predicted distributions or metrics may be directed to a best case, worst case, trendline extrapolation, or may be predicted based on an automatically delivered action, a recommended coaching, or other action taken or recommended by the system [0070]. The KPI prediction component 363 may generate one or more predictions of the change of a KPI in one or more future periods. The predicted change may be a vector to be used for comparison against vectors in historical data from prior periods in the data model database 151.The predicted change(s) may include a confidence interval or predict a range of possible changes (e.g. 5% improvement+/-0.5% ) (Fig. 3 and [0089]).
Minter nor Basu teach, however, Bly teaches identifying causal features based on partial dependence plots and the partial dependence plots define a relationship between a change to the measured value and a probability of changing the KPI metric. (Bly e.g. A system and methods for improving the ability of a business or other entity to monitor business related metrics (Abstract). In some embodiments, information and data are represented in the form of a data structure termed a "Feature Graph" herein. A Feature Graph is a graph or diagram that includes nodes and edges, where the edges serve to "connect" a node to one or more other nodes [0053]. A node in a Feature Graph may represent a variable (i.e., a measurable quantity), an object, a characteristic, a feature, or a factor, as examples. An edge in a Feature Graph may represent a measure of a statistical association between a node and one or more other nodes [0053]. The association may be expressed in numerical and/or statistical terms and may vary from an observed (or possibly anecdotal) relationship to a measured correlation, to a causal relationship [0054]. System 100 employs mathematical, language-based, and visual methods to express the epistemological and underlying properties of the data and information available, for example the quality, rigor, trustworthiness, reproducibility, and completeness of the information and/or data supporting a given statistical association (as non-limiting examples); [0139]. For example, a statistical association with characteristics including a high and significant "feature importance" score measured in a model with a high area under the curve (AUC) score, with a partial dependence plot (PDP), and that is documented for reproducibility might be considered a "strong" (and presumably more reliable) statistical association in the Feature Graph and given an identifying color or icon in a graphical user interface; [0141].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Minter in view of Basu to include using partial dependence plots to determine features that when improved causes the value for the KPI metric to meet or exceed the target value as taught by Bly in order to help users manage, discover, and use the statistical relationships generated from correlations and associations and enable users to make more informed decisions regarding the operation of a business (Bly e.g. [0237] and [0241])
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ayanna Minor whose telephone number is (571)272-3605. The examiner can normally be reached M-F 9am-5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O'Connor can be reached at 571-272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.M./Examiner, Art Unit 3624
/Jerry O'Connor/Supervisory Patent Examiner,Group Art Unit 3624
Prosecution Insights