DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20 are pending in the instant patent application.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Regarding Claims 1-10, they are directed to a method, however the claims are directed to a judicial exception without significantly more. Claims 1-10 are directed to the abstract idea of predicting and visualizing a trend of a set of data.
Performing the Step 2A Prong 1 analysis while referring specifically to independent Claim 1, claim 1 recites tracking a plurality of metrics associated with a project, the plurality of metrics including a set of process metrics and a set of performance metrics; extracting, historical data of the plurality of metrics including a temporal series of historical metric indicators of each metric, each historical metric indicator corresponding to a respective sampling window having a respective temporal length; generating current data including a temporal series of current metric indicators of each of the plurality of metrics, each current metric indicator corresponding to a respective sampling window having a respective temporal length; identifying a target projection length; further including: grouping the temporal series of historical metric indicators of a subset of metrics to a plurality of metric indicator sets, each metric indicator set corresponding to a respective trend window having the target projection length; for each of the plurality of metric indicator sets: determining a respective performance trend corresponding to the respective trend window for one or more first performance metrics; using the respective performance trend as a ground truth; identifying a subset of historical metric indicators, which is sampled in a respective prediction window that precedes at least a subset of the respective trend window; and at a first time, while collecting the current data, identifying a subset of current metric indicators that corresponds to a current prediction window and includes a recent current indicator sampled immediately before or at the first time; applying the performance projection model to process the subset of current metric indicators, thereby generating a predicted performance trend of one or more first performance metrics corresponding to a current trend window identified by the target projection length; and visualizing the predicted performance trend of the one or more first performance metrics jointly with the subset of current metric indicators.
These claim limitations fall within the Mental Processes grouping of abstract ideas for they are concepts that can be performed in the human mind and/or with pen/paper. Examiner respectfully reminds Applicant, regardless of the complexity and/or granularity of the type of data, computational data analysis without meaningful limitations within the claims that amount to significantly more is a judicial exception (i.e. abstract idea). Furthermore, the courts have found claims requiring a generic computer or nominally reciting a generic computer may still recite a mental process even though the claim limitations are not performed entirely in the human mind (see MPEP 2106.04(a)(2)(III)(C).
Accordingly, the claim recites an abstract idea and dependent claims 2-10 further recite the abstract idea.
Regarding Step 2A Prong 2 analysis, the judicial exception is not integrated into a practical application. In particular the claim recites the elements of an information management application, a historical database, training a performance model using the historical data and training the performance projection model using the subset of historical metric indicators and the respective performance trend. The information management application, a historical database, training a performance model using the historical data and training the performance projection model using the subset of historical metric indicators and the respective performance trend are merely generic computing devices and do not integrate the judicial exception into a practical application. In addition, the “extracting” limitation cites mere data gathering which the courts have found to be insignificant extra-solution activity.
With respect to 2B, the claims do not include additional elements amounting to significantly more than the abstract idea. Claim 1 includes various elements that are not directed to the abstract idea under 2A. These elements include a information management application, a historical database, training a performance model using the historical data, training the performance projection model using the subset of historical metric indicators and the respective performance trend and the generic computing elements described in the Applicant's specification in at least Para 0038-0044. These elements do not amount to more than the abstract idea because it is a generic computer performing generic functions.
Therefore, Claim 1 is not drawn to eligible subject matter as it is directed to abstract ideas without significantly more.
Regarding Claims 11-15, they are directed to a system, however the claims are directed to a judicial exception without significantly more. Claims 11-15 are directed to the abstract idea of predicting and visualizing a trend of a set of data.
Performing the Step 2A Prong 1 analysis while referring specifically to independent Claim 11, claim 11 recites tracking a plurality of metrics associated with a project, the plurality of metrics including a set of process metrics and a set of performance metrics; extracting, historical data of the plurality of metrics including a temporal series of historical metric indicators of each metric, each historical metric indicator corresponding to a respective sampling window having a respective temporal length; generating current data including a temporal series of current metric indicators of each of the plurality of metrics, each current metric indicator corresponding to a respective sampling window having a respective temporal length; identifying a target projection length; further including: grouping the temporal series of historical metric indicators of a subset of metrics to a plurality of metric indicator sets, each metric indicator set corresponding to a respective trend window having the target projection length; for each of the plurality of metric indicator sets: determining a respective performance trend corresponding to the respective trend window for one or more first performance metrics; using the respective performance trend as a ground truth; identifying a subset of historical metric indicators, which is sampled in a respective prediction window that precedes at least a subset of the respective trend window; and at a first time, while collecting the current data, identifying a subset of current metric indicators that corresponds to a current prediction window and includes a recent current indicator sampled immediately before or at the first time; applying the performance projection model to process the subset of current metric indicators, thereby generating a predicted performance trend of one or more first performance metrics corresponding to a current trend window identified by the target projection length; and visualizing the predicted performance trend of the one or more first performance metrics jointly with the subset of current metric indicators.
These claim limitations fall within the Mental Processes grouping of abstract ideas for they are concepts that can be performed in the human mind and/or with pen/paper. Examiner respectfully reminds Applicant, regardless of the complexity and/or granularity of the type of data, computational data analysis without meaningful limitations within the claims that amount to significantly more is a judicial exception (i.e. abstract idea). Furthermore, the courts have found claims requiring a generic computer or nominally reciting a generic computer may still recite a mental process even though the claim limitations are not performed entirely in the human mind (see MPEP 2106.04(a)(2)(III)(C).
Accordingly, the claim recites an abstract idea and dependent claims 12-15 further recite the abstract idea.
Regarding Step 2A Prong 2 analysis, the judicial exception is not integrated into a practical application. In particular the claim recites the elements of one or more processors, memory, an information management application, a historical database, training a performance model using the historical data and training the performance projection model using the subset of historical metric indicators and the respective performance trend. The one or more processors, memory, information management application, a historical database, training a performance model using the historical data and training the performance projection model using the subset of historical metric indicators and the respective performance trend are merely generic computing devices and do not integrate the judicial exception into a practical application. In addition, the “extracting” limitation cites mere data gathering which the courts have found to be insignificant extra-solution activity.
With respect to 2B, the claims do not include additional elements amounting to significantly more than the abstract idea. Claim 11 includes various elements that are not directed to the abstract idea under 2A. These elements include one or more processors, memory, a information management application, a historical database, training a performance model using the historical data, training the performance projection model using the subset of historical metric indicators and the respective performance trend and the generic computing elements described in the Applicant's specification in at least Para 0038-0044. These elements do not amount to more than the abstract idea because it is a generic computer performing generic functions.
Therefore, Claim 11 is not drawn to eligible subject matter as it is directed to abstract ideas without significantly more.
Regarding Claims 16-20, they are directed to a non-transitory computer readable medium, however the claims are directed to a judicial exception without significantly more. Claims 16-20 are directed to the abstract idea of predicting and visualizing a trend of a set of data.
Performing the Step 2A Prong 1 analysis while referring specifically to independent Claim 16, claim 16 recites tracking a plurality of metrics associated with a project, the plurality of metrics including a set of process metrics and a set of performance metrics; extracting, historical data of the plurality of metrics including a temporal series of historical metric indicators of each metric, each historical metric indicator corresponding to a respective sampling window having a respective temporal length; generating current data including a temporal series of current metric indicators of each of the plurality of metrics, each current metric indicator corresponding to a respective sampling window having a respective temporal length; identifying a target projection length; further including: grouping the temporal series of historical metric indicators of a subset of metrics to a plurality of metric indicator sets, each metric indicator set corresponding to a respective trend window having the target projection length; for each of the plurality of metric indicator sets: determining a respective performance trend corresponding to the respective trend window for one or more first performance metrics; using the respective performance trend as a ground truth; identifying a subset of historical metric indicators, which is sampled in a respective prediction window that precedes at least a subset of the respective trend window; and at a first time, while collecting the current data, identifying a subset of current metric indicators that corresponds to a current prediction window and includes a recent current indicator sampled immediately before or at the first time; applying the performance projection model to process the subset of current metric indicators, thereby generating a predicted performance trend of one or more first performance metrics corresponding to a current trend window identified by the target projection length; and visualizing the predicted performance trend of the one or more first performance metrics jointly with the subset of current metric indicators.
These claim limitations fall within the Mental Processes grouping of abstract ideas for they are concepts that can be performed in the human mind and/or with pen/paper. Examiner respectfully reminds Applicant, regardless of the complexity and/or granularity of the type of data, computational data analysis without meaningful limitations within the claims that amount to significantly more is a judicial exception (i.e. abstract idea). Furthermore, the courts have found claims requiring a generic computer or nominally reciting a generic computer may still recite a mental process even though the claim limitations are not performed entirely in the human mind (see MPEP 2106.04(a)(2)(III)(C).
Accordingly, the claim recites an abstract idea and dependent claims 17-20 further recite the abstract idea.
Regarding Step 2A Prong 2 analysis, the judicial exception is not integrated into a practical application. In particular the claim recites the elements of an information management application, a historical database, training a performance model using the historical data and training the performance projection model using the subset of historical metric indicators and the respective performance trend. The information management application, a historical database, training a performance model using the historical data and training the performance projection model using the subset of historical metric indicators and the respective performance trend are merely generic computing devices and do not integrate the judicial exception into a practical application. In addition, the “extracting” limitation cites mere data gathering which the courts have found to be insignificant extra-solution activity.
With respect to 2B, the claims do not include additional elements amounting to significantly more than the abstract idea. Claim 16 includes various elements that are not directed to the abstract idea under 2A. These elements include a information management application, a historical database, training a performance model using the historical data, training the performance projection model using the subset of historical metric indicators and the respective performance trend and the generic computing elements described in the Applicant's specification in at least Para 0038-0044. These elements do not amount to more than the abstract idea because it is a generic computer performing generic functions.
Therefore, Claim 16 is not drawn to eligible subject matter as it is directed to abstract ideas without significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6-8, 10-11, 15-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bilicki et al. (US 2011/0061013 A1) in view of Achin et al. (US 2018/0046926 A1) further in view of Rajnayak et al. (US 2019/0325354 A1).
Regarding Claim 1, Bilicki teaches the limitations of Claim 1 which state
executing an information management application for tracking a plurality of metrics associated with a project, the plurality of metrics including a set of process metrics and a set of performance metrics (Bilicki: Para 0013-0014, 0019, 0025-0027, 0039-0042, 0049-0051 via the teaching of receiving operational data, maintaining/monitoring business performance metrics, collection/storing business metrics in a database, analyzing trends/indicators and displaying KPIs/status/trend indicators in a dashboard);
extracting, from a historical database, historical data of the plurality of metrics including a temporal series of historical metric indicators of each metric, each historical metric indicator corresponding to a respective sampling window having a respective temporal length (Bilicki: Para 0025-0027 via The metrics engine 120 can collect and store large quantities of data (e.g., in a database, a datacenter, a data warehouse, or the like). The metrics engine 120 can be a server, or other computer that can retain specific data (e.g., business metrics, including various retail performance metrics) collected from sources such as the collection of stores 104, the online store 106, the other operational inputs 108, and from other sources (e.g., human resources, marketing, R&D, or the like). The metrics engine 120 may include a metrics application module 122, such as a database server application, that can reside in a memory module 124 and can be executed by a processor module 126. The metrics engine 120 may store collected information in a storage module 128… The knowledge engine 110 may process and analyze operations data collected from the stores 104, the online store 106, the other operational inputs 108, and from other sources of information. The knowledge engine 110 may be used to help users identify trends, thresholds, and other indicators within the operational data. The metrics engine 120 can be a server, or other computer that can analyze information stored by the metrics engine 120. The knowledge engine 110 may include a knowledge application module 112, such as a business operations application, an online analytical processing (OLAP), or other server application that can reside in a memory module 114 and be executed by a processor module 126. The knowledge engine 110 may store information, such as configuration data, queries, measures, dimensions, fact tables, or other information in a storage module 118… A user can access the collected operations data through an operations dashboard 130. The operations dashboard 130 may provide an intuitive user interface that can present information from the metrics engine 120 and the knowledge engine 110, through the network 102. In some implementations, the operations dashboard 130 can provide a combined graphical and textual display of current business performance parameters, historical information, trend data, and other business metrics that can be filtered, sorted, listed, graphed, or otherwise presented to the user. For example, the operations dashboard 130 can be configured to help the user quickly identify metrics, trends, or combinations thereof that can indicate challenges within the business (e.g., low inventory, declining sales, increases in customer complaints, or the like). Similarly, the operations dashboard 130 may be configured to help the user quickly identify operational information that demonstrates outstanding performance by one or more different business units);
generating current data including a temporal series of current metric indicators of each of the plurality of metrics, each current metric indicator corresponding to a respective sampling window having a respective temporal length (Bilicki: Para 0027, 0041-0042 via A user can access the collected operations data through an operations dashboard 130. The operations dashboard 130 may provide an intuitive user interface that can present information from the metrics engine 120 and the knowledge engine 110, through the network 102. In some implementations, the operations dashboard 130 can provide a combined graphical and textual display of current business performance parameters, historical information, trend data, and other business metrics that can be filtered, sorted, listed, graphed, or otherwise presented to the user. For example, the operations dashboard 130 can be configured to help the user quickly identify metrics, trends, or combinations thereof that can indicate challenges within the business (e.g., low inventory, declining sales, increases in customer complaints, or the like). Similarly, the operations dashboard 130 may be configured to help the user quickly identify operational information that demonstrates outstanding performance by one or more different business units… The user may select a graph checkbox 204 to request that the selected data be graphed or otherwise graphically displayed when the data is presented. When the user has made the desired selections, a view category button 206 can be clicked to request the selected data (e.g., from the metrics engine 120). Upon clicking the view category button 206, the data presented on the operations dashboard may be updated to reflect the user's selection of parameters… The user interface 200 can variously display requested business information using various graphical indicators, such as a gauge 208, a trend chart 210, a pie chart 212, and/or one or more tables 214 and 216. In the illustrated example, the gauge 208 can show, for example, one or more KPIs for one or more periods, such as the current period and the prior period. The trend chart 210 may display trend or other data for one or more periods. In the illustrated example, the trend chart 210 can be configured to graphically display a KPI for the first six months of three consecutive years. In this way, the user may be able to compare the relative performance of one or more KPIs over a similar time period in multiple years);
However, Bilicki does not explicitly disclose the limitations of Claim 1 which state identifying a target projection length; training a performance projection model using the historical data, further including: grouping the temporal series of historical metric indicators of a subset of metrics to a plurality of metric indicator sets, each metric indicator set corresponding to a respective trend window having the target projection length.
Achin though, with the teachings of Bilicki, teaches of
identifying a target projection length (Achin: Para 0342 via The user may indicate a desired forecast range (e.g., the number of future time periods to be predicted by the model or the number of distinct future events to be predicted by the model). In some cases, this range may only be one observation, for example, the next quarter's sales. In others, like the supermarket chain example, the range may include several future periods, which in that example is 42 days);
training a performance projection model using the historical data, further including: grouping the temporal series of historical metric indicators of a subset of metrics to a plurality of metric indicator sets, each metric indicator set corresponding to a respective trend window having the target projection length (Achin: Para 0345, 0355-0370 via teaching of generating training data from time-series data, separating training-input and training-output data, identifying subsets of training data and fitting a predictive model to the training data. In addition, training ranges with validation ranges and subsets of training data that start at different times with durations that are integer multiples of the forecast range).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki with the teachings of Achin in order to have identifying a target projection length; training a performance projection model using the historical data, further including: grouping the temporal series of historical metric indicators of a subset of metrics to a plurality of metric indicator sets, each metric indicator set corresponding to a respective trend window having the target projection length. The motivations behind this being to incorporate the teachings of time-series predictive analysis as taught by Achin. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
The combination of Bilicki/Achin further teach the limitations of Claim 1 which state
for each of the plurality of metric indicator sets:determining a respective performance trend corresponding to the respective trend window for one or more first performance metrics (Blicki: Para 0027, 0042 via A user can access the collected operations data through an operations dashboard 130. The operations dashboard 130 may provide an intuitive user interface that can present information from the metrics engine 120 and the knowledge engine 110, through the network 102. In some implementations, the operations dashboard 130 can provide a combined graphical and textual display of current business performance parameters, historical information, trend data, and other business metrics that can be filtered, sorted, listed, graphed, or otherwise presented to the user. For example, the operations dashboard 130 can be configured to help the user quickly identify metrics, trends, or combinations thereof that can indicate challenges within the business (e.g., low inventory, declining sales, increases in customer complaints, or the like). Similarly, the operations dashboard 130 may be configured to help the user quickly identify operational information that demonstrates outstanding performance by one or more different business units.The user interface 200 can variously display requested business information using various graphical indicators, such as a gauge 208, a trend chart 210, a pie chart 212, and/or one or more tables 214 and 216. In the illustrated example, the gauge 208 can show, for example, one or more KPIs for one or more periods, such as the current period and the prior period. The trend chart 210 may display trend or other data for one or more periods. In the illustrated example, the trend chart 210 can be configured to graphically display a KPI for the first six moths of three consecutive years. In this way, the user may be able to compare the relative performance of one or more KPIs over a similar time period in multiple years.;
using the respective performance trend as a ground truth (Achin: Para 0368, 0376 via In step 950, training data are generated from the time-series data. The training data include a first subset of the observations of at least one of the data sets. The first subset of observations includes training-input and training-output collections of the observations. The times associated with the observations in the training-input and training-output collections correspond, respectively, to a training-input time range and a training-output time range. The skip range separates an end of the training-input time range from a beginning of the training-output time range. A duration of the training-output time range is at least as long as the forecast range…In step 970, a predictive model is fitted to the training data. In step 980, the fitted model is tested on the testing data. Cross-validation (including but not limited to nested cross-validation) and holdout techniques may be used for fitting and/or testing the predictive model. For purposes of cross-validation, the time-series data may be partitioned cross-sectionally and/or temporally);
identifying a subset of historical metric indicators, which is sampled in a respective prediction window that precedes at least a subset of the respective trend window (Achin: Para 0341. 0356, 0367-0370 via teaching of a skip range/gap between a training window and validation/holdout window; training input range preceding the training-output time range; temporal lag between latest observations and earliest prediction; training data ending at the end of the training-input time range and different rate sampling);
and training the performance projection model using the subset of historical metric indicators and the respective performance trend (Achin: Para 0368-0370, 0376 via training-input/training-output collections, identifying subsets of training data for training and fitting a predictive model to the training data).
In addition, Blicki does not explicitly disclose the limitations of Claim 1 which state at a first time, while collecting the current data, identifying a subset of current metric indicators that corresponds to a current prediction window and includes a recent current indicator sampled immediately before or at the first time; applying the performance projection model to process the subset of current metric indicators, thereby generating a predicted performance trend of one or more first performance metrics corresponding to a current trend window identified by the target projection length.
Rajnayak though, with the teachings of Blicki/Achin, teaches of
at a first time, while collecting the current data, identifying a subset of current metric indicators that corresponds to a current prediction window and includes a recent current indicator sampled immediately before or at the first time; applying the performance projection model to process the subset of current metric indicators, thereby generating a predicted performance trend of one or more first performance metrics corresponding to a current trend window identified by the target projection length (Rajnayak: Para 0027, 0037-0038, 0045-0048, 0050-0052 via teaching of receiving/pushing/pulling current data periodically, refreshed in data cycles, received from condition detectors and used to update the record and a next data cycle receiving new data; Selecting/applying a machine learning model to current data, calculating a propensity score, refining models as current data refreshes and predicting future status).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin with the teachings of Rajnayak in order to have at a first time, while collecting the current data, identifying a subset of current metric indicators that corresponds to a current prediction window and includes a recent current indicator sampled immediately before or at the first time; applying the performance projection model to process the subset of current metric indicators, thereby generating a predicted performance trend of one or more first performance metrics corresponding to a current trend window identified by the target projection length. The motivations behind this being to incorporate the teachings of using machine learning for performance prediction as taught by Rajnayak. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
The combination of Bilicki/Achin/Rajnayak further teaches the limitations of Claim 1 which state
visualizing the predicted performance trend of the one or more first performance metrics jointly with the subset of current metric indicators (Bilicki: Para 0027, 0039-0042, 0046-0051 via operational dashboard that displays and combines current business performance, historical data, trend data, metrics, projected data and trend indicators).
Regarding Claim 6, Bilicki/Achin/Rajnayak teaches the limitations of Claim 6 which state
wherein identifying the target projection length further comprises: identifying a plurality of predefined projection lengths; and receiving a user selection of the target projection length from the plurality of predefined projection lengths (Achin: Para 0366 via a forecast range associated with a prediction problem represented by the time-series data is determined. The forecast range may indicate a duration of a time period for which values of the targets are to be predicted. The forecast range may be determined based on (1) the time interval of the time-series data, (2) the number of observations included in the time-series data, (3) the time period covered by the observations in the time-series data, (4) a natural time period selected from the group consisting of microseconds, milliseconds, seconds, minutes, hours, days, weeks, months, quarters, seasons, years, decades, centuries, and millennia, (5) user input, etc. In some embodiments, the forecast range is an integer multiple of the time interval of the time-series data. In general, the forecast range may increase as number of observations increases in the time-series data increases)
Regarding Claim 7, Bilicki/Achin/Rajnayak teaches the limitations of Claim 7 which state
wherein visualizing the predicted performance trend further comprising: displaying the subset of current metric indicators with reference to a temporal axis; and rendering a curve corresponding to the predicted performance trend of the one or more first performance metrics, the curve originating from the subset of current metric indicators and extending towards a direction of the temporal axis (Bilicki: Para 0041-0042, 0046-0047 via use of customizable graphical displays, trend charts, historical and predictive data).
Regarding Claim 8, Bilicki/Achin/Rajnayak teaches the limitations of Claim 8 which state
wherein the predicted performance trend is selected from an upward trend, a steady trend, and a downward trend, visualizing the predicted performance trend further comprising: displaying the subset of current metric indicators with reference to a temporal axis; and displaying an arrow visually indicating one of the predicted performance trend (Bilicki: Para 0049-0051 via The status and trend indicator 310 can display a number of operational KPIs 312, such as "revenue," "profit," "margin," and "customer count" in the illustrated example. The status and trend indicator 310 can also display a number of status indicators 314 and trend indicators 316 (e.g., one status indicator and one trend indicator for each displayed operational KPIs 312). In some examples, the status and trend indicators 314 and 316 can provide the user with simple status information scorecards that can help the user quickly understand the current status of a number of the operational KPIs 312… scorecards can present the user with one or more items of information that can be associated with a simple indication of data associated with the item or items. In an example of a store management scorecard, a store manager may be presented with a list of departments within the store, and each listed item may be accompanied by a symbol (e.g., up/down arrows, thumbs up/down, smiley/sad face, an A-F letter grade) to indicate each department's performance. In other implementations, scorecard items may be accompanied by indicators such as a color code (e.g., semaphore), a word or short phrase (e.g., "PASS"/"FAIL", "ONLINE"/"OFFLINE"), or a number (e.g., a percentage, a dollar amount) to quickly convey the status of the items listed. In some implementations, the status information scorecard can be a static report displaying historical data over a given time period. For example, the scorecard may have multiple data layers, and may represent a high level overview of multiple metrics… the trend indicators 316 may be symbols that can show the user that each of the operational KPIs 312 is trending upward, downward, or has shown negligible change. It should be understood that the aforementioned examples of symbols and meanings are not limited to the examples given. For example, the status and trend indicators 314 and 316 may include other symbols that represent additional meanings. In another example, the status and trend indicators 314 and 316 may be color coded or animated to indicate such things as severity, rate, recent changes, or other similar meanings).
Regarding Claim 10, Bilicki/Achin/Rajnayak teaches the limitations of Claim 10 which state
wherein the set of performance metrics include one or more: quality of documents, completing request, finding request, FP portal, accuracy level, submitting request, timeliness, asset transfer, annuity tracking, delivering an insurance policy, and satisfaction level (Bilicki: Para 0019 via The operations system 100 may receive operational data, such as gross sales, inventory quantities, and other information related to its business operations primarily from the business's operational units. In the example of a retail business, the company's various stores 104 may each be considered an individual operational unit. The operations system 100 may maintain and monitor various business performance metrics related to each of the stores, such as inventory, retail pricing, customer satisfaction, and the like. In some implementations, such data may be stored in databases at the stores 104 themselves, or may be stored in other offsite locations).
Regarding Claim 11, it is analogous to Claim 1 and is rejected for the same reasons. (Blicki: Para 0073).
Regarding Claim 15, Bilicki/Achin/Rajnayak teaches the limitations of Claim 15 which state
wherein respective sampling windows of the set of process metrics have a first average temporal length, and respective sampling windows of the set of performance metrics have a second average temporal length that is greater than the first average temporal length (Achin: Para 0022-0026, 0358, 0371 via time-series datasets having different time intervals, ability to select a longer time interval, down-sampling the shorter interval dataset to match the longer interval and aggregate observations for the interval).
Regarding Claim 16, it is analogous to Claim 1 and is rejected for the same reasons. (Bilicki: Para 0077).
Regarding Claim 17, Bilicki/Achin/Rajnayak teaches the limitations of Claim 17 which state
wherein for one of the set of metrics, each current or historical metric indicator includes one of (1) a single metric indicator sampled during the respective sampling window and (2) an average of the respective metric indicators sampled during the respective sampling window (Achin: Para 0025-0026, 0371 via the actions of the method further include determining a duration of the training-input time range based, at least in part, on a total number of observations included in the time-series data, an amount of variation in values of at least one of the variables over time, an amount of seasonal variation in values of at least one of the variables, a consistency of variation in values of at least one of the variables over a plurality of time periods, and/or a duration of the forecast range. In some embodiments, fitting the predictive model to the training data includes fitting the predictive model to a subset of the training data corresponding to a portion of the training-input time range, wherein the portion of the training-input time range starts at a time subsequent to a starting time of the training-input time range and ends at an ending time of the training-input time range. In some embodiments, a duration of the portion of the training-input time range is an integer multiple of the duration of the forecast range… the actions of the method further include down-sampling the training data prior to fitting the predictive model to the training data. In some embodiments, down-sampling the training data includes: removing, from the training data, all observations obtained from at least one of the data sets. In some embodiments, down-sampling the training data includes setting a down-sampled time interval of the training data to an integer multiple of the time-interval of the time series data; and for each instance of the down-sampled time interval of the training data: identifying all observations in the training data associated with times corresponding to the respective instance of the down-sampled time interval of the training data, aggregating the identified observations to generate an aggregate observation, and replacing the identified observations in the training data with the aggregate observation. In some embodiments, the actions of the method further include down-sampling the testing data prior to testing the fitted model on the testing data… the training data may be down-sampled (e.g., temporally down-sampled or cross-sectionally down-sampled). The training data may be temporally down-sampled by selecting a down-sampled time interval and down-sampling each of the data sets in the training data according to the down-sampled time interval (e.g., using the techniques described above). The training data may be cross-sectionally down-sampled by removing one or more of the data sets from the training data. In some embodiments, the training data is both temporally down-sampled and cross-sectionally down-sampled).
Regarding Claim 18, Bilicki/Achin/Rajnayak teaches the limitations of Claim 18 which state
determining a second time that follows by the first time by the target projection length; collecting target data between the first time and the second time; determining a real performance trend based on at least the target data; and retaining the performance projection model using the subset of current metric indicators and a ground truth including the real performance trend (Achin: Para 0355-0356, 0366-0370, 0376-0377 via training-output collections/time ranges following input time ranges which have target observations for the forecast period; support using validation/testing output data to evaluate model prediction and fitting the predictive model and deploying and refreshing fitted models based on new time series data).
Regarding Claim 20, it is analogous to Claim 7 and is rejected for the same reasons.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bilicki et al. (US 2011/0061013 A1) in view of Achin et al. (US 2018/0046926 A1) in view of Rajnayak et al. (US 2019/0325354 A1) further in view of Joglekar et al. (US 2021/0374027 A1).
Regarding Claim 2, while Bilicki/Achin/Rajnayak teaches the limitations of Claim 1, it does not explicitly disclose the limitations of Claim 2 which state for a first metric of the plurality of metrics: determining an average and a standard deviation based on the historical data of the first metric; setting one or more thresholds based on the average and the standard deviation of the first metric; in real time, while collecting a subset of current data corresponding to the first metric, comparing each current metric indicator of the first metric with the one or more thresholds; and based on a comparison result, generating an alert associated with the first metric.
Joglekar though, with the teachings of Bilicki/Achin/Rajnayak, teaches of
for a first metric of the plurality of metrics: determining an average and a standard deviation based on the historical data of the first metric; setting one or more thresholds based on the average and the standard deviation of the first metric; in real time, while collecting a subset of current data corresponding to the first metric, comparing each current metric indicator of the first metric with the one or more thresholds; and based on a comparison result, generating an alert associated with the first metric (Joglekar: Para 0090, 0096, 0136-0142, 0147-0153 via threshold module 310 may be used by monitoring server 300 to determine metric thresholds. The metric thresholds may comprise a metric lower threshold and a metric upper threshold. The metric thresholds may define the boundaries for a normal metric value. A metric may be considered normal if its corresponding metric value is greater than or equal to the metric lower threshold, and less than or equal to the metric upper threshold. The metric thresholds may correspond to each metric of a set of metrics… the monitoring server determines a set of metric averages and a set of metric standard deviations based on the predetermined number of sets of metrics and their associated metric values. This may involve calculating the metric average and metric standard deviation corresponding to each metric of the predetermined number of sets of metrics and their associated metric values… the monitoring server determines a set of metric standard deviation products by multiplying each metric standard deviations of the set of metric standard deviations by a corresponding deviation multiplier of a set of deviation multipliers. As an example of this calculation, the metric standard deviation for CPU cycles may be 20 cycles and the deviation multiplier for CPU cycles may be 1.5. The metric standard deviation product for CPU cycles may be determined by 20*1.5 = 30 CPU cycles. The metric standard deviation product for each metric standard deviation of the set of metric standard deviations may be calculated in a similar way… the monitoring server can determine a set of metric upper thresholds by summing the set of metric averages and the set of metric standard deviation products, and determine a set of metric lower thresholds by determining a difference between the set of metric averages and the set of metric standard deviation products. The set of metric thresholds, referred to in step 516, may comprise the metric upper thresholds and metric lower thresholds corresponding to each metric, for example, the metric upper and lower threshold for CPU cycles, the metric upper and lower threshold for latency, etc… the monitoring server can determine whether each current metric value of the set of current metric values is less than or equal to each metric upper threshold of the set of metric upper thresholds. For example, if the current metrics include the metrics CPU cycles, memory allocated, and heartbeats, the monitoring server can compare the current metric values corresponding to CPU cycles, memory allocated, and heartbeats (e.g., 280 CPU cycles, 200 MB allocated, and 3 heartbeats) against the corresponding metric upper thresholds… If the monitoring server determines that one or more of the current metric values corresponding to the set of current metrics are less than the corresponding metric lower threshold or are greater than the corresponding metric upper threshold, the monitoring server may determine that the current set of metrics are anomalous; Para 0149-0153 teach of anomaly score/pattern and alert flow).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin/Rajnayak with the teachings of Joglekar in order to have for a first metric of the plurality of metrics: determining an average and a standard deviation based on the historical data of the first metric; setting one or more thresholds based on the average and the standard deviation of the first metric; in real time, while collecting a subset of current data corresponding to the first metric, comparing each current metric indicator of the first metric with the one or more thresholds; and based on a comparison result, generating an alert associated with the first metric. The motivations behind this being to incorporate the teachings of evaluating metrics, detecting anomalies and issuing alerts as taught by Joglekar. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
Claim(s) 3-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bilicki et al. (US 2011/0061013 A1) in view of Achin et al. (US 2018/0046926 A1) in view of Rajnayak et al. (US 2019/0325354 A1) further in view of Bailey et al. (US 2017/0070523 A1).
Regarding Claim 3, while Bilicki/Achin/Rajnayak teaches the limitations of Claim 2, it does not explicitly disclose the limitations of Claim 3 which state
wherein the alert corresponds to a state of a hierarchy of alert states defined based on the standard deviation (Bailey: Para 0150, 0232 via thresholds for dividing numeric measurements into buckets may be chosen based on observations from population data. For instance, the inventors have recognized and appreciated that the time from product view to checkout is rarely less than 10 seconds in a legitimate digital interaction, and therefore a high count for the bucket 581 may be a good indicator of an anomaly. In some embodiments, buckets may be defined based on a population mean and a population standard deviation. For instance, there may be a first bucket for values that are within one standard deviation of the mean, a second bucket for values that are between one and two standard deviations away from the mean, a third bucket for values that are between two and three standard deviations away from the mean, and a fourth bucket for values that are more than three standard deviations away from the mean…raise an alert when significant deviation from an expected baseline is observed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin/Rajnayak with the teachings of Bailey in order to have wherein the alert corresponds to a state of a hierarchy of alert states defined based on the standard deviation. The motivations behind this being to incorporate the teachings of detecting anomalies. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
Regarding Claim 4, while Bilicki/Achin/Rajnayak teaches the limitations of Claim 2, it does not explicitly disclose the limitations of Claim 4 which state wherein generating the alert further comprises: in accordance with a determination that a current metric indicator of the first metric deviates from the average greater than twice of the standard deviation, increasing an issue count by 1; and displaying, in real time and on a user interface, information of the first metric including the issue count.
Bailey though, with the teachings of Bilicki/Achin/Rajnayak, teaches of
wherein generating the alert further comprises: in accordance with a determination that a current metric indicator of the first metric deviates from the average greater than twice of the standard deviation, increasing an issue count by 1; and displaying, in real time and on a user interface, information of the first metric including the issue count (Bailey: Para 0150, 0162-0164, 0182-0183, 0211, 0218-0220 via teaching of standard deviation buckets, comparing current counts to historical expected counts, storing extent of observed count deviation, incrementing/counting anomalous matches and using the count deviation in a score).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin/Rajnayak with the teachings of Bailey in order to have wherein generating the alert further comprises: in accordance with a determination that a current metric indicator of the first metric deviates from the average greater than twice of the standard deviation, increasing an issue count by 1; and displaying, in real time and on a user interface, information of the first metric including the issue count. The motivations behind this being to incorporate the teachings of detecting anomalies. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
Regarding Claim 5, while Bilicki/Achin/Rajnayak teaches the limitations of Claim 3, it does not explicitly disclose the limitations of Claim 5 which state wherein generating the alert further comprises: in accordance with a determination that the current metric indicator of the first metric deviates from the average between the standard deviation and twice of the standard deviation, increasing a risk counter by 1.
Bailey though, with the teachings of Bilicki/Achin/Rajnayak, teaches of
wherein generating the alert further comprises: in accordance with a determination that the current metric indicator of the first metric deviates from the average between the standard deviation and twice of the standard deviation, increasing a risk counter by 1 (Bailey: Para 0150, 0211, 0218-0220 via thresholds for dividing numeric measurements into buckets may be chosen based on observations from population data. For instance, the inventors have recognized and appreciated that the time from product view to checkout is rarely less than 10 seconds in a legitimate digital interaction, and therefore a high count for the bucket 581 may be a good indicator of an anomaly. In some embodiments, buckets may be defined based on a population mean and a population standard deviation. For instance, there may be a first bucket for values that are within one standard deviation of the mean, a second bucket for values that are between one and two standard deviations away from the mean, a third bucket for values that are between two and three standard deviations away from the mean, and a fourth bucket for values that are more than three standard deviations away from the mean. However, it should be appreciated that aspects of the present disclosure are not limited to the use of population mean and population standard deviation to define buckets… an anomalous attribute may be product SKU, and an anomalous attribute may be a particular hash-mod bucket (e.g., the last bucket in the illustrative histogram 1260 shown in FIG. 12). The profile may store an indication of an extent to which an observed count for that bucket (e.g., the count 1266) deviates from an expected count (e.g., the count 1272). As one example, the profile may store a percentage by which the observed count exceeds the expected count. As another example, the profile may store an amount by which the observed count exceeds the expected count… the security system may determine if there is another anomalous attribute to be processed. If so, the security system may return to act 1410. Otherwise, the security system may proceed to act 1425 to calculate a penalty score. The penalty score may be calculated in any suitable manner. In some embodiments, the penalty score is determined on a ratio between a count of anomalous attributes with respect to which the digital interaction matches the profile, and a total count of anomalous attributes… an attribute penalty score may be determined for a matching attribute based on an extent to which an observed count for a matching bucket deviates from an expected count for that bucket. An overall penalty score may then be calculated based on one or more attribute penalty scores (e.g., as a weighted sum)…).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin/Rajnayak with the teachings of Bailey in order to have wherein generating the alert further comprises: in accordance with a determination that the current metric indicator of the first metric deviates from the average between the standard deviation and twice of the standard deviation, increasing a risk counter by 1. The motivations behind this being to incorporate the teachings of detecting anomalies. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bilicki et al. (US 2011/0061013 A1) in view of Achin et al. (US 2018/0046926 A1) in view of Rajnayak et al. (US 2019/0325354 A1) further in view of Sethi et al. (US 2014/0365266 A1).
Regarding Claim 9, while the combination of Bilicki/Achin/Rajnayak teaches the limitations of Claim 1, it does not explicitly disclose the limitations of Claim 9 which state wherein the set of process metrics include one or more of:a number of requests with not in good order (NIGO) issues, an average call per request, a percentage of paper requests, an average request turnaround time, a percentage of requests requiring asset transfer, and an average asset transfer turnaround time.
Sethi though, with the teachings of Bilicki/Achin/Rajnayak, teaches of
wherein the set of process metrics include one or more of: a number of requests with not in good order (NIGO) issues, an average call per request, a percentage of paper requests, an average request turnaround time, a percentage of requests requiring asset transfer, and an average asset transfer turnaround time (Sethi: Para 0029 via The capturing module 120 may further receive business data from the user through a template for understanding an end customer to whom a product or a service is delivered using the enterprise process. For the purpose, the user may interview the business users during workshops and interviews sessions for gathering the business data. The business data may further be used to analyze key end-to-end business metrics that the enterprise process is impacting and corresponding values of the end-to-end metrics. End-to-end business metrics may be defined as data about the enterprise process which is being analyzed and optimized. Examples of end-to-end business metrics for an enterprise process pertaining to settlement of insurance claims include, but are not limited to, policy issuance name, target rate, TAT, count of backdate errors, and not-in-good-order (NIGO). The business data received by the capturing module 120 may be further stored as the process data 128).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin/Rajnayak with the teachings of Sethi in order to have wherein the set of process metrics include one or more of: a number of requests with not in good order (NIGO) issues, an average call per request, a percentage of paper requests, an average request turnaround time, a percentage of requests requiring asset transfer, and an average asset transfer turnaround time. The motivations behind this being to incorporate the teachings of using metrics for enterprise evaluations as taught by Sethi. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
Claim(s) 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bilicki et al. (US 2011/0061013 A1) in view of Achin et al. (US 2018/0046926 A1) in view of Rajnayak et al. (US 2019/0325354 A1) further in view of Gates et al. (US 2010/0332287 A1).
Regarding Claim 12, while Bilicki/Achin/Rajnayak teaches the limitations of Claim 11, it does not explicitly disclose the limitations of Claim 12 which state receiving a plurality of user messages in reply to a plurality of queries; and extracting a temporal series of current metric indicators of a second performance metric from the plurality of user messages.
Gates though, with the teachings of Bilicki/Achin/Rajnayak teaches of
receiving a plurality of user messages in reply to a plurality of queries; and extracting a temporal series of current metric indicators of a second performance metric from the plurality of user messages (Gates: Para 0021-0026, 0032-0033 via using e-mails, call summaries, call logs, chat sessions and comments as inputs for customer satisfaction. Extracting unstructured and structured features from text and generating a satisfaction score in real time and estimates at various points during an interaction; capturing interaction text and analyzing the text).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin/Rajnayak with the teachings of Gates in order to have receiving a plurality of user messages in reply to a plurality of queries; and extracting a temporal series of current metric indicators of a second performance metric from the plurality of user messages. The motivations behind this being to incorporate the teachings of real-time prediction of customer satisfaction based on automatic analysis of customer interaction. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
Regarding Claim 13, the combination of Bilicki/Achin/Rajnayak/Gates teaches the limitations of Claim 13 which state
applying a message classification model to process each of the plurality of user messages to determine a temporal series of satisfaction states corresponding to the second performance metric (Gates: Para 0009-0012, 0022, 0033-0034 via sentiment analysis/classification and machine learning applied to interactions to generate a satisfaction score…using classification methods such as naïve bayes, logistic regression and SVM for the C-SAT model…using NLP/text analysis to extract satisfaction features);
and determining a temporal series of satisfaction rates based on the temporal series of satisfaction states corresponding to the second performance metric (Gates: Para 0024-0026 via measure customer satisfaction into the five C-SAT scores and/or into a binary distinction of customer satisfaction (i.e., "satisfied" vs. "dissatisfied")…generating customer satisfaction scores in real time from interaction text).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bilicki et al. (US 2011/0061013 A1) in view of Achin et al. (US 2018/0046926 A1) in view of Rajnayak et al. (US 2019/0325354 A1) further in view of Henry (US 2019/0065948 A1).
Regarding Claim 14, while the combination of Bilicki/Achin/Rajnayak teaches the limitations of Claim 11, it does not explicitly disclose the limitations of Claim 14 which state wherein: a historical sample time corresponds to a respective historical metric indicator of each first performance metric and a historical ease of doing business (EODB) indicator, which is a combination of the respective historical metric indicators of the one or more first performance metrics; a current sample time corresponds to a respective current metric indicator of each first performance metric and a current EODB indicator, which is a combination of the respective current metric indicators of the one or more first performance metrics; and the predicted performance trend includes a predicted change of the current EODB indicator.
Henry though, with the teachings of Bilicki/Achin/Rajnayak, teaches of
wherein: a historical sample time corresponds to a respective historical metric indicator of each first performance metric and a historical ease of doing business (EODB) indicator, which is a combination of the respective historical metric indicators of the one or more first performance metrics (Henry: Para 0039-0044, 0053 via model training over multiple time periods using performance scores…disclose performance scores per time period, including customer satisfaction and handling rate…multiple scores combined into a total score using weighted combinations);
a current sample time corresponds to a respective current metric indicator of each first performance metric and a current EODB indicator, which is a combination of the respective current metric indicators of the one or more first performance metrics (Henry: Para 0042-0048 via computing current performance scores once per time period and computing average rating for that time period…computing current scores using current period and previous period performance scores); and
the predicted performance trend includes a predicted change of the current EODB indicator (Henry: Para 0046-0048, 0053 via reward/performance change between a current and previous period… a combined weighted reward/performance score. In combination, the predicted trend/change of a composite ease of doing business indicator is taught).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Bilicki/Achin/Rajnayak with the teachings of Henry in order to have wherein: a historical sample time corresponds to a respective historical metric indicator of each first performance metric and a historical ease of doing business (EODB) indicator, which is a combination of the respective historical metric indicators of the one or more first performance metrics; a current sample time corresponds to a respective current metric indicator of each first performance metric and a current EODB indicator, which is a combination of the respective current metric indicators of the one or more first performance metrics; and the predicted performance trend includes a predicted change of the current EODB indicator. The motivations behind this being to incorporate the teachings of performance model training as taught by Henry. Furthermore, in addition to being in the same CPC class, the teachings, suggestions, and motivations in this prior art would have led one of ordinary skill to modify the prior art reference or combine prior art reference teachings to arrive at the claimed invention.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Anderson et al. (US 2015/0317589 A1)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYRONE E SINGLETARY whose telephone number is (571)272-1684. The examiner can normally be reached 9 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Beth Boswell can be reached at 571-272-6737. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.E.S./Examiner, Art Unit 3625
/BETH V BOSWELL/Supervisory Patent Examiner, Art Unit 3625
Prosecution Insights