DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In response to Applicant’s claims filed on February 11, 2026, claims 1-20 are now pending for examination in the application.
Response to Arguments
The 112 rejection under USC 112 set forth in the 02/11/26 office action is hereby withdrawn.
This office action is in response to amendment filed 02/11/2026. In this action Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over of Walters et al. (US Pub. No. 20200012900) and Stockdale et al. (US Pub. No. 20200244673) in further view of Oberbreckling et al. (US Pub. No. 20180074786). The Stockdale et al. reference has been added to address the amendment of for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state.
Applicant’s arguments:
In regards to claim 1 on Page(s) 7, applicant argues “Additionally, a human cannot receive a high-frequency digital data stream, apply a recursive exponential decay formula to update a histogram state in real-time, and compare it against a pre-sampled divergence distribution. The volume and velocity of data preclude mental performance.”
Examiner’s Reply:
The examiner respectfully disagrees and would like to point out that human mind using a
computer as a tool is fully capable of determining features and visualizations and calculating statistical values. A human would be able to iteratively follow these steps along with any needed additional elements (eg using receiving streaming data).
Applicant’s arguments:
In regards to claim 1 on Page(s) 7, applicant argues “management and signal processing algorithms. Additionally, concepts are integrated into a practical application for resource-constrained real-time monitoring. The amended claims also recite that the recursive technique is used "to consume fewer computational resources compared with an update that does not use a constant memory reference state." This is an express recitation of a technical benefit.”
Examiner’s Reply:
The examiner notes that the computer (being used as a generic tool) as recited in the claims is being used for monitoring features and analyzing large volumes of data. The use of mathematical calculations does not improve the functioning of a computer. Therefore, the abstract idea recited in the claims is generally linking it to a computer environment, and does not integrate the abstract idea into a practical application.
Applicant’s arguments:
In regards to claim 1 on Page(s) 7, applicant argues “Even if the claim failed Step 2A, it satisfies Step 2B by adding significantly more to the judicial exception. The combination of generating a distribution of divergence values via subset sampling (e.g., a specific training phase technique), passing this as part of a triplet (e.g., a specific data structure); and updating runtime states via recursive decay (e.g., a specific runtime optimization) constitutes specific orchestration of steps is not well-understood, routine, or conventional.”
Examiner’s Reply:
Machine learning and is well-understood, routine and conventional and using a model in order to generate data for improved searching WURC. There is nothing novel about feature monitoring and analysis.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
*-Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The eligibility analysis in support of these findings is provided below, in accordance with the 2019 Revised Patent Subject Matter Eligibility Guidance, hereinafter 2019 PEG.
Step 1. In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), it is noted that the system, method, and portable device of claims 1-20 are directed to one of the eligible categories of subject matter and therefore satisfy Step 1.
Step 2A. In accordance with Step 2A, prong one of the 2019 PEG, it is noted that the independent claims recite an abstract idea falling within the Mathematical Concepts & Mental Processes enumerated groupings of abstract ideas set forth in the 2019 PEG. Examiner is of the position that independent claims 1, 19, and 20 are directed towards the Mathematical Concepts & Mental Process Grouping of Abstract Ideas.
Independent claims 1, 19, and 20 recites the following limitations directed towards a Mathematical Concepts & Mental Processes:
aggregating the determined divergences (The limitation recites a mathematical concept; aggregating);
for each feature of the set of features, using the one or more events to update, a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool);
for each feature of the set of features, using the corresponding updated distribution and a corresponding reference distribution to determine a corresponding divergence value (The limitation recites a mathematical concept; calculating);
for each feature of the set of features, using the corresponding determined divergence value and a corresponding distribution of divergences to determine a corresponding statistical value, wherein the statistical value indicates a probability of observing the determined divergence value based on historical fluctuations characterized by the distribution of divergence values (The limitation recites a mathematical concept; calculating); and
using the statistical values each corresponding to a different feature of the set of features, performing, a statistical analysis to determine a result associated with a likelihood of data drift detection in real-time or (The limitation recites a mathematical concept; calculating),
determining a signal visualization based on the result (mental step of determining a visualization on a computer screen, computer is being used as a generic tool).
Step 2A. In accordance with Step 2A, prong two of the 2019 PEG, the judicial exception is not integrated into a practical application because of the recitation in claim(s) 1, 19, and 20:
One or more processors (i.e., as a generic processor performing a generic computer function) configured to:
Receiving one or more events in a data stream (recites insignificant extra solution activity that amounts to mere data gathering);
receiving, for each feature of a set of features, a corresponding triplet of distribution representations comprising (i) a reference distribution derived from a reference dataset, (ii) an initial sample distribution, and (iii) a distribution of divergence values, wherein the distribution of divergence values is generated by sampling a plurality of subsets from the reference dataset (recites insignificant extra solution activity that amounts to mere data gathering)
outputting the signal visualization associated with the determined result (recites insignificant extra solution activity that amounts to outputting data)
a memory coupled to at least one of the one or more processors and configured to provide at least one of the one or more processors with instructions (i.e., as a generic processor performing a generic computer function).
Step 2B. Similar to the analysis under 2A Prong Two, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Because the additional elements of the independent claims amount to insignificant extra solution activity and/or mere instructions, the additional elements do not add significantly more to the judicial exception such that the independent claims as a whole would be patent eligible.
Therefore, independent claims 1, 19, and 20 are rejected under 35 U.S.C. 101.
With respect to claim(s) 2:
Step 2A, prong one of the 2019 PEG:
Examiner is of the position the dependent claim is directed toward additional elements.
Step 2A Prong Two Analysis:
wherein at least a portion of the one or more events have is occurred at distinct points in time (recites insignificant extrasolution activity of receiving event data).
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 3:
Step 2A, prong one of the 2019 PEG:
Examiner is of the position the dependent claim is directed toward additional elements.
Step 2A Prong Two Analysis:
wherein the one or more events correspond to information associated with transactions being analyzed to detect fraud (recites insignificant extrasolution activity of receiving and event data).
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 4:
Step 2A, prong one of the 2019 PEG:
wherein one or more features of the set of features are associated with a numerical measurement of data (The limitation recites a mathematical concept; calculating).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 5:
Step 2A, prong one of the 2019 PEG:
wherein one or more features of the set of features are utilized by a machine learning model for predictive tasks (The limitation recites a mathematical concept; calculating).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 6:
Step 2A, prong one of the 2019 PEG:
wherein using the one or more events to update the corresponding distribution of data from the data stream includes assigning each of the one or more events to a category among a plurality of categories associated with the corresponding distribution of data and correspondingly incrementing counts of events in categories of the plurality of categories (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 7:
Step 2A, prong one of the 2019 PEG:
wherein the corresponding distribution of data from the data stream is represented as a histogram (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 8:
Step 2A, prong one of the 2019 PEG:
wherein the histogram is generated including by applying an exponential moving average suppression of older events (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 9:
Step 2A, prong one of the 2019 PEG:
wherein the corresponding statistical value is a p-value (The limitation recites a mathematical concept; calculating).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 10:
Step 2A, prong one of the 2019 PEG:
Examiner is of the position the dependent claim is directed toward additional elements.
Step 2A Prong Two Analysis:
receiving, for each feature of the set of features, the corresponding reference distribution and the corresponding distribution of divergences (recites insignificant extra solution activity that amounts to mere data gathering).
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 11:
Step 2A, prong one of the 2019 PEG:
wherein performing the statistical analysis includes performing a multivariate hypothesis test (The limitation recites a mathematical concept; calculating).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 12:
Step 2A, prong one of the 2019 PEG:
wherein performing the multivariate hypothesis test includes scaling the statistical values (The limitation recites a mathematical concept; calculating).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 13:
Step 2A, prong one of the 2019 PEG:
wherein the statistical analysis is performed each time a batch of events is received (The limitation recites a mathematical concept; calculating).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 14:
Step 2A, prong one of the 2019 PEG:
analyzing the result to determine whether a specified condition has been satisfied (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 15:
Step 2A, prong one of the 2019 PEG:
in response to a determination that the specified condition has been satisfied, providing an alarm (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 16:
Step 2A, prong one of the 2019 PEG:
in response to the alarm, generating an alarm report that includes:
a ranking of features of the set of features according to how much each feature of the set of features contributed to the alarm (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool); and
an explanation to identify a root cause associated with the data drift detection (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 17:
Step 2A, prong one of the 2019 PEG:
wherein the alarm causes retraining of a machine learning model (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
With respect to claim(s) 18:
Step 2A, prong one of the 2019 PEG:
wherein the specified condition is associated with one or more comparisons to a threshold value (mental step of observing and/or evaluation of event data on a computer screen, computer is being used as a generic tool).
Step 2A Prong Two Analysis:
This judicial exception is not integrated into a practical application because there are no
additional elements to provide practical application.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over of Walters et al. (US Pub. No. 20200012900) and Stockdale et al. (US Pub. No. 20200244673) in further view of Oberbreckling et al. (US Pub. No. 20180074786).
With respect to claim 1, Walters et al. discloses a method, comprising:
Receiving one or more events of a data stream (Paragraph 127 discloses streaming data source 1301 can be configured to periodically retrieve all log data created within a certain period (e.g., a five-minute interval). In some embodiments, the data can be application logs. The application logs can include event information);
receiving, for each feature of a set of features, a corresponding triplet of distribution representations comprising (i) a reference distribution derived from a reference dataset, (ii) an initial sample distribution, and (iii) a distribution of divergence values, wherein the distribution of divergence values is generated by sampling a plurality of subsets from the reference dataset, determining a divergence for each of the plurality of subsets relative to the reference distribution, and aggregating the determined divergences (Paragraph 78 discloses dataset generator 103 can be configured with a statistical distribution of subclasses (e.g., a univariate distribution of subclasses) for that class and can select one of the subclasses for generating the synthetic data according to the statistical distribution);
for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state (Paragraph 185 discloses the model may be corrected (updated) based on detected drift. Correcting the model may include model training and/or hyperparameter tuning, consistent with disclosed embodiments. Correcting the model may be involve model training or hyperparameter tuning using the received event data and/or other data);
for each feature of the set of features, using the corresponding determined divergence value and a corresponding distribution of divergences to determine a corresponding statistical value for that class and can select one of the subclasses for generating the synthetic data according to the statistical distribution, wherein the statistical value indicates a probability of observing the determined divergence value based on historical fluctuations characterized by the distribution of divergence values (Paragraph 98 discloses dataset generator 103 can be configured with a statistical distribution of subclasses (e.g., a univariate distribution of subclasses and Paragraph 136 discloses Dataset generator 1307 can then predict the next value using the updated value string that includes the new value. In some embodiments, rather than selecting the most likely new value, dataset generator 1307 can be configured to probabilistically choose a new value);
teaches using the statistical values each corresponding to a different feature of the set of features, performing a statistical analysis to determine a result associated with a likelihood of data drift detection in real-time (Paragraph 176 discloses Process 1800 may be performed to detect drift in data used for models, including predictive models (e.g., forecasting models or classification models). Models of process 1800 may include recurrent neural networks, kernel density estimators, or the like. In some embodiments, process 1800 may be performed to detect data drift used in neural network models used for data classification or data forecasting (e.g., economic models, financial models, water resource models, employee performance models, climate models, traffic models, weather models, or the like)). Walters et al. does not explicitly disclose for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state
However, Stockdale et al. discloses for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state (Paragraph 48 discloses multivariate anomaly detector can now use “batching” to identify incremental attacks).
Therefore, it would have been obvious before the effective filing data of invention was made to a person having ordinary skill in the art to modify Walters et al. with Stockdale et al. This would have facilitated resource usage during data streaming analysis. See Stockdale et al. Paragraphs 4-7.
Walters et al. as modified by Stockdale et al. does not disclose determining a signal visualization based on the result.
However, Oberbreckling et al. discloses determining a signal visualization based on the result (Paragraph 8 discloses analysis, generation, and visualization of data obtained from multiple data sources); and
outputting the signal visualization associated with the determined result (Paragraph 118 discloses graphical dashboard may indicate a plurality of metrics, each of the plurality of metrics indicating a real time metric of the data relative to a time that the data is profiled. A graphical visualization may be displayed in a user interface).
Therefore, it would have been obvious before the effective filing data of invention was made to a person having ordinary skill in the art to modify Walters et al. and Stockdale et al. with Oberbreckling et al. This would have facilitated resource usage during data streaming analysis. See Oberbreckling et al. Paragraphs 2-7.
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 2, Walters et al. teaches the method of claim 1, wherein at least a portion of the one or more events have is occurred at distinct points in time (Paragraph 183 discloses a schedule (e.g., event data is received hourly or daily)).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 3, Walters et al. teaches the method of claim 1, wherein the one or more events correspond to information associated with transactions being analyzed to detect fraud (Paragraph 68 discloses a production instance with a data model for detecting fraudulent transactions).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 4, Walters et al. teaches the method of claim 1, wherein one or more features of the set of features are associated with a numerical measurement of data (Paragraph 90 discloses normalize numerical data in the reference dataset as well, mapping the values of the numerical data to a predetermined range).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 5, Walters et al. teaches the method of claim 1, wherein one or more features of the set of features are utilized by a machine learning model for predictive tasks (Paragraph 131 discloses data models perform similarly for both synthetic and actual data. The prediction metrics can include a prediction accuracy check, a prediction accuracy cross check, a regression check, a regression cross check, and a principal component analysis check).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 6, Walters et al. teaches the method of claim 1, wherein using the one or more events to update the corresponding distribution of data from the data stream includes assigning each of the one or more events to a category among a plurality of categories associated with the corresponding distribution of data and correspondingly incrementing counts of events in categories of the plurality of categories (Paragraph 166 discloses characteristics having numerical and/or categorical values).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 7, Walters et al. teaches the method of claim 1, wherein the corresponding distribution of data from the data stream is represented as a histogram (Paragraph 100 discloses generate histograms for each column of data for each of the normalized reference dataset and the synthetic dataset).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 7. With respect to claim 8, Walters et al. teaches the method of claim 7, wherein the histogram is generated including by applying an exponential moving average suppression of older events (Paragraph 100 discloses generate histograms for each column of data for each of the normalized reference dataset and the synthetic dataset. For each bin, system 100 can determine the difference between the count of datapoints in the normalized reference dataset histogram and the synthetic dataset histogram. System 100 can determine the value for this column to be the maximum of the differences for each bin. System 100 can determine the value for the similarity metric to be the average of the values for the columns).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 9, Walters et al. teaches the method of claim 1, wherein the corresponding statistical value is a p-value (Paragraph 48 discloses values, value distributions (e.g., univariate and multivariate statistics of the synthetic data may be similar to that of the actual data)).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 10, Walters et al. teaches the method of claim 1, further comprising receiving, for each feature of the set of features, the corresponding reference distribution and the corresponding distribution of divergences (Paragraph 48 discloses values, value distributions (e.g., univariate and multivariate statistics of the synthetic data may be similar to that of the actual data)).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 11, Walters et al. teaches the method of claim 1, wherein performing the statistical analysis includes performing a multivariate hypothesis test (Paragraph 48 discloses value distributions (e.g., univariate and multivariate statistics of the synthetic data may be similar to that of the actual data), structure and ordering, or the like. In this manner, the data model for the machine learning application can be generated without directly using the actual data).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 12, Walters et al. teaches the method of claim 11, wherein performing the multivariate hypothesis test includes scaling the statistical values (Paragraph 48 discloses value distributions (e.g., univariate and multivariate statistics of the synthetic data may be similar to that of the actual data), structure and ordering, or the like. In this manner, the data model for the machine learning application can be generated without directly using the actual data).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 13, Walters et al. teaches the method of claim 1, wherein the statistical analysis is performed each time a batch of events is received (Paragraph 127 discloses streaming data source 1301 can be configured to retrieve batches of new data).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 1. With respect to claim 14, Walters et al. teaches the method of claim 1, further comprising analyzing the result to determine whether a specified condition has been satisfied (Paragraph 132 discloses a prediction metric satisfies a predetermined threshold).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 14. With respect to claim 15, Simoudis teaches the method of claim 14, further comprising, in response to a determination that the specified condition has been satisfied, providing an alarm (Paragraph 69 discloses Alerts on model accuracy may be generated and delivered when new ground data becomes available. The model monitor system may al so provide dashboards to track model performance/model risk for a portfolio of models based on the training/prediction data and model registration data collected as part of data drift, accuracy and data integrity checks).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 15. With respect to claim 16, Stockdale et al. teaches the method of claim 15, further comprising, in response to the alarm, generating an alarm report that includes: a ranking of features of the set of features according to how much each feature of the set of features contributed to the alarm (Paragraph 180 discloses Non-zero anomaly predictions are made if the “predict” flag in the state is “True”. The centrality processing module toggles the flag to True upon updating the history. The anomaly detector module toggles the flag back to False after a prediction has been made. This flag system works with the existing detector infrastructure, which reports anomalies every minute because other detectors do not need a batch process to complete before predicting anomalies); and
an explanation to identify a root cause associated with the data drift detection (Paragraph 34 discloses the ingestion module monitoring network and email activity, the comparison module to apply one or more models trained on different aspects of this process, and the cyber threat module to identify cyber threats based on comparisons by the comparison module).
The motivation to combine statement previously provided in the rejection of independent claim 15 provided above, combining the Walters et al. reference and the Stockdale et al. reference is applicable to dependent claim 16.
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 15. With respect to claim 17, Walters et al. teaches the method of claim 15, wherein the alarm causes retraining of a machine learning model (Paragraph 169 discloses the computing resources can be configured to reset the model to the original state and retrain the model according to the new hyperparameters).
The Walters et al. as modified by Stockdale et al. and Oberbreckling et al. teaches all the limitations of claim 14. With respect to claim 18, Walters et al. teaches the method of claim 14, wherein the specified condition is associated with one or more comparisons to a threshold value (Paragraph 209 discloses data drift may be detected if a difference meets or exceeds a threshold difference in at least one of: a comparison of a generated data covariance matrix to a current data covariance matrix).
With respect to claim 19, Walters et al. teaches a system, comprising:
one or more processors (Paragraph 12 discloses a processor) configured to:
Receiving one or more events of a data stream (Paragraph 127 discloses streaming data source 1301 can be configured to periodically retrieve all log data created within a certain period (e.g., a five-minute interval). In some embodiments, the data can be application logs. The application logs can include event information);
receiving, for each feature of a set of features, a corresponding triplet of distribution representations comprising (i) a reference distribution derived from a reference dataset, (ii) an initial sample distribution, and (iii) a distribution of divergence values, wherein the distribution of divergence values is generated by sampling a plurality of subsets from the reference dataset, determining a divergence for each of the plurality of subsets relative to the reference distribution, and aggregating the determined divergences (Paragraph 78 discloses dataset generator 103 can be configured with a statistical distribution of subclasses (e.g., a univariate distribution of subclasses) for that class and can select one of the subclasses for generating the synthetic data according to the statistical distribution);
for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state (Paragraph 185 discloses the model may be corrected (updated) based on detected drift. Correcting the model may include model training and/or hyperparameter tuning, consistent with disclosed embodiments. Correcting the model may be involve model training or hyperparameter tuning using the received event data and/or other data);
for each feature of the set of features, using the corresponding determined divergence value and a corresponding distribution of divergences to determine a corresponding statistical value for that class and can select one of the subclasses for generating the synthetic data according to the statistical distribution, wherein the statistical value indicates a probability of observing the determined divergence value based on historical fluctuations characterized by the distribution of divergence values (Paragraph 98 discloses dataset generator 103 can be configured with a statistical distribution of subclasses (e.g., a univariate distribution of subclasses and Paragraph 136 discloses Dataset generator 1307 can then predict the next value using the updated value string that includes the new value. In some embodiments, rather than selecting the most likely new value, dataset generator 1307 can be configured to probabilistically choose a new value);
teaches using the statistical values each corresponding to a different feature of the set of features, performing a statistical analysis to determine a result associated with a likelihood of data drift detection in real-time (Paragraph 176 discloses Process 1800 may be performed to detect drift in data used for models, including predictive models (e.g., forecasting models or classification models). Models of process 1800 may include recurrent neural networks, kernel density estimators, or the like. In some embodiments, process 1800 may be performed to detect data drift used in neural network models used for data classification or data forecasting (e.g., economic models, financial models, water resource models, employee performance models, climate models, traffic models, weather models, or the like)). Walters et al. does not explicitly disclose for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state
However, Stockdale et al. discloses for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state (Paragraph 48 discloses multivariate anomaly detector can now use “batching” to identify incremental attacks).
Therefore, it would have been obvious before the effective filing data of invention was made to a person having ordinary skill in the art to modify Walters et al. with Stockdale et al. This would have facilitated resource usage during data streaming analysis. See Stockdale et al. Paragraphs 4-7.
Walters et al. as modified by Stockdale et al. does not disclose determining a signal visualization based on the result.
However, Oberbreckling et al. discloses determining a signal visualization based on the result (Paragraph 8 discloses analysis, generation, and visualization of data obtained from multiple data sources); and
outputting the signal visualization associated with the determined result (Paragraph 118 discloses graphical dashboard may indicate a plurality of metrics, each of the plurality of metrics indicating a real time metric of the data relative to a time that the data is profiled. A graphical visualization may be displayed in a user interface).
Therefore, it would have been obvious before the effective filing data of invention was made to a person having ordinary skill in the art to modify Walters et al. and Stockdale et al. with Oberbreckling et al. This would have facilitated resource usage during data streaming analysis. See Oberbreckling et al. Paragraphs 2-7.
With respect to claim 20, Walters et al. teaches a computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
Receiving one or more events of a data stream (Paragraph 127 discloses streaming data source 1301 can be configured to periodically retrieve all log data created within a certain period (e.g., a five-minute interval). In some embodiments, the data can be application logs. The application logs can include event information);
receiving, for each feature of a set of features, a corresponding triplet of distribution representations comprising (i) a reference distribution derived from a reference dataset, (ii) an initial sample distribution, and (iii) a distribution of divergence values, wherein the distribution of divergence values is generated by sampling a plurality of subsets from the reference dataset, determining a divergence for each of the plurality of subsets relative to the reference distribution, and aggregating the determined divergences (Paragraph 78 discloses dataset generator 103 can be configured with a statistical distribution of subclasses (e.g., a univariate distribution of subclasses) for that class and can select one of the subclasses for generating the synthetic data according to the statistical distribution);
for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state (Paragraph 185 discloses the model may be corrected (updated) based on detected drift. Correcting the model may include model training and/or hyperparameter tuning, consistent with disclosed embodiments. Correcting the model may be involve model training or hyperparameter tuning using the received event data and/or other data);
for each feature of the set of features, using the corresponding determined divergence value and a corresponding distribution of divergences to determine a corresponding statistical value for that class and can select one of the subclasses for generating the synthetic data according to the statistical distribution, wherein the statistical value indicates a probability of observing the determined divergence value based on historical fluctuations characterized by the distribution of divergence values (Paragraph 98 discloses dataset generator 103 can be configured with a statistical distribution of subclasses (e.g., a univariate distribution of subclasses and Paragraph 136 discloses Dataset generator 1307 can then predict the next value using the updated value string that includes the new value. In some embodiments, rather than selecting the most likely new value, dataset generator 1307 can be configured to probabilistically choose a new value);
teaches using the statistical values each corresponding to a different feature of the set of features, performing a statistical analysis to determine a result associated with a likelihood of data drift detection in real-time (Paragraph 176 discloses Process 1800 may be performed to detect drift in data used for models, including predictive models (e.g., forecasting models or classification models). Models of process 1800 may include recurrent neural networks, kernel density estimators, or the like. In some embodiments, process 1800 may be performed to detect data drift used in neural network models used for data classification or data forecasting (e.g., economic models, financial models, water resource models, employee performance models, climate models, traffic models, weather models, or the like)). Walters et al. does not explicitly disclose for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state
However, Stockdale et al. discloses for each feature of a set of features, using the one or more events to update a corresponding distribution of data from the data stream, wherein the update uses a constant computer memory reference state and a constant computer memory streaming state updated via a recursive technique using a decay factor to consume and consumes fewer computational resources compared with an update that does not use a constant memory reference state and a constant memory streaming state (Paragraph 48 discloses multivariate anomaly detector can now use “batching” to identify incremental attacks).
Therefore, it would have been obvious before the effective filing data of invention was made to a person having ordinary skill in the art to modify Walters et al. with Stockdale et al. This would have facilitated resource usage during data streaming analysis. See Stockdale et al. Paragraphs 4-7.
Walters et al. as modified by Stockdale et al. does not disclose determining a signal visualization based on the result.
However, Oberbreckling et al. discloses determining a signal visualization based on the result (Paragraph 8 discloses analysis, generation, and visualization of data obtained from multiple data sources); and
outputting the signal visualization associated with the determined result (Paragraph 118 discloses graphical dashboard may indicate a plurality of metrics, each of the plurality of metrics indicating a real time metric of the data relative to a time that the data is profiled. A graphical visualization may be displayed in a user interface).
Therefore, it would have been obvious before the effective filing data of invention was made to a person having ordinary skill in the art to modify Walters et al. and Stockdale et al. with Oberbreckling et al. This would have facilitated resource usage during data streaming analysis. See Oberbreckling et al. Paragraphs 2-7.
Relevant Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US PG-PUB 20190121350 is directed to SYSTEMS AND METHODS FOR LEARNING DATA PATTERNS PREDICTIVE OF AN OUTCOME: Paragraph [0228] As billions of IoT devices are deployed, with countless connections, the amount of available data will proliferate. To enable access and utilization of data, the cognitive data marketplace 4102 enables various components, features, services, and processes for enabling users to supply, find, consume, and transact in packages of data, such as batches of data, streams of data (including event streams), data from various data pools 4120, and the like. In embodiments, the cognitive data marketplace 4102 may be included in, connected to, or integrated with, one or more other components of a host processing architecture 4024 of a host processing system 112, such as a cloud-based system, as well as to various sensors, input sources 115, data collection systems 102 and the like. The cognitive data marketplace 4102 may include marketplace interfaces 4108, which may include one or more supplier interfaces by which data suppliers may make data available and one more consumer interfaces by which data may be found and acquired.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS E ALLEN whose telephone number is (571)270-3562. The examiner can normally be reached Monday through Thursday 830-630.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at (571) 270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.E.A/Examiner, Art Unit 2154
/BORIS GORNEY/Supervisory Patent Examiner, Art Unit 2154
Prosecution Insights