Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 03/05/26 have been fully considered but they are not persuasive. Applicant amended preamble which does not receive any patentable weight and moved rejected claims into independent claims so no new amendments were made and final rejection is repeated below.
It is well-settled that collecting and analyzing information by steps people go through in their minds or by mathematical algorithms, without more, are mental processes in the abstract-idea category. Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350, 1353-54 (Fed. Cir. 2016); see SAP Am., Inc. v. InvestPic, LLC, 898 F.3d 1161, 1167 (Fed. Cir. 2018) ("[S]electing certain information, analyzing it using mathematical techniques, and reporting or displaying the results of the analysis" is abstract); Intellectual Ventures I LLC v. Cap. One Fin. Corp., 850 F.3d 1332, 1341 (Fed. Cir. 2017) ("Organizing, displaying, and manipulating data of particular documents" is abstract.); FairWarning IP, LLC v. Iatric Sys., Inc., 839 F.3d 1089, 1096-97 (Fed. Cir. 2016) (compiling and combining disparate data sources to generate a full picture of a user's activity, identity, frequency of activity, and the like in a computer environment to detect potential fraud does not differentiate a process from ordinary mental processes); In re Killian, 45 F.4th 1373, 1379 (Fed. Cir. 2022) ("These steps can be performed by a human, using 'observation, evaluation, judgment, [and] opinion,' because they involve making determinations and identifications, which are mental tasks humans routinely do").
The claims amount to data analysis/manipulation and using some form of AI as a tool. The transformation of data, or the mere "manipulation of basic mathematical constructs [i.e.,] the paradigmatic 'abstract idea,"' is not a transformation sufficient to integrate a judicial exception into a practical application. CyberSource v. Retail Decisions, 654 F.3d 1366, 1372 n.2 (Fed. Cir. 2011) (quoting In re Warmerdam, 33 F.3d 1354, 1355, 1360 (Fed. Cir. 1994)).
Claiming AI on a high level can amount to using a black box without specifying any real details of how the AI operates or what’s in the black box. The claims need to specify the technical details of the AI.
Although the claims may specify an improvement they are only improving the abstract idea not a computer.
Claims do not specify a clear practical application. In order for an abstract idea to be integrated into a practical application, the improvement in a given technical field must be a byproduct of the additional elements. An inventive concept "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself”, as stated in MPEP 2106.5 (1). Applicant should state where within the claim limitations such an improvement is made. Practical applications must be additional elements, not abstract ideas. Prong Two: evaluate whether the claim recites additional elements that integrate the exception into a practical application of the exception.
"It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)) in subsection II, below. In addition, the improvement can be provided by the additional element(s) in combination with the recited judicial exception." paragraph is on 2106.05(a) Improvements to the Functioning of a Computer or To Any Other Technology or Technical Field [R-07.2022].
Regarding arguments pertaining to improvements
MPEP 2106.05(a) states, "the disclosure must provide sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement." The present disclosure provides the requisite detail.
MPEP 2106.05(a) also states, "After the examiner has consulted the specification and
determined that the disclosed invention improves technology, the claim must be evaluated to ensure the claim itself reflects the disclosed improvement in technology."
Applicant would need to be clear about any improvement specified in the disclosure and the improvement should be specified in the claim language as well; one should be able to point to the disclosure and say this part of the claim language is explicitly referring to the part of the disclosure that is discussing the improvement, unless of course, the improvement clearly specifically in the claims. The specification needs to include sufficient details such that one of ordinary skill in the art recognizes the claimed invention as providing an improvement. The claim needs to include the components or steps of the invention that provide the improvement described in the specification. The improvement can't be in the abstract idea itself, there has to be an additional element which integrates the abstract idea into a practical application; even a better way of performing mathematical concepts is still a mathematical concept. Applicant discloses various hardware and software of the claimed invention as being standard and conventional in the art and does not disclose transforming/converting a generic computing machine into a special purpose machine. In short, first the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. The specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art. Conversely, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology. Second, if the specification sets forth an improvement in technology, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. That is, the claim includes the components or steps of the invention that provide the improvement described in the specification. The claim itself does not need to explicitly recite the improvement described in the specification (e.g., "thereby increasing the bandwidth of the channel"). - See MPEP 2106.05(d)(1). Abstract concepts include: observation, evaluation, judgement, and opinion. See Mayo Collaborative Servs. v. Prometheus Labs., Inc., 566 U.S. 66, 71 (2012) (quoting Gottschalk v. Benson, 409 U.S. 63, 67 (1972) ("'[M]ental processes and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work"'). It is the examiner’s position that applicant’s claims do no more than merely invoking generic computer components merely as a tool in which the computer instructions apply the judicial exception. MPEP § 2106.05(f): Mere Instructions to Apply an Exception.
Examples that the courts have indicated may not be sufficient to show an improvement in computer-functionality:
i. Generating restaurant menus with functionally claimed features, Ameranth, 842 F.3d at 1245, 120 USPQ2d at 1857;
ii. Accelerating a process of analyzing audit log data when the increased speed comes solely from the capabilities of a general-purpose computer, FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016);
iii. Mere automation of manual processes, such as using a generic computer to process an application for financing a purchase, Credit Acceptance Corp. v. Westlake Services, 859 F.3d 1044, 1055, 123 USPQ2d 1100, 1108-09 (Fed. Cir. 2017) or speeding up a loan-application process by enabling borrowers to avoid physically going to or calling each lender and filling out a loan application, LendingTree, LLC v. Zillow, Inc., 656 Fed. App'x 991, 996-97 (Fed. Cir. 2016) (non-precedential);
iv. Recording, transmitting, and archiving digital images by use of conventional or generic technology in a nascent but well-known environment, without any assertion that the invention reflects an inventive solution to any problem presented by combining a camera and a cellular telephone, TLI Communications, 823 F.3d at 611-12, 118 USPQ2d at 1747;
v. Affixing a barcode to a mail object in order to more reliably identify the sender and speed up mail processing, without any limitations specifying the technical details of the barcode or how it is generated or processed, Secured Mail Solutions, LLC v. Universal Wilde, Inc., 873 F.3d 905, 910-11, 124 USPQ2d 1502, 1505-06 (Fed. Cir. 2017);
vi. Instructions to display two sets of information on a computer display in a non-interfering manner, without any limitations specifying how to achieve the desired result, Interval Licensing LLC v. AOL, Inc., 896 F.3d 1335, 1344-45, 127 USPQ2d 1553, 1559-60 (Fed. Cir. 2018);
vii. Providing historical usage information to users while they are inputting data, in order to improve the quality and organization of information added to a database, because “an improvement to the information stored by a database is not equivalent to an improvement in the database’s functionality,” BSG Tech LLC v. Buyseasons, Inc., 899 F.3d 1281, 1287-88, 127 USPQ2d 1688, 1693-94 (Fed. Cir. 2018); and
viii. Arranging transactional information on a graphical user interface in a manner that assists traders in processing information more quickly, Trading Technologies v. IBG LLC, 921 F.3d 1084, 1093-94, 2019 USPQ2d 138290 (Fed. Cir. 2019). MPEP 2106.05(a).
In pgs. 10-11, applicant argues Crawley fails to disclose generating a confidence interval.
In response the limitation actually specifies the confidence interval generation circuitry is configured to generate the representation of the confidence interval from boundaries of a plurality of future time series forecasts of the aspect of the system.
Crawley discloses generate the representation of the confidence interval (“The time series data is monitored for anomalies by comparing actual observed values in the time series with the predicted values and confidence intervals”, abstract; “system generates forecasts of the time series data along with confidence intervals”, 0013; “an SCR may specify that an anomaly is detected when an observed datapoint is outside the confidence interval of its prediction”, 0015; “a confidence interval or error bound of the predicted value. In some embodiments, the confidence interval may be generated so that the likelihood that the observed value will fall within the interval is above a certain probability threshold. For example, the confidence interval may be chosen so that the observed value fall within the interval with 90% probability. In some embodiments, the confidence intervals may be generated by the data forecaster 150 itself. In other embodiments, the confidence intervals may be generated by a separate function or machine learning model, based on observed divergence of the predicted values.”, 0024; “the forecast data 160 includes a confidence interval 340 for each predicted value 330”, 0053; “the size of the confidence interval is also configurable. In some embodiments, the forecast model may be configured to generate multiple confidence intervals corresponding to different probability levels, and each of the multiple confidence intervals can be used in the SCRs.”, 0063).
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1,3, 6-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: claims 1,3,6-19 are directed to either a process, machine, manufacture or composition of matter.
With respect to claims 1, 18-19:
2A Prong 1:
label the new measurement as an unknown-unknown (mental process of modeling/classifying with assistance of pen and paper);
determine whether a new measurement falls outside confidence intervals (Abstract idea of analyzing data. Mental process. A human- mind with pen and paper can generate/determine data)
generate, for each set of historical measurements, the future time series forecaster of the aspect of the system (Abstract idea of analyzing data. Mental process. A human- mind with pen and paper can generate/determine data);
generate, for each future time series forecast of the aspect of the system, the confidence interval of the future time series forecast of the aspect of the system (Abstract idea of analyzing data. Mental process. A human- mind with pen and paper can generate/determine data);
generate, for each future time series forecaster of the aspect of the system, the confidence interval of the aspect of the system by using bootstrapping (Abstract idea of analyzing data. Mental process. A human- mind with pen and paper can generate/determine data);
generate the representation of the confidence interval from boundaries of a plurality of future time series forecasts of the aspect of the system (Abstract idea of analyzing data. Mental process. A human- mind with pen and paper can generate/determine data).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
Processing apparatus, storage circuitry, (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
unknown-unknown detection circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
Claim 19 computer-readable medium (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g));
store a plurality of future time series forecasters of an aspect of a system (mere data gathering and output recited at a high level of generality - insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g));
second storage circuitry configured to store (mere data gathering and output recited at a high level of generality - insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g))a plurality of sets of historical measurements of the aspect of a system; forecast circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component); and confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
Processing apparatus, storage circuitry, (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
unknown-unknown detection circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
Claim 19 computer-readable medium (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g));
store a plurality of future time series forecasters of an aspect of a system (mere data gathering and output recited at a high level of generality - insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g));
second storage circuitry configured to store (mere data gathering and output recited at a high level of generality - insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g))a plurality of sets of historical measurements of the aspect of a system; forecast circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component); and confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component);
confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component).
Further, the receiving/storing steps were considered to be extra-solution activity in Step 2A Prong 2, and thus it is re-evaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The court decisions cited in MPEP 2106.05(d)(II) indicate that merely Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information). Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. The claim is not patent eligible.
3. The data processing apparatus according to claim 2, wherein each set of historical measurements in the sets of historical measurements is a time series(further expand mental process user can perform math on incoming data/samples and calculate confidence intervals and determine if data falls outside a range).
6. The data processing apparatus according to claim 2, wherein the confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to generate the representation of the confidence interval from the confidence interval via distillation (mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation).
7. The data processing apparatus according to claim 2, wherein the new measurement is absent from the sets of historical measurements (further expand mental process user can perform math on incoming data/samples and calculate confidence intervals and determine if data falls outside a range).
8. The data processing apparatus according to claim 2, wherein in response to the new measurement being labelled as the unknown- unknown, the new measurement is added to one of the plurality of sets of historical measurements(mere data gathering and output recited at a high level of generality - insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
9. The data processing apparatus according to claim 2, wherein in response to the new measurement being labelled as the unknown- unknown, the new measurement is added to a new set of historical measurements(mere data gathering and output recited at a high level of generality - insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
10. The data processing apparatus according to claim 11, wherein the representation of the confidence interval is generated by random sampling of the confidence interval(mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation).
11. The data processing apparatus according to claim 2, wherein the forecast circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to generate, as the future time series forecaster of the aspect of the system and the confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to generate, as the confidence interval of the future time series forecast of the aspect of the system, probability distributions generated based on the historical measurements of the aspect of the system; and the unknown-unknown detection circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to determine a distance between a test distribution and the probability distributions and in response to the distance between the test distribution and the probability distributions exceeding a threshold, to determine that the test probability distribution represents an unknown-unknown (mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation).
12. The data processing apparatus according to claim 1, comprising: measurement circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) configured to generate the new measurement (mere data gathering and output recited at a high level of generality - insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
13. The data processing apparatus according to claim 12, comprising: forecast circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) configured to generate future time series forecasts of the aspect of the system from the plurality of future time series forecasters; and estimated confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) configured to generate, for each future time series forecast of the future time series forecasts, the confidence interval of that future time series forecast using the representation of the confidence interval associated with that future time series forecast, wherein the confidence interval is an estimated confidence interval(mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation).
14. The data processing apparatus according to claim 13, wherein the representation of the confidence interval is defined as a multi-variate Gaussian distribution(mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation).
15. The data processing apparatus according to claim 13, comprising: error calculation circuitry to calculate an error between at least one of the future time series forecasters and the new measurement, wherein the confidence interval generation circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to adjust the confidence interval of the at least one of the future time series forecasters of the aspect of the system based on the error(mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation).
16. The data processing apparatus according to claim 13, wherein the unknown-unknown detection circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to determine that an unknown-unknown exists in response to a predetermined number of new measurements falling outside the confidence interval associated with each future time series forecaster of the aspect of the system (mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation).
17. The data processing apparatus according to claim 12, wherein the future time series forecaster of the aspect of the system and the confidence interval of the future time series forecast of the aspect of the system are provided as probability distributions generated based on historical measurements of the aspect of the system; the measurement circuitry (computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to generate a plurality of new measurements and a test distribution of the new measurements; and the unknown-unknown detection circuitry(computer component is recited at a high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component) is configured to determine a distance between the test distribution and the probability distributions and in response to the distance between the test distribution and the probability distributions exceeding a threshold (mental process – user can manually perform raw thinking in their head as a first stage and then using paper and pen to perform mathematical operation), to determine that the test probability distribution represents an unknown-unknown (Abstract idea of analyzing data. Mental process. A human- mind with pen and paper can generate/determine data).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 7-9, 12-13, 15-16, 18-19 under 35 U.S.C. 103 as being unpatentable over Cawley (US 2023/0291657) in view of Daughton (US 11,901,076).
Cawley discloses:
1, 18-19. A data processing apparatus comprising:
storage circuitry to store a plurality of future time series forecasters (reads on e.g., forecast models, “forecast models created by users may be stored in a library maintained by the system, so that they can be shared with and reused by other users”, 0062) of an “aspect” (reads on any data sample or time series data being input or stored, 110, 132, 140, 142, Fig. 1; “Systems and methods are disclosed to implement a time series anomaly detection system that uses configurable statistical control rules (SCRs) and a forecasting system to detect anomalies in a time series data (e.g. fluctuating values of a network activity metric)”, abstract; network activity metrics, 0020, 0041; “alerts may cause the assessment service(s) 270 to perform a more in-depth examination 256 of the relevant portions of the event logs stored in the log repository 254. For example, in some embodiments, a malware detection module 272 may examine the machine event logs to detect the installation of a particular type of malware executable.”, 0047) of a system and, for each of the future time series forecasters (“the system forecasts future values of the time series data along with a confidence interval based on seasonality characteristics of the data. The time series data is monitored for anomalies by comparing actual observed values in the time series with the predicted values and confidence intervals”, abstract), a representation of a confidence interval associated with that future time series forecaster (representations of confidence intervals, Figs. 4 and respective disclosure; “the system generates forecasts of the time series data along with confidence intervals”, 0013; or “the size of the confidence interval is also configurable. In some embodiments, the forecast model may be configured to generate multiple confidence intervals corresponding to different probability levels, and each of the multiple confidence intervals can be used in the SCRs”, 0063); and
unknown-unknown (reads on new/unknown data/sample or an anomaly that is outside the confidence intervals, Figs. 4; new and anomalous data can be labeled and becomes historical/known data “the forecasting model may be continuously updated as new data is received”, 0065; “some observation datapoints in the time graph are labelled as confirmed anomalies”, 0069) detection circuitry (“network anomaly detection system”, 0013; 0018-0020, Fig. 1) configured to determine whether a new measurement falls outside confidence intervals generated from the representation of the confidence interval associated with each future time series forecaster of the aspect of the system (“network anomaly detection system”, 0013; 0018-0020, Figs. 1, 4), and in response to the new measurement falling outside the confidence intervals, to label the new measurement as an unknown-unknown (“the detected anomalies are labeled as true anomalies or false positives”, 0082; “network anomaly detection system”, 0013; 0018-0020, Figs. 1, 4)
second storage circuitry configured to store a plurality of sets of historical measurements of the aspect of a system (memory/buffer locations, Fig. 1; 250, Fig. 2; Fig. 3; 542, 548, Fig. 5; 710, Fig. 7; 1020, Fig. 10; “forecaster 150 may be a sinusoidal function that is fitted to the historical behavior of the activity metric”, 0023; “in each time period, a current value 142 for of the time series data is generated by the metric extractor, a next value 162 of the time series data is generated by the data forecaster, and the current value is compared with a previous value predicted by the data forecaster in a previous time period”, 0025; “forecast data may be generated by a static function or a machine-learned forecasting model trained with previous time series data”, 0076);
forecast circuitry configured to generate, for each set of historical measurements, the future time series forecaster of the aspect of the system (“Systems and methods are disclosed to implement a time series anomaly detection system that uses configurable statistical control rules (SCRs) and a forecasting system to detect anomalies in a time series data (e.g. fluctuating values of a network activity metric). In embodiments, the system forecasts future values of the time series data along with a confidence interval based on seasonality characteristics of the data. The time series data is monitored for anomalies by comparing actual observed values in the time series with the predicted values and confidence intervals”, abstract; “the data forecaster 150 will continuously generate forecast data 160 for the activity metric, which comprises a time series of forecast data values 162a-e. For example, in some embodiments, the forecaster 150 may predict one time period ahead of the time series data 140 to generate the next value of the time series data in the future. Each output value of the forecaster (e.g. value 162c) will include a predicted value of the activity metric at a future time interval, and a confidence interval or error bound of the predicted value”, 0024); and
confidence interval generation circuitry configured to generate, for each future time series forecast of the aspect of the system, the confidence interval of the future time series forecast of the aspect of the system (“the system forecasts future values of the time series data along with a confidence interval ”, abstract; “system generates forecasts of the time series data along with confidence intervals”, 0013-0014; 0022, 0024, 0053)
the confidence interval generation circuitry is configured to generate the representation of the confidence interval from boundaries of a plurality of future time series forecasts of the aspect of the system (“The time series data is monitored for anomalies by comparing actual observed values in the time series with the predicted values and confidence intervals”, abstract; “system generates forecasts of the time series data along with confidence intervals”, 0013; “an SCR may specify that an anomaly is detected when an observed datapoint is outside the confidence interval of its prediction”, 0015; “a confidence interval or error bound of the predicted value. In some embodiments, the confidence interval may be generated so that the likelihood that the observed value will fall within the interval is above a certain probability threshold. For example, the confidence interval may be chosen so that the observed value fall within the interval with 90% probability. In some embodiments, the confidence intervals may be generated by the data forecaster 150 itself. In other embodiments, the confidence intervals may be generated by a separate function or machine learning model, based on observed divergence of the predicted values.”, 0024; “the forecast data 160 includes a confidence interval 340 for each predicted value 330”, 0053; “the size of the confidence interval is also configurable. In some embodiments, the forecast model may be configured to generate multiple confidence intervals corresponding to different probability levels, and each of the multiple confidence intervals can be used in the SCRs.”, 0063) confidence interval generation circuitry is configured to generate, for each future time series forecaster of the aspect of the system, the confidence interval of the aspect of the system by using bootstrapping.
Cawley fails to particularly call for bootstrapping
Daughton teaches using bootstrapping (“a desired performance level may be set, and bootstrapping may be performed to get the confidence interval of the uncertainty cutoff (or the threshold range) when the performance of remaining images reaches the desired level”, para nos. 99, 95, 41).
It would have been obvious to combine the references before the effective filing date because they are in the same field of endeavor and bootstrapping is a known method to be used in determining e.g., confidence intervals. Bootstrapping offers several key advantages, including reduced overfitting, improved model stability and accuracy, and the ability to estimate uncertainty in predictions, and is particularly useful when dealing with limited data.
3. The data processing apparatus according to claim 2, wherein each set of historical measurements (time series data) in the sets of historical measurements is a time series (time series data saved in various memories, Fig. 1; “in each time period, a current value 142 for of the time series data is generated by the metric extractor, a next value 162 of the time series data is generated by the data forecaster, and the current value is compared with a previous value predicted by the data forecaster in a previous time period. In this manner, the network anomaly detection system 120 will continuously monitor the time series data 140 for anomalies by examining each observed value 142 in the time series.”, 0025; “ the forecast data may be generated by a static function or a machine-learned forecasting model trained with previous time series data”, 0076).
7. The data processing apparatus according to claim 2, wherein the new measurement is absent from the sets of historical measurements (reads on new, “the forecasting model may be continuously updated as new data is received”, 0065 unknown or anomalous data, Figs. 4; received/processed/labelled data are stored in various memories, 142, Figs. 4; 250, Fig. 2; 300, Fig. 3).
8. The data processing apparatus according to claim 2, wherein in response to the new measurement being labelled as the unknown-unknown, the new measurement is added to one of the plurality of sets of historical measurements (new and anomalous data can be labeled and becomes historical/known data “the forecasting model may be continuously updated as new data is received”, 0065; “some observation datapoints in the time graph are labelled as confirmed anomalies”, 0069; Figs. 4).
9. The data processing apparatus according to claim 2, wherein in response to the new measurement being labelled as the unknown-unknown, the new measurement is added to a new set of historical measurements (new and anomalous data can be labeled and becomes historical/known data “the forecasting model may be continuously updated as new data is received”, 0065; “some observation datapoints in the time graph are labelled as confirmed anomalies”, 0069; Figs. 4).
12. The data processing apparatus according to claim 1, comprising: measurement circuitry configured to generate the new measurement (110, 132, Fig. 1; 230, Fig. 2; “The time series data is monitored for anomalies by comparing actual observed values in the time series with the predicted values and confidence intervals”, abstract).
13. The data processing apparatus according to claim 12, comprising:
forecast circuitry configured to generate future time series forecasts of the aspect of the system from the plurality of future time series forecasters; and
estimated confidence interval generation circuitry configured to generate, for each future time series forecast of the future time series forecasts, the confidence interval of that future time series forecast using the representation of the confidence interval associated with that future time series forecast, wherein the confidence interval is an estimated confidence interval (“the system forecasts future values of the time series data along with a confidence interval based on seasonality characteristics of the data”, abstract; “Each output value of the forecaster (e.g. value 162c) will include a predicted value of the activity metric at a future time interval, and a confidence interval or error bound of the predicted value. In some embodiments, the confidence interval may be generated so that the likelihood that the observed value will fall within the interval is above a certain probability threshold”, 0024).
15. The data processing apparatus according to claim 13, comprising: error calculation circuitry to calculate an error between at least one of the future time series forecasters and the new measurement (Cawley: new, unknown, time series data are compared to confidence intervals and previous data and if they exceed a threshold or distance metric they are labelled as anomalous, Figs. 4; “a next value 162 of the time series data is generated by the data forecaster, and the current value is compared with a previous value predicted by the data forecaster in a previous time period. In this manner, the network anomaly detection system 120 will continuously monitor the time series data 140 for anomalies”, 0025), wherein the confidence interval generation circuitry is configured to adjust the confidence interval of the at least one of the future time series forecasters of the aspect of the system based on the error (Cawley: “Each output value of the forecaster (e.g. value 162c) will include a predicted value of the activity metric at a future time interval, and a confidence interval or error bound of the predicted value. In some embodiments, the confidence interval may be generated so that the likelihood that the observed value will fall within the interval is above a certain probability threshold. For example, the confidence interval may be chosen so that the observed value fall within the interval with 90% probability”, 0024; “In some embodiments, the confidence interval may be generated by the data forecaster itself. In other embodiments, the confidence interval may be generated by a separate component that adjusts the intervals based on the actual values 320 of the time series and the predicted values 330”, 0053).
16. The data processing apparatus according to claim 13, wherein the unknown-unknown detection circuitry is configured to determine that an unknown-unknown exists in response to a predetermined number (reads on Zero) of new measurements falling outside the confidence interval associated with each future time series forecaster of the aspect of the system (Cawley: new, unknown, time series data are compared to confidence intervals and previous data and if they exceed a threshold or distance metric they are labelled as anomalous, Figs. 4; “the distance threshold may be computed based on the confidence intervals, (e.g. a multiple of the size of the confidence interval). If the distance metric is calculated using the formula described previously (where the size of the confidence interval is normalized to a distance of 1), the distance threshold will simply be the selected multiple. As shown in this example, the last observed value of the activity metric deviates significantly from the predicted value, exceeding the configured distance threshold specified for SCR 410. As a result, an anomaly alert will be generated under SCR 410.”, 0056-0057; “the forecaster 150 may be a sinusoidal function that is fitted to the historical behavior of the activity metric”, 0023; “a next value 162 of the time series data is generated by the data forecaster, and the current value is compared with a previous value predicted by the data forecaster in a previous time period. In this manner, the network anomaly detection system 120 will continuously monitor the time series data 140 for anomalies ”, 0025).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Cawley and Daughton as set forth above in view of Zoldi (US 2023/0080851).
6. The data processing apparatus according to claim 1, wherein the confidence interval generation circuitry is configured to generate the representation of the confidence interval from the confidence interval via distillation.
Cawley fails to particularly call for using distillation.
Zoldi teaches distillation (“the method outlined here may take an initial base model (e.g., base model 120) and by exploding the nodes and identifying and eliminating interactions with high variation leads to a new model (e.g., stepdown model 322) with lower levels of predictive variance, thereby helping to ease some of the unintended consequences of model underspecification. Even if the underlying base model is not a neural network, using model distillation, it can be transformed into a neural network, where the above methodology can now be applied”, 0066; “ More concrete information can be provided by a confidence interval and the WoE differential. A confidence interval at a 95% confidence level may be constructed using the 11 model scores by assuming a Gaussian distribution for the scores. In the example above, the sample mean of the scores is 0.747, the sample standard deviation is 0.12, and the multiplier is provided by the corresponding value of the t-distribution, which is 2.228.”, 0078; 0091, 0096).
It would have been obvious to combine the references before the effective filing date because they are in the same field of endeavor and adding distillation can amount to packet filtering malware/anomalies or transforming a model into a neural network.
14. The data processing apparatus according to claim 13, wherein the representation of the confidence interval is defined as a multi-variate Gaussian distribution.
Cawley fails to particularly call for Gaussian.
Zoldi teaches Gaussian (“A confidence interval at a 95% confidence level may be constructed using the 11 model scores by assuming a Gaussian distribution for the scores”, 0078).
It would have been obvious to combine the references before the effective filing date because they are in the same field of endeavor and Gaussian distributions are widely used in machine learning for several reasons, primarily due to their mathematical tractability and the applicability of the Central Limit Theorem. They offer benefits like efficient computation, uncertainty quantification, and flexibility in modeling complex data. Gaussian processes, which utilize Gaussian distributions, are particularly useful for regression, classification, and other tasks, especially when dealing with noisy or uncertain data.
Claim Rejections - 35 USC § 103
Claim(s) 10, 11, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Cawley and Daughton, as set forth above in view of Kobayashi (US 2019/0197435).
10. The data processing apparatus according to claim 1, wherein the representation of the confidence interval is generated by random sampling of the confidence interval.
Cawley fails to particularly call for confidence interval is generated by random sampling
Kobayashi teaches confidence interval is generated by random sampling (“The first confidence interval calculation method is a simple random sampling method. In the first calculation method, a plurality of parameter vectors are sampled from a parameter space 51 by using the MCMC methods or the like. Next, in a data space 52, by using a plurality of prediction performance curves in accordance with the sampled parameter vectors, a probability distribution of estimated prediction performance values corresponding to a sample size x.sub.0 is approximated”, 0136; Fig. 1).
It would have been obvious to combine the references before the effective filing date because they are in the same field of endeavor and generating a confidence interval by using random sampling allows for less data to be saved in memory than sampling every data point.
11. The data processing apparatus according to claim 1, wherein the forecast circuitry is configured to generate, as the future time series forecaster of the aspect of the system and the confidence interval generation circuitry is configured to generate, as the confidence interval of the future time series forecast of the aspect of the system, probability distributions generated based on the historical measurements (Cawley: “forecast data may be generated by a static function or a machine-learned forecasting model trained with previous time series data”, 0076) of the aspect of the system; and
the unknown-unknown detection circuitry is configured to determine a distance between a test distribution (incoming, new, unknown data/sample) and the probability distributions and in response to the distance between the test distribution and the probability distributions exceeding a threshold, to determine that the test probability distribution represents an unknown-unknown (Cawley: new, unknown, time series data are compared to confidence intervals and previous data and if they exceed a threshold or distance metric they are labelled as anomalous, Figs. 4; “the distance threshold may be computed based on the confidence intervals, (e.g. a multiple of the size of the confidence interval). If the distance metric is calculated using the formula described previously (where the size of the confidence interval is normalized to a distance of 1), the distance threshold will simply be the selected multiple. As shown in this example, the last observed value of the activity metric deviates significantly from the predicted value, exceeding the configured distance threshold specified for SCR 410. As a result, an anomaly alert will be generated under SCR 410.”, 0056-0057; “the forecaster 150 may be a sinusoidal function that is fitted to the historical behavior of the activity metric”, 0023; “a next value 162 of the time series data is generated by the data forecaster, and the current value is compared with a previous value predicted by the data forecaster in a previous time period. In this manner, the network anomaly detection system 120 will continuously monitor the time series data 140 for anomalies ”, 0025).
Cawley fails to particularly call for probability distributions.
Kobayashi teaches probability distributions (“The first confidence interval calculation method is a simple random sampling method. In the first calculation method, a plurality of parameter vectors are sampled from a parameter space 51 by using the MCMC methods or the like. Next, in a data space 52, by using a plurality of prediction performance curves in accordance with the sampled parameter vectors, a probability distribution of estimated prediction performance values corresponding to a sample size x.sub.0 is approximated”, 0136).
It would have been obvious to combine the references before the effective filing date because they are in the same field of endeavor and probability distributions offer several key benefits in machine learning, including enhanced uncertainty modeling, improved prediction reliability, and better data understanding. They allow for the representation of uncertainty in data and predictions, leading to more robust and adaptable models.
17. The data processing apparatus according to claim 12, wherein the future time series forecaster of the aspect of the system and the confidence interval of the future time series forecast of the aspect of the system are provided as probability distributions generated based on historical measurements of the aspect of the system;
the measurement circuitry is configured to generate a plurality of new measurements and a test distribution of the new measurements (Cawley: Figs. 1-3); and the unknown-unknown detection circuitry is configured to determine a distance between the test distribution and the probability distributions and in response to the distance between the test distribution and the probability distributions exceeding a threshold, to determine that the test probability distribution represents an unknown-unknown(Cawley: new, unknown, time series data are compared to confidence intervals and previous data and if they exceed a threshold or distance metric they are labelled as anomalous, Figs. 4; “the distance threshold may be computed based on the confidence intervals, (e.g. a multiple of the size of the confidence interval). If the distance metric is calculated using the formula described previously (where the size of the confidence interval is normalized to a distance of 1), the distance threshold will simply be the selected multiple. As shown in this example, the last observed value of the activity metric deviates significantly from the predicted value, exceeding the configured distance threshold specified for SCR 410. As a result, an anomaly alert will be generated under SCR 410.”, 0056-0057; “the forecaster 150 may be a sinusoidal function that is fitted to the historical behavior of the activity metric”, 0023; “a next value 162 of the time series data is generated by the data forecaster, and the current value is compared with a previous value predicted by the data forecaster in a previous time period. In this manner, the network anomaly detection system 120 will continuously monitor the time series data 140 for anomalies ”, 0025).
Cawley fails to particularly call for probability distributions.
Kobayashi teaches probability distributions (“The first confidence interval calculation method is a simple random sampling method. In the first calculation method, a plurality of parameter vectors are sampled from a parameter space 51 by using the MCMC methods or the like. Next, in a data space 52, by using a plurality of prediction performance curves in accordance with the sampled parameter vectors, a probability distribution of estimated prediction performance values corresponding to a sample size x.sub.0 is approximated”, 0136).
It would have been obvious to combine the references before the effective filing date because they are in the same field of endeavor and probability distributions offer several key benefits in machine learning, including enhanced uncertainty modeling, improved prediction reliability, and better data understanding. They allow for the representation of uncertainty in data and predictions, leading to more robust and adaptable models.
Response to Arguments
Applicant's arguments filed 10/7/25 have been fully considered but they are not persuasive. In response to applicants other conclusory arguments pertaining to claims 2-19 see the rejections above.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Edwards (EP 3672153) teaches confidence intervals (models (68) previously trained on historical data, and configured to define confidence intervals for said residuals; and,- output interfaces (66) configured to transmit anomalies based on said residuals and said confidence intervals, to an external application (67)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID R VINCENT whose telephone number is (571)272-3080. The examiner can normally be reached ~Mon-Fri 12-8:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at 5712703428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID R VINCENT/Primary Examiner, Art Unit 2123