Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
This action is responsive to the Amendment filed on 6/16/2026
Claims 1-7, 10-16, 21-26 are pending in the case. Claim(s) 8, 9, 17-20 has/have been cancelled. Claim(s) 21-26 is/are new.
Response to Arguments
Applicant's arguments and amendments with regards to the 35 U.S.C. § 101 rejection of claim(s) 1-7, 10-16, 21-26 have been fully considered and are not persuasive. The 35 U.S.C. § 101 rejection of claim(s) 1-7, 10-16, 21-26 is respectfully maintained.
Regarding claim(s) 1-7, 10-16, 21-26 applicant argues “Revised independent claim 1 recites a concrete technical solution to a specific technical problem. See e.g.,1 [0212] of the application as filed (re the resource utilization associated with using historical data from a CDW), see also 11 [0012]-[0017]. Taken as a whole, the revised claims implement a specific technical solution that improves the functioning of the data access and analysis system by enabling reliable ML model training against historical data stored in a data warehouse that could not otherwise feasibly be used for that purpose. This is analogous to the type of claims found eligible in cases such as Enfish, LLC v. Microsoft Corp., 822 F.3d 1327 (Fed. Cir. 2016) (claims improving the self-referential operation of a database), and McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299 (Fed. Cir. 2016) (specific rules-based
computer-implemented method producing results not previously achievable by prior art techniques).
Applicant accordingly requests that the rejection of claims 1-20 under 35 U.S.C. § 101 be reconsidered and withdrawn”.
Examiner respectfully disagrees.
Applicant’s amended claims are not analogous to Enfish, LLC v. Microsoft Corp., 822 F.3d 1327 (Fed. Cir. 2016) and McRO, Inc. v. Bandai Namco Games Am. Inc, and applicant has provided no information as to how they might be analogous.
As noted below in the 101 rejection, the amended claims are directed towards an abstract idea to determine useful information and using a subset of information based on the determination. The amended claims do not constitute a technological advancement.
Applicant’s arguments and amendments with regards to the 35 U.S.C. § 102 and 103 rejection of claim(s) 1-7, 10-16, 21-26 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-7, 10-16, 21-26 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim(s) 1, 10 and 21 recite(s) “generating filtered sampled historical data by filtering the sampled historical data to exclude invalid dimensions in accordance with the statistical data”.
The original specification does not teach “exclude invalid dimensions in accordance with the statistical data” and therefore does not teach ““generating filtered sampled historical data by filtering the sampled historical data to exclude invalid dimensions in accordance with the statistical data”.
The original specification teaches ignoring invalid dimensions, rather than excluding invalid dimensions from sampled historical data.
Therefore the above noted limitations of claims 1, 10 and 21 do not have support in the original specification.
Claims 2-7, 11-16, 22-26 merely recite additional functions performed by the inventions of claims 1, 10 and 21.
Accordingly, claims 2-7, 11-16, 22-26are also rejected under 35 U.S.C. 112(a).
Claim(s) 6, 15 and 26 recite(s) “filtering the sampled historical data based on the skewness comprises: excluding a dimension in response to a determination that a percentage of rows of the sampled historical data that include matching values for the dimension is greater than a defined skewness threshold”.
The original specification does not teach “filtering the sampled historical data based on the skewness comprises: excluding a dimension in response to a determination that a percentage of rows of the sampled historical data that include matching values for the dimension is greater than a defined skewness threshold”.
Therefore the above noted limitations of claims 6, 15 and 26 do not have support in the original specification.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-7, 10-16, 21-26, are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim(s) 1, 10 and 21 recite(s) “metric predictor models“ which could be multiple models, and “the trained metric predictor model”. It is unclear which of the possible multiple metric predictor models is referred to by “the metric predictor model”, rendering the claim(s) indefinite.
Claim(s) 6, 15 and 26 recite(s) “filtering the sampled historical data based on the skewness comprises: excluding a dimension in response to a determination that a percentage of rows of the sampled historical data that include matching values for the dimension is greater than a defined skewness threshold”.
It is unclear
what is being matched against dimension values, and
how matching values are related to determining skewness
, rendering the claim(s) indefinite.
Claim(s) 2-7, 11-16, 22-26 do not contain claim limitations that cure the indefiniteness of claim(s) 1, 10 and 21 respectively, and therefore are also indefinite under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-7, 10-16, 21-26 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim(s) 1-7 is/are method type claim. Claim(s) 10-16 is/are system type claim(s). Claim(s) 21-26 is/are product type claim(s). Therefore, claims 1-7, 10-16, 21-26 is/are directed to either a process, machine, manufacture or composition of matter.
Independent claim(s):
Step 2A Prong 1:
Regarding claim(s) 1, 10 and 21, this/these claim(s) recite(s)
sampling a training subset of historical data stored in a data warehouse to obtain sampled historical data representative of the training subset;
generating statistical data for dimensions of the sampled historical data, wherein the statistical data includes, on a per-dimension basis for the dimensions, at least one of a cardinality or a skewness;
generating filtered sampled historical data by filtering the sampled historical data to exclude invalid dimensions in accordance with the statistical data;
selecting a trained metric predictor model,
determining that a difference between the predicted value and the current value meets a reporting criterion.
The above limitations of selecting and determining appear to be practically implementable in the human mind and is understood to be a recitation of a mental process and math – a user can mentally sample and filter data using statistical mathematics, select a model and determine whether a difference between two values exceeds a threshold.
Step 2A Prong 2:
Regarding claim(s) 1, 10 and 21, this judicial exception is not integrated into a practical application.
Additional elements:
Regarding claim(s) 10 and 21, this/these claim(s) recite(s) processor and medium to perform the step of selecting and determining (mere instructions stored in a generic memory component to apply the exception using a generic computer component);
Regarding claim(s) 1, 10 and 21, this/these claim(s) further recite(s)
generating trained metric predictor models for predicting values of a data metric by training metric predictor models using the filtered sampled historical data as training data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data);;
a metric predictor model for predicting values of a data metric, the metric predictor model trained using ... data related to the data metric (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data);
testing the trained metric predictor models using a testing subset of the historical data obtained from the data warehouse (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of testing a machine learning model with previously determined data);;
obtaining a predicted value of the data metric using the trained metric predictor model (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level application of a previously trained model to make a prediction);
obtaining a .... value .... using data other than the historical data (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Examiner’s note: the obtaining is recited at a high level of generality and could constitute mere receiving of transmitted information);
historical data related to the data metric, current value of the data metric using data other than the historical data (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h));
in response to determining that the difference meets the reporting criterion, outputting a notification descriptive of the difference (Adding insignificant extra-solution activity to the judicial exception (mere data output of the abstract idea)- see MPEP 2106.05(g)).
The additional element(s) as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above.
Therefore, the claim(s) is/are directed to an abstract idea.
Step 2B:
Regarding claim(s) 1, 10 and 21, this/these claim(s) do/does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
Regarding claim(s) 10 and 21, this/these claim(s) recite(s) processor and medium to perform the step of selecting and determining (mere instructions stored in a generic memory component to apply the exception using a generic computer component);
Regarding claim(s) 1, 10 and 21, this/these claim(s) further recite(s)
generating trained metric predictor models for predicting values of a data metric by training metric predictor models using the filtered sampled historical data as training data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data);;
a metric predictor model for predicting values of a data metric, the metric predictor model trained using ... data related to the data metric (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data);
testing the trained metric predictor models using a testing subset of the historical data obtained from the data warehouse (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of testing a machine learning model with previously determined data);;
obtaining a predicted value of the data metric using the trained metric predictor model (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level application of a previously trained model to make a prediction);
obtaining a .... value .... using data other than the historical data (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Furthermore, MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data” buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed obtaining step is well-understood, routine, conventional activity is supported under Berkheimer);
historical data related to the data metric, current value of the data metric using data other than the historical data (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h));
in response to determining that the difference meets the reporting criterion, outputting a notification descriptive of the difference ((These limitations appear to represents extrasolution activity because it is a mere nominal or tangential addition to the claim, amounting to mere data output, Furthermore, these limitations directed towards outputting information determined by the abstract idea, is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed outputting step is well-understood, routine, conventional activity).
The additional element(s) as disclosed above in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above.
Therefore, the claim(s) is/are not patent eligible.
Step 2A Prong 1, Dependent claims:
Regarding claim(s) 2, 11, 22, this/these claim(s) recite(s) generating time series data of the data metric using the historical data;
Regarding claim(s) 3, 12, 23, this/these claim(s) recite(s) determining that none of the trained at least the subset of the one or more models provides an expected prediction of the second data metric;
Regarding claim(s) 4, 13 and 24, this/these claim(s) recite(s) identifying the dimensions in accordance with previously obtained utility data for the dimensions, wherein identifying the dimensions includes identifying the dimensions in descending utility order;
Regarding claim(s) 5, 14 and 25, this/these claim(s) recite(s) identifying a dimension as invalid in response to determining that a ratio of the cardinality of the dimension to a count of rows in the sampled historical data is greater than a defined cardinality threshold;
Regarding claim(s) 6, 15 and 26, this/these claim(s) recite(s) filtering the sampled historical data based on the skewness comprises: excluding a dimension in response to a determination that a percentage of rows of the sampled historical data that include matching values for the dimension is greater than a defined skewness threshold;
Regarding claim(s) 7 and 16, this/these claim(s) recite(s) including the data other than the historical data in the historical data.
The above limitations appear to be practically implementable in the human mind and is understood to be a recitation of a mental process.
Step 2A Prong 2, Dependent claims:
Regarding claim(s) 2 and 11, this/these claim(s) recite(s) wherein the metric predictor model is trained using the historical data related to the data metric by steps comprising: ... obtaining the metric predictor model using the time series data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data).
Regarding claim(s) 3 and 12, this/these claim(s) recite(s)
wherein the data metric is a first data metric (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h));
receiving a second data metric (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Examiner’s note: the obtaining is recited at a high level of generality and could constitute mere receiving of transmitted information);
training at least a subset of one or more models using historical data related to the second data metric (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data); and
outputting a notification indicating that predicting the second data metric using future data related to the second data metric will not be performed (Adding insignificant extra-solution activity to the judicial exception (mere data output of the abstract idea)- see MPEP 2106.05(g)).
Regarding claim(s) 4, 13 and 20, this/these claim(s) recite(s)
wherein the data metric is a first data metric, wherein the second data metric comprises a dimension (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h));
receiving a second data metric (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Examiner’s note: the obtaining is recited at a high level of generality and could constitute mere receiving of transmitted information);
outputting a notification indicating that the second data metric cannot be predicted based determining that the dimension is invalid (Adding insignificant extra-solution activity to the judicial exception (mere data output of the abstract idea)- see MPEP 2106.05(g)).
Regarding claim(s) 7 and 16, this/these claim(s) recite(s) re-training the metric predictor model using at least the data other than the historical data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data).
Regarding claim(s) 9, this/these claim(s) recite(s) receiving, from a user, an indication (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Examiner’s note: the obtaining is recited at a high level of generality and could constitute mere receiving of transmitted information),
indication whether to use the metric predictor model to predict the data metric (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h)).
The additional element(s) as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above.
Therefore, the claim(s) is/are directed to an abstract idea.
Step 2B, Dependent claims:
Regarding claim(s) 2 and 11, this/these claim(s) recite(s) wherein the metric predictor model is trained using the historical data related to the data metric by steps comprising: ... obtaining the metric predictor model using the time series data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data).
Regarding claim(s) 3 and 12, this/these claim(s) recite(s)
wherein the data metric is a first data metric (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h));
receiving a second data metric (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Furthermore, MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data” buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed obtaining step is well-understood, routine, conventional activity is supported under Berkheimer);
training at least a subset of one or more models using historical data related to the second data metric (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data); and
outputting a notification indicating that predicting the second data metric using future data related to the second data metric will not be performed (Adding insignificant extra-solution activity to the judicial exception (mere data output of the abstract idea)- see MPEP 2106.05(g). Furthermore, these limitations directed towards outputting information determined by the abstract idea, is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed outputting step is well-understood, routine, conventional activity).
Regarding claim(s) 4, 13 and 20, this/these claim(s) recite(s)
wherein the data metric is a first data metric, wherein the second data metric comprises a dimension (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h));
receiving a second data metric (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Furthermore, MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data” buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed obtaining step is well-understood, routine, conventional activity is supported under Berkheimer);
outputting a notification indicating that the second data metric cannot be predicted based determining that the dimension is invalid (Adding insignificant extra-solution activity to the judicial exception (mere data output of the abstract idea)- see MPEP 2106.05(g). Furthermore, these limitations directed towards outputting information determined by the abstract idea, is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed outputting step is well-understood, routine, conventional activity).
Regarding claim(s) 7 and 16, this/these claim(s) recite(s) re-training the metric predictor model using at least the data other than the historical data (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training a machine learning model with previously determined data).
Regarding claim(s) 9, this/these claim(s) recite(s) receiving, from a user, an indication (Adding insignificant extra-solution activity (receiving information) to the judicial exception - see MPEP 2106.05(g). Furthermore, MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data” buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed obtaining step is well-understood, routine, conventional activity is supported under Berkheimer),
indication whether to use the metric predictor model to predict the data metric (These limitations appear to be directed to the specification of information to be used, and is understood to be generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application. MPEP 2106.05(h)).
The additional element(s) as disclosed above in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above.
Therefore, the claim(s) is/are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 10-13, 21-24, are rejected under 35 U.S.C. 103 as being unpatentable over Du (US 20230136094 A1), in view of Barkan (US 20230418909 A1) and Nazir (US 11798090 B1).
Regarding claim 1, Du teaches a method, comprising (Du [9, 82-86] method to perform processes, processor executes instructions stored in non-transitory memory):
sampling a training subset of historical data stored in a data warehouse to obtain sampled historical data representative of the training subset; generating statistical data for dimensions of the sampled historical data, wherein the statistical data includes, on a per-dimension basis for the dimensions, at least one of a cardinality or a skewness; generating …second… sampled historical data …based on… invalid dimensions in accordance with the statistical data (Du [41, 46-48, 52, 54, 55, 79] stored datasets are used, statistical analysis determines cardinality and skew in data and determines important/useful features (valid features) for the data based on the analysis to generate data with meta features);
generating trained metric predictor models for predicting values of a data metric by training metric predictor models using the …second… sampled historical data as training data (Du [41, 51, 56] models are trained using dataset(s) and meta-features);
selecting, from the trained metric predictor models, a trained metric predictor model by testing the trained metric predictor models using a testing … data …set… obtained from the data warehouse (Du [10, 41, 79, 82] models are used for metric service, model for determining data metrics for stored (historical) datasets may be used at appropriate time (selected), model may be trained using stored datasets);
obtaining a predicted value of the data metric using the trained metric predictor model; obtaining a current value of the data metric using data other than the historical data; determining that a ...score based on... the predicted value and the current value meets a reporting criterion (Du [3, 6, 9, 10, 51, 79] trained model predicts value(s) of data metric(s) and ground truth for new (current) data is obtained, score of prediction performance is determined and determination is made whether the score meets metric threshold (reporting criterion); and
in response to determining that the ...score...meets the reporting criterion, outputting a notification descriptive of the difference (Du [43] based on whether or not score(s) meets metric threshold(s)- notification(s) (alerts) are displayed to use).
Du does not specifically teach a difference between the predicted value and the current value;
generating filtered sampled historical data by filtering the sampled historical data to exclude invalid dimensions in accordance with the statistical data;
generating filtered sampled historical data by filtering the sampled historical data to exclude invalid dimensions in accordance with the statistical data;
generating trained metric predictor models for predicting values of a data metric by training metric predictor models using the filtered sampled historical data as training data;
selecting, from the trained metric predictor models, a trained metric predictor model by testing the trained metric predictor models using a testing subset of the historical data obtained from the data warehouse
However Barkan teaches obtaining a predicted value of the data metric using the metric predictor model; obtaining a current value of the data metric using data other than the historical data (Barkan [25-27, 34, 35] model predictions for data metric value(s) (number of instances of dog) are made, predictions are compared to actual data metric values(s) (actual instances of dog), Barkan [21] predictions may be for real-time data) ;
determining that a difference between the predicted value and the current value meets a reporting criterion (Barkan [27, 28, 34, 35, 17] difference between actual and predicted metric (recall) is determined, difference is compared to metric threshold (reporting criterion), Barkan [28] using recall to determine quality of predictions provides higher sensitivity by reducing the false negatives).
It would have been obvious to one of an ordinary skill in the art before the effective filing date of the claimed invention, to have incorporated the concept taught by Barkan of obtaining a predicted value of the data metric using the metric predictor model; obtaining a current value of the data metric using data other than the historical data; determining that a difference between the predicted value and the current value meets a reporting criterion, into the invention suggested by Du,
so that determining that the ...score...meets the reporting criterion, comprises
determining that a difference between the predicted value and the current value meets a reporting criterion, as taught by Barkan;
since both inventions are directed towards analyzing predicted and current values of data metric(s) against reporting criterion, and incorporating the teaching of Barkan into the invention suggested by Du would provide the added advantage of providing higher sensitivity by reducing the false negatives, and the combination would perform with a reasonable expectation of success (Barkan [25-28, 34, 35, 21, 17]).
Du and Barkan does not specifically teach generating filtered sampled historical data by filtering the sampled historical data to exclude invalid dimensions in accordance with the statistical data; generating filtered sampled historical data by filtering the sampled historical data to exclude invalid dimensions in accordance with the statistical data; generating trained metric predictor models for predicting values of a data metric by training metric predictor models using the filtered sampled historical data as training data; selecting, from the trained metric predictor models, a trained metric predictor model by testing the trained metric predictor models using a testing subset of the historical data obtained from the data warehouse
However Nazir teaches generating filtered sampled historical data by filtering the sampled historical data to exclude …unimportant… dimensions in accordance with the statistical data; generating trained metric predictor models for predicting values of a data metric by training metric predictor models using the filtered sampled historical data as training data; selecting, from the trained metric predictor models, a trained metric predictor model by testing the trained metric predictor models using a testing subset of the historical data obtained from the data warehouse (Nazir Abstract Col 18, line 1-14, 21-25, Col 19, lines 23-27, Col 23, lines 1-11, dataset may be historical and split into test and training datasets, dataset may have unimportant dimensions removed to reduce the input size, reduced dataset may be used to train models, model(s) performing best when tested may be selected).
It would have been obvious to one of an ordinary skill in the art before the effective filing date of the claimed invention, to have incorporated the concept taught by Nazir of generating filtered sampled historical data by filtering the sampled historical data to exclude …unimportant… dimensions in accordance with the statistical data; generating trained metric predictor models for predicting values of a data metric by training metric predictor models using the filtered sampled historical data as training data; selecting, from the trained metric predictor models, a trained metric predictor model by testing the trained metric predictor models using a testing subset of the historical data obtained from the data warehouse, into the invention suggested by Du and Barkan; since both inventions are directed towards considering importance of dimensions when training and using a model, and incorporating the teaching of Nazir into the invention suggested by Du and Barkan would provide the added advantage of reducing the input size and allowing best performing model(s) to be used, and the combination would perform with a reasonable expectation of success (Nazir Abstract Col 18, line 1-14, 21-25, Col 19, lines 23-27, Col 23, lines 1-11).
Regarding claim 2, Du, Barkan and Nazir teach the invention as claimed in claim 1 above.
Claim 1 further teaches using the …second… sampled historical data as training data.
Du further teaches wherein the metric predictor model is trained using the …second… sampled historical data related to the data metric by steps comprising: generating time series data of the data metric using the …second… sampled historical data; and obtaining the metric predictor model using the time series data (Du [3, 47, 78-80] data quality metric(s) for stored data may be converted to time series and monitored, based on monitoring adjustments (obtaining model predictor model) may be made to how model decodes new data).
Regarding claim 3, Du, Barkan and Nazir teach the invention as claimed in claim 1 above.
Du further teaches wherein the data metric is a first data metric, further comprising: generating second trained metric predictor models for predicting values of (Du [46, 52] multiple metrics are calculated for data);
training using second historical data related to the second data metric(Du [10] model(s) is/are trained using stored data) ;
determining that none of the second trained metric predictor models provides an expected prediction of the second data metric; and outputting a notification indicating that predicting the second data metric using future data related to the second data metric will not be performed (Du [43, 47, 48] Figs. 4A, 4B determination is made whether models are good for each (second) metric based on threshold, user may be notified (dashboard alert(s)) that threshold is not met, user can use notification to specify whether or not to use model for metric associated with alert).
Regarding claim 4, Du, Barkan and Nazir teach the invention as claimed in claim 1 above. Du further teaches wherein generating the statistical data includes: identifying the dimensions in accordance with previously obtained utility data for the dimensions (Du [51, 52, 57] importance determined for historical data).
Du does not specifically teach wherein identifying the dimensions includes identifying the dimensions in descending utility order
However Nazir teaches wherein identifying the dimensions includes identifying the dimensions in descending utility order (Nazir Col 19, lines 1-10, dimensions are ranked according to importance to pick top-n).
Claim 10 is directed towards a system executing instructions similar in scope to the instructions performed by the method of claim 1, and is rejected under the same rationale.
Du further teaches a device, comprising: a memory; and a processor, the processor configured to execute instructions stored in the memory (Du [9, 82-86] method to perform processes, processor executes instructions stored in non-transitory memory).
Claim(s) 11-13, is/are dependent on claim 10 above, is/are directed towards a system executing instructions similar in scope to the instructions performed by the method of claim(s) 2-4, respectively, and is/are rejected under the same rationale.
Claim 21 is directed towards a medium storing instructions similar in scope to the instructions performed by the method of claim 1, and is rejected under the same rationale. Du further teaches non-transitory computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors of a data access and analysis system, cause the data access and analysis system to (Du [9, 82-86] method to perform processes, processor executes instructions stored in non-transitory memory).
Claim(s) 22-24 is/are dependent on claim 19 above, is/are directed towards a medium storing instructions similar in scope to the instructions performed by the method of claim(s) 2-4 respectively, and is/are rejected under the same rationale.
Claims 5, 14 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Du (US 20230136094 A1) in view of Barkan (US 20230418909 A1) and Nazir (US 11798090 B1), and further in view of Bestgen (US 20150363470 A1).
Regarding claim 5, Du, Barkan and Nazir teach the invention as claimed in claim 1 above. Du does not specifically teach wherein filtering the sampled historical data comprises :identifying a dimension as invalid in response to determining that a ratio of the cardinality of the dimension to a count of rows in the sampled historical data is greater than a defined cardinality threshold
However Bestgen wherein filtering the sampled historical data comprises: identifying a dimension as unimportant in response to determining that a ratio of the cardinality of the dimension to a count of rows in the sampled historical data is greater than a defined cardinality threshold (Bestgen [29] based on cardinality ratio, dimension (column) importance may be determined).
It would have been obvious to one of an ordinary skill in the art before the effective filing date of the claimed invention, to have incorporated the concept taught by Bestgen of wherein filtering the sampled historical data comprises: identifying a dimension as invalid in response to determining that a ratio of the cardinality of the dimension to a count of rows in the sampled historical data is greater than a defined cardinality threshold, into the invention suggested by Du, Barkan and Nazir; since both inventions are directed towards determining importance of dimensions based on cardinality measures, and incorporating the teaching of Bestgen into the invention suggested by Du, Barkan and Nazir would provide the added advantage of using the relative cardinality to determine importance rather than just the number indicated by the cardinality by taking into account the number of datapoints (number of rows), and the combination would perform with a reasonable expectation of success (Bestgen [29]).
Claim(s) 14, is/are dependent on claim 10 above, is/are directed towards a system executing instructions similar in scope to the instructions performed by the method of claim(s) 5, and is/are rejected under the same rationale.
Claim(s) 25 is/are dependent on claim 19 above, is/are directed towards a medium storing instructions similar in scope to the instructions performed by the method of claim(s) 5, and is/are rejected under the same rationale.
Claims 7, 16, are rejected under 35 U.S.C. 103 as being unpatentable over Du (US 20230136094 A1) in view of Barkan (US 20230418909 A1) and Nazir (US 11798090 B1), and further in view of Ghanta (US 20200034665 A1).
Regarding claim 7, Du, Barkan and Nazir teach the invention as claimed in claim 1 above.
Du, Barkan and Nazir do not specifically teach including the data other than the historical data in the historical data; and re-training the metric predictor models
However Ghanta teaches including the data other than the historical data in the historical data; and re-training the metric predictor models (Ghanta [91, 92, 109] difference between predicted value and actual value of data metric(s) for inference data (other data) is determined (deviation), if difference does not satisfy threshold then model may be retrained using other data, Ghanta [38] retraining a model when it is not a good fit for a data set, results in generating a more accurate machine learning model).
It would have been obvious to one of an ordinary skill in the art before the effective filing date of the claimed invention, to have incorporated the concept taught by Ghanta of including the data other than the historical data in the historical data; and re-training the metric predictor models, into the invention suggested by Du, Barkan and Nazir; since both inventions are directed towards analyzing predicted versus actual data metric values provided by a model, and incorporating the teaching of Ghanta into the invention suggested by Du, Barkan and Nazir would provide the added advantage of generating a more accurate machine learning model when a model it is not a good fit for a data set, and the combination would perform with a reasonable expectation of success (Ghanta [91, 92, 109, 38]).
Claim(s) 16 is/are dependent on claim 10 above, is/are directed towards a system executing instructions similar in scope to the instructions performed by the method of claim(s) 7 respectively, and is/are rejected under the same rationale.
Claim(s) 26 is/are dependent on claim 19 above, is/are directed towards a medium storing instructions similar in scope to the instructions performed by the method of claim(s) 7, and is/are rejected under the same rationale.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SANCHITA ROY whose telephone number is (571)272-5310. The examiner can normally be reached Monday-Friday 12-8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SANCHITA ROY
Primary Examiner
Art Unit 2146
/SANCHITA ROY/Primary Examiner, Art Unit 2146
Prosecution Insights