Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
Claim Rejections – 35 U.S.C. 112(b)
Applicant’s amendments have been fully considered but they are only partially persuasive.
Applicant argues (pg. 8-9) that their replacement of the terms “frequent” and “synthetic” with “real” and “fake”, respectively, makes the metes and bounds of the limitations sufficiently definite. Applicant cites paragraphs [0020] and [0045]-[0047] of the specification as explanations for that a “real” category is one that appears in the dataset while a “fake” category is one which does not appear in the dataset.
Examiner only partially agrees. Examiner agrees that the specification sufficiently defines the “fake” category as one that is not in the dataset (in 0020, “fake rare values (i.e., simulated contributions)” is mentioned).
However, in regards to the “real” category, Examiner respectfully disagrees. The term “real” is not defined in any of the cited paragraphs [0020] and [0045]- [0047]. Furthermore, upon searching for the term “real” in the specification, there is no mention of how “real” category is defined as one that appears in the dataset.
Claim Rejections – 35 U.S.C. 101
Applicant’s arguments have been fully considered but they are not persuasive.
Applicant argues (pg. 10) that the disclosed system improves upon conventional approaches for quantifying uncertainty in machine learning models by providing an uncertainty interval for any contribution of a rare value, thus constituting an improvement to a computer or technical field.
Examiner respectfully disagrees. This is an improvement to the statistical method of determining uncertainty of a rare value and is not an improvement to a computer or to the field of machine learning directly. Instead, the determination of the uncertainty of the rare value is related to the spread/distribution of the rare values themselves, which is independent from a computer or from the field of uncertainty quantification for machine learning.
Applicant argues (pg. 10-11) that the additional limitations allow for generating uncertainty intervals for the predictive model.
Examiner respectfully disagrees. The cited limitations are simply steps for carrying out the abstract concept of determining the uncertainty of rare values, at a high level of generality (see rejection below for more details). The training of the machine learning model is recited at a high level of generality, where the details (layers, parameters, hyperparameters, etc.) of the model are not reflected in the claim and are essentially “black boxed”. Examiner suggests amending the claim with additional details regarding the specific use of the model and how it directly contributes to the alleged improvement.
Applicant argues (pg. 11-12) that the claim must be considered as a whole, and under that standard, the claim improves the technical field of uncertainty quantification for machine learning.
Examiner respectfully disagrees. Considered as a whole, the claim recites the abstract mathematical idea of determining the uncertainty of the spread/distribution of rare values. It does not improve a computer or the field of machine learning; instead, it only gives better mathematical context (by learning the uncertainty/variance) of a distribution of variables/values. Furthermore, the additional limitations are general, high-level steps for carrying out this abstract mathematical concept.
Claim Rejections – 35 U.S.C. 103
Applicant’s amendments have been fully considered but they are not persuasive.
Applicant argues (pg. 12-13) that Li, in addition to Atay, Alasalmi, and Khan, fail to teach: “creating, by the at least one processor, a second dataset for training the predictive model, the second dataset including an altered version of the plurality of variables in the first dataset, such that one or more variables in the first dataset having a value assigned to a frequent category are replaced in the second dataset with one or more synthetic variables having a corresponding value assigned to a synthetic category”. Applicant argues that Li is “silent regarding the creation of a second, altered dataset for training a predictive model in which one or more variables in a first dataset having a value assigned to a real category are replaced in the second dataset with one or more variables having a corresponding value assigned to a fake category, as presently claimed. The disclosure of Li makes no mention, whatsoever, of replacing variables having a value assigned to a real category with variables assigned to a fake category. Furthermore, because Li is silent regarding the replacement of variables having a value assigned to a real category with variables assigned to a fake category, Li necessarily fails to teach that the fake category occurs in the second, altered dataset with a predefined frequency, as presently claimed”
Examiner respectfully disagrees. Li does indeed teach that the replacement of variables having a real category value with variables having a fake category value is done with the fake category having a predefined frequency. As previously cited, Li states: “[Page 2, Paragraph 1]: “To more effectively obtain information from marginals, we want to reduce the domain size of attributes. We apply recoding and compressing to one-way marginals as well as a few groups of attributes that are obviously highly correlated together. After obtain noisy marginals, we keep marginal cells that have count above a threshold θ. For the cells that are below θ we add them up, if the total is below θ we assign 0 to all these cells. If their total is above θ then we create a new value to represent all values that have low counts. After synthesizing the dataset, this new value is replaced by the values it represents using the original noisy marginal.” Li teaches that the groups that are highly correlated together, or have a high overlapping frequency / represented multiple times in everything but name, are treated as one new value. This means that the real values are being replaced by one fake value, as the values are categorical. Furthermore, Equation 1 on Page 2 shows that the threshold for creating the new categorical fake value is dependent on standard deviation for Gaussian noises, which is a deterministic number once the dataset is defined. While the Gaussian noise is not necessarily deterministic, the error (or standard deviation) is a calculatable scalar number. This shows that since the threshold is predefined, given the values of the dataset, the frequency of the fake values is also in effect predefined.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claims 1, 8, 15 recites the limitation "real (category)", which is a relative term that renders the claim indefinite. The phrase “real category” is not defined by the claim. For examination purposes the examiner will treat the real data as that has any degree of realness. The examiner suggests providing a definition of the real category in the claim.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 5-10, 12-17, 19-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 8-10, 12-17, 19-20 are machine/system/product claims. Claims 1-3, 5-7 are method claims. Therefore, claims 1-3, 5-10, 12-17, 19-20 are directed to either a process, machine, manufacture or composition of matter.
With respect to claim 1:
Step 2A – Prong 1:
… to determine a prediction comprising a target value as a function of one or more variables in the plurality of variables; (mental process – a person can manually apply the predictive model to variables in a dataset to generate a prediction with the assistance of a pen/paper)
…
… to one or more variables including the one or more values to generate the prediction; (mental process – a person can manually apply a predictive model to variables in dataset to generate prediction with the assistance of a pen/paper)
generating, … one or more first uncertainty intervals for one or more contributions of one or more missing values in the plurality of variables, wherein the generating of the one or more first uncertainty intervals for the one or more missing values is based on a distribution of contributions assigned to all values of the one or more variables in the plurality of variables; (mental process – a person can manually calculate uncertainty intervals corresponding to contributions of missing variables with the assistance of a pen/paper)
creating, …, a second dataset for training the predictive model, the second dataset including an altered version of the plurality of variables in the first dataset, such that one or more variables in the first dataset having a value assigned to a real category are replaced in the second dataset with one or more variables having a corresponding value assigned to a fake category, and such that the fake category occurs in the second dataset with a predefined frequency; (mental process – a person can manually create a second dataset for training the predictive model, the second dataset including an altered version of the plurality of variables in the first dataset, such that one or more variables in the first dataset having a value assigned to a real category are replaced in the second dataset with one or more synthetic variables having a corresponding value assigned to a fake category with the assistance of a pen/paper)
…
generating, … one or more second uncertainty intervals for one or more contributions of one or more rare values in the plurality of variables, wherein the generating of the one or more second uncertainty intervals for the one or more rare values is based on the one or more variables having the corresponding values assigned to the fake category, the corresponding value based on at least one contribution from the real category; (mental process – a person can manually calculate uncertainty intervals corresponding to contributions of rare/fake/real variables with the assistance of a pen/paper)
and generating, …, an alert indicative of an accuracy of the at least one prediction, the alert generated based on the one or more first uncertainty intervals and/or the one or more second uncertainty intervals. (mental process – a person can manually generate alert based on the uncertainty intervals with the assistance of a pen/paper)
Step 2A – Prong 2: This judicial exception is not integrated into a practical application.
A computer-implemented method, comprising: … by at least one processor … (mere instructions to apply the exception using a generic computer component – computer/processor applies exception); receiving, …, a first dataset for training a predictive model, the first dataset including a plurality of variables associated with one or more values … (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)). the predictive model is configured … (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.);
training, by the at least one processor, the predictive model, wherein the predictive model is trained using the first dataset; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training predictive model using dataset.);
applying, … the trained predictive model (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.); … by at least one processor … (mere instructions to apply the exception using a generic computer component –processor applies exception)
… by the at least one processor … (mere instructions to apply the exception using a generic computer component –processor applies exception) … and based on the applying, (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.);
…
training, …, the predictive model using the second dataset; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the predictive model using the second dataset.);
… by the at least one processor … (mere instructions to apply the exception using a generic computer component –processor applies exception) … and based on the applying, (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.);
…by the at least one processor… (mere instructions to apply the exception using a generic computer component –processor applies exception)
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
A computer-implemented method, comprising: … by at least one processor … (mere instructions to apply the exception using a generic computer component – computer/processor applies exception); receiving, …, a dataset for training a predictive model, the dataset including a plurality of variables associated with one or more values … (MPEP 2106.05(d)(II) indicate that merely “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – dataset is merely received for training). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer). the predictive model is configured for… (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.);
training, by at least one processor, the predictive model, wherein the predictive model is trained using the received dataset; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training predictive model using dataset.);
applying, … the trained predictive model (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.); … by at least one processor … (mere instructions to apply the exception using a generic computer component –processor applies exception)
… by at least one processor … (mere instructions to apply the exception using a generic computer component –processor applies exception) … and based on the applying, (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.);
…
training, …, the predictive model using the second dataset; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the predictive model using the second dataset.);
… by at least one processor … (mere instructions to apply the exception using a generic computer component –processor applies exception) … and based on the applying, (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of applying using predictive model.);
…by the at least one processor… (mere instructions to apply the exception using a generic computer component –processor applies exception)
With respect to claim 2:
Step 2A – Prong 1:
The method according to claim 1, wherein the first dataset includes at least one missing value including at least one unknown value. (mental process – a person can recognize an unknown or missing value.)
With respect to claim 3:
Step 2A – Prong 1:
The method according to claim 2, wherein the at least one unknown and/or the at the least one missing value is a randomly unknown and/or randomly missing value. (mental process – a person can recognize an randomly unknown or randomly missing value.)
With respect to claim 5:
Step 2A – Prong 1:
The method according to claim 1, wherein at least one variable in the first dataset has at least one value assigned to a rare category. (mental process – a person can recognize a rare value.)
With respect to claim 6:
Step 2A – Prong 1:
The method according to claim 5, wherein the at least one variable includes the at least one value assigned to the rare category, wherein the at least one value is a randomly occurring value. (mental process – a person can recognize a randomly rare value.)
With respect to claim 7:
Step 2A – Prong 1:
The method according to claim 1, wherein the creating of the second dataset comprises altering the plurality of variables such that the synthetic category occurs in the second dataset with a predefined frequency … (mental process – a person can manually create the second dataset comprises altering the plurality of variables such that the synthetic category occurs in the second dataset with a predefined frequency with the assistance of a pen/paper.)
With respect to claim 8:
Step 2A – Prong 1:
Claim 8 (system) is substantially similar to claim 1 (method), but has the following additional elements:
Step 2A – Prong 2:
A system comprising: at least one processor; (mere instructions to apply the exception using a generic computer component –processor applies exception)
and a non-transitory machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising (mere instructions to apply the exception using a generic computer component – storage medium applies exception)
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
Claims 9, 10, 12, 13, 14 are rejected on the same grounds under 35 U.S.C. 101 as claims 2, 3, 5, 6, 7 as they are substantially similar, respectively. Mutatis mutandis.
With respect to claim 15:
Step 2A – Prong 1:
Claim 15 (product) is substantially similar to claim 1 (method), but has the following additional elements:
Step 2A – Prong 2:
A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: (mere instructions to apply the exception using a generic computer component –processor and storage medium applies exception)
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
Claims 16, 17, 19, 20 are rejected on the same grounds under 35 U.S.C. 101 as claims 2, 3, 5, 7 as they are substantially similar, respectively. Mutatis mutandis.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-10, 12-17, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Atay et al. (US10990891B1) hereinafter known as Atay in view of Alasalmi et al. (“Classification Uncertainty of Multiple Imputed Data”), as included by the applicant in IDS 05/10/2022, hereinafter known as Alasalmi in view of Khan et al. (“Striking the Right Balance With Uncertainty”), as included by the applicant in IDS 05/10/2022, hereinafter known as Khan, in view of Li et al. (“DPSyn: Differentially Private Synthetic Data Publication”) hereinafter known as Li.
Regarding independent claim 1, Atay teaches:
A computer-implemented method, comprising: receiving, by at least one processor, a first dataset for training a predictive model, the first dataset including a plurality of variables associated with one or more values, the predictive model is configured to determine a prediction comprising a target value as a function of one or more variables in the plurality of variables; (Atay [Col 5. Line 1-5]: “In general, the metrics described in the present disclosure includes time series data to enable the predictive modeling engine 110 to generate predictions based at least in part on previously collected metrics information.” Atay teaches that the model generates predictions in part based on the variables. Atay [Col. 18 Line 5-7]: “Updating the predictive model, in various embodiments, includes determining new values for various values of the predictive modeling algorithm.” Atay teaches that the predictive model is configured for determination of a target value.)
training, by the at least one processor, the predictive model, wherein the predictive model is trained using the first dataset; (Atay [Col 14. Line 12-22]: “… may be performed under the control of one or more computer systems including executable instructions and/or other data, and may be implemented as executable instructions executing collectively on one or more processors.” Atay teaches that a processor is used to train the predictive model. Atay [Col 14. Line 47-49]: “For example, the predictive modeling engine may designate a portion of the aggregated metrics data as training data” Atay teaches that training the predictive model is done using the received dataset.)
applying, by the at least one processor, the trained predictive model to one or more variables including the one or more values to generate the prediction; (Atay [Col. 5 Line 1-8]: “In general, the metrics described in the present disclosure includes time series data to enable the predictive modeling engine 110 to generate predictions based at least in part on previously collected metrics information. In one example, the metrics data includes central processing unit (CPU) usage information with sub-second granularity obtained from millions of computers systems in a fleet.” Atay teaches that the information from millions of computers, which is part of the dataset, are used to generate a prediction using the model. This is done using a predictive modeling engine which is a trained model.)
…
…
training, by the at least one processor, the predictive model using the second dataset; (Atay [Col 14. Line 12-22]: “… may be performed under the control of one or more computer systems including executable instructions and/or other data, and may be implemented as executable instructions executing collectively on one or more processors.” Atay teaches that a processor is used to train the predictive model. Atay [Col 14. Line 47-49]: “For example, the predictive modeling engine may designate a portion of the aggregated metrics data as training data” Atay teaches that training the predictive model is done using the received dataset.)
applying, by the at least one processor, the trained predictive model to one or more variables including the one or more values to generate the prediction; (Atay [Col. 5 Line 1-8]: “In general, the metrics described in the present disclosure includes time series data to enable the predictive modeling engine 110 to generate predictions based at least in part on previously collected metrics information. In one example, the metrics data includes central processing unit (CPU) usage information with sub-second granularity obtained from millions of computers systems in a fleet.” Atay teaches that the information from millions of computers, which is part of the dataset, are used to generate a prediction using the model. This is done using a predictive modeling engine which is a trained model. The act of training the neural network using different data is taught by using the same reference as the idea is training using data, not particularly the second dataset.)
…
and generating, by the at least one processor, an alert indicative of an accuracy of the at least one prediction, the alert generated based on the one or more first uncertainty intervals and/or the one or more second uncertainty intervals. (Atay [Col. 7 Line 38-41]: “For example, the customer 102 defines a confidence interval of 5 standard deviations above and below a prediction and associate an alarm with severity level 5 with the confidence interval.” Atay teaches generating an alarm if there is a large difference as defined by the user, based on the uncertainty interval between the prediction and the data. This alert is based on the accuracy of the prediction as the threshold is a severity level that is a deviation from the central statistical estimate.)
Atay does not teach:
generating, by at the least one processor, based on the applying, one or more uncertainty intervals corresponding to one or more contributions of one or more missing values corresponding to one or more variables in the plurality of variables;
However, Alasalmi teaches:
generating, by at the least one processor and based on the applying, one or more first uncertainty intervals for one or more contributions of one or more missing values in the plurality of variables, wherein the generating of the one or more first uncertainty intervals for the one or more missing values is based on a distribution of contributions assigned to all values of the one or more variables in the plurality of variables; (Alasalmi [Page 1, Paragraph 4-5]: “Multiple imputation, where two or more possible values are imputed for each missing value, solves these problems but machine learning algorithms cannot directly handle multiple imputed data sets. It is straightforward to estimate the uncertainty, i.e. standard error, of statistical demands with multiple imputed data by using simple rules, called Rubin's rules. These rules, take into account the between-imputation variance and within-imputation variance when estimating the standard error of the statistical estimand in question.” Alasalmi teaches generating a measure of uncertainty corresponding to missing values in the context of multiple imputation. One example as such is generating standard error, which is synonymous to generating the spread of the confidence interval.)
Atay and Alasalmi are in the same field of endeavor as the present invention, as the
references are directed to training predictive models and generating uncertainty intervals of the accuracy of the model. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine training a predictive model, generating a confidence interval, and generating an alarm based on the confidence interval as taught in Atay with multiple imputation to address randomly missing values in the dataset as taught in Alasalmi. Alasalmi provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Atay to include teachings of Alasalmi because the combination would allow for increasing the accuracy of predictive models when datasets contain missing values.
Atay and Alasalmi do not teach:
generating, by the at least one processor and based on the applying, one or more second uncertainty intervals for one or more contributions of one or more rare values in the plurality of variables, wherein the generating of the one or more second uncertainty intervals for the one or more rare values is based on the one or more synthetic variables having the corresponding value assigned to the synthetic category, the corresponding value based on at least one contribution from the frequent category;
However, Khan teaches:
generating, by the at least one processor and based on the applying, one or more second uncertainty intervals for one or more contributions of one or more rare values in the plurality of variables, wherein the generating of the one or more second uncertainty intervals for the one or more rare values is based on the one or more synthetic variables having the corresponding value assigned to the synthetic category, the corresponding value based on at least one contribution from the frequent category; (Khan [Page 3, Paragraph 5]: “Bayesian models can provide uncertainty estimates alongside output predictions. Given an input, the uncertainty estimates correspond to the confidence level for each outcome predicted by the model. We hypothesize that the confidence-level of predictions is directly related to the class representation in the training set. As illustrated in Fig. 3, under-represented classes in the training set lead to higher uncertainty and bigger confidence intervals.” Khan teaches generating uncertainty estimates based on under-represented, or rare, variables. Khan [Page 4, Paragraph 7]: “we propose to represent a single sample as a function of its first and second order moments. To this end, consider that the deep feature representations of input media is randomly sampled from a multi-variate Gaussian distribution: f ∼ N (µf , Σf ), where µf and Σf , respectively denote mean and covariance of the features.” Khan teaches modeling by sampling the variables from a Gaussian distribution. This is the generation of synthetic variables using a model distribution to model the rare values. Khan [Page 5, Paragraph 2]: “We are interested in quantifying the probability of misclassification taking into account the distribution of each sample. It can provide a measure of confidence for loss estimates computed on the training samples.” Khan teaches that this sampling is then used to quantify the probability of misclassification of the rare variables in the form of a confidence interval, which is analogous to an uncertainty interval.)
Khan is in the same field of endeavor as the present invention, since it is directed to training predictive models with datasets containing rare values. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine training a predictive model and generating a confidence interval taught in Atay as modified by Alasalmi with training the model on datasets with rare values as taught in Khan. Khan provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Atay as modified by Alasalmi to include teachings of Khan because the combination would allow for increasing the accuracy of predictive models when datasets contain rare values.
Atay, Alasalmi, and Khan do not teach:
creating, by the at least one processor, a second dataset for training the predictive model, the second dataset including an altered version of the plurality of variables in the first dataset, such that one or more variables in the first dataset having a value assigned to a frequent category are replaced in the second dataset with one or more synthetic variables having a corresponding value assigned to a synthetic category;
However, Li teaches:
creating, by the at least one processor, a second dataset for training the predictive model, the second dataset including an altered version of the plurality of variables in the first dataset, such that one or more variables in the first dataset having a value assigned to a real category are replaced in the second dataset with one or more variables having a corresponding value assigned to a fake category, and such that the fake category occurs in the second dataset with a predefined frequency; (Li [Page 2, Paragraph 1]: “To more effectively obtain information from marginals, we want to reduce the domain size of attributes. We apply recoding and compressing to one-way marginals as well as a few groups of attributes that are obviously highly correlated together. After obtain noisy marginals, we keep marginal cells that have count above a threshold θ. For the cells that are below θ we add them up, if the total is below θ we assign 0 to all these cells. If their total is above θ then we create a new value to represent all values that have low counts. After synthesizing the dataset, this new value is replaced by the values it represents using the original noisy marginal.” Li teaches that the groups that are highly correlated together, or have a high overlapping frequency / represented multiple times in everything but name, are treated as one new value.)
Li is in the same field of endeavor as the present invention, since it is directed to generating and training the model on a second dataset with frequent values replaced by synthetic values. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine training a predictive model and generating a confidence interval taught in Atay as modified by Alasalmi as modified by Khan with generating and training the model on a second dataset with frequent values replaced by synthetic values as taught in Li. Li provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Atay as modified by Alasalmi as modified by Khan to include teachings of Khan because the combination would allow for the data to be privatized, giving the subjects additional security and complying with legal guidelines.
Regarding dependent claim 2, Atay, Alasalmi, and Khan teach:
The method according to claim 1,
Alasalmi teaches:
wherein the first dataset includes at least one missing value including at least one unknown value. (Alasalmi [Page 1, Paragraph 3]: “Many real world data sets are, however cursed with missing data. There are many reason why this the case. For example many medical data sets contain patient records that do not all contain the same variables simply because it is not realistic to expect every patient has been measured for every possible blood test etc.” Alasalmi teaches that some datasets, such as medical datasets, have unknown or missing values. It is reasonable to expect that medical data has a missing value because a patient missed the test and thus the value is unknown.)
The reasons to combine are substantially similar to those of claim 1
Regarding dependent claim 3, Atay, Alasalmi, and Khan teach:
The method according to claim 2,
Alasalmi teaches:
wherein the at least one unknown and/or the at the least one missing value is a randomly unknown and/or randomly missing value. (Alasalmi [Page 2, Paragraph 4]: “If all data points have exactly the same probability of being missing, the data are described as being missing completely at random (MCAR)” Alasalmi teaches that some of the values that are missing is randomly missing, or otherwise known as “missing completely at random”.)
The reasons to combine are substantially similar to those of claim 1
Regarding dependent claim 5, Atay, Alasalmi, and Khan teach:
The method according to claim 1,
Khan teaches:
wherein at least one variable in the first dataset has at least one value assigned to a rare category. (Khan [Page 2, Paragraph 4]: “Further, assume that the normalized class frequencies TAU_A and TAU_B are related as TAU_A + TAU_B = 1” Khan teaches that normalized class frequencies TAU_A and TAU_B, when added up, equal 1 because it is the total frequencies of the variables. Khan [Page 2, Paragraph 5]: “Note that TAU_A NOT EQUAL TAU_B NOT EQUAL 0.5 due to class imbalance and typically |TAU_A – TAU_B| > 0.5 in practical cases where a significant imbalance ratio exists” Khan teaches that the difference between the frequencies of a variable relative to another is greater than 50%. This means that at least one of the variables is much rarer than another variable.)
The reasons to combine are substantially similar to those of claim 1
Regarding dependent claim 6, Atay, Alasalmi, and Khan teach:
The method according to claim 5,
Khan teaches:
wherein the at least one variable includes the at least one value assigned to the rare category, wherein the at least one value is a randomly occurring value. (Khan [Page 5, Paragraph 6]: “Edinburgh Dermofit Image Library (DIL) consists of 1.300 high quality skin lesion images…” Khan teaches that the dataset containing rare values comes from skin lesion images. The specification of the present invention defines “random” as without any causal relationship to the target, with examples being forgotten responses in loan responses and rare occupation in foreign country [0003]. Accordingly, skin lesion images are rare in populations and can be randomly occurring, per the definition in the instant specification.)
The reasons to combine are substantially similar to those of claim 1
Regarding dependent claim 7, Atay, Alasalmi, and Khan teach:
The method according to claim 1,
Li teaches:
wherein the creating of the second dataset comprises altering the plurality of variables such that the synthetic category occurs in the second dataset with a predefined frequency. (Li [Page 2, Paragraph 7]: “We first use Gaussian mechanism (consume privacy budget) to obtain the histogram, and then take only values with estimations above the threshold given in Equation (1).” Li teaches that only the values that have the estimations above the thresholds are altered/replaced with the synthetic data. This threshold is a predetermined value, therefore creating a predetermined frequency.)
The reasons to combine are substantially similar to those of claim 1
Regarding independent claim 8, Atay teaches:
A system comprising: at least one processor; (Atay [Col. 9 Line 46-50]: “The predictive modeling engine 210 may include software or other executable code that when executed by one or more processors causes the predictive modeling engine 210 to implement the predictive modeling algorithm as described above.” Atay teaches the method has a processor for causing the predictive modeling engine to implement the algorithm.)
and a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one processor to perform operations comprising: (Atay [Col. 14 Line 16-26]: “Some or all of the process 500 (or any other processes described, or variations and/or combinations of those processes) may be performed under the control of one or more computer systems including executable instructions and/or other data, and may be implemented as executable instructions executing collectively on one or more processors. The executable instructions and/or other data may be stored on a non-transitory computer-readable storage medium (e.g., a computer program persistently stored on magnetic, optical, or flash media).” Atay teaches that the processes can be implemented as instructions on non-transitory storage medium, executable by a processor.)
The reasons to combine are substantially similar to those of claim 1. The remaining steps of claim 8 are rejected under the same rationale as the analogous steps of claim 1.
Claims 9, 10, 12, 13, 14 (system) are rejected on the same grounds under 35 U.S.C. 103 as claims 2, 3, 5, 6, 7 (method) as they are substantially similar, respectively. Mutatis mutandis.
Regarding independent claim 15, Atay teaches:
A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: (Atay [Col. 9 Line 46-50]: “The predictive modeling engine 210 may include software or other executable code that when executed by one or more processors causes the predictive modeling engine 210 to implement the predictive modeling algorithm as described above.” Atay teaches the method has a processor for causing the predictive modeling engine to implement the algorithm. Atay [Col. 14 Line 16-26]: “Some or all of the process 500 (or any other processes described, or variations and/or combinations of those processes) may be performed under the control of one or more computer systems including executable instructions and/or other data, and may be implemented as executable instructions executing collectively on one or more processors. The executable instructions and/or other data may be stored on a non-transitory computer-readable storage medium (e.g., a computer program persistently stored on magnetic, optical, or flash media).” Atay teaches that the processes can be implemented as instructions on non-transitory storage medium, executable by a processor.)
The reasons to combine are substantially similar to those of claim 1. The remaining steps of claim 15 are rejected under the same rationale as the analogous steps of claim 1.
Claims 16, 17, 19, 20 (product) are rejected on the same grounds under 35 U.S.C. 103 as claims 2, 3, 5, 7 (method) as they are substantially similar, respectively. Mutatis mutandis.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYU HYUNG HAN whose telephone number is (703) 756-5529. The examiner can normally be reached on MF 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kyu Hyung Han/
Examiner
Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights