Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/19/2023 was filed before the mailing date of the first office action. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101. Claims 1-7 are directed to a system, claims 8-14 are directed to a method, and claim 15 is are directed to a non-transitory computer-readable storage medium; therefore, claims 1-15 fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter). However, claims 1-15 fall within the judicial exception of an abstract idea, specifically the abstract ideas of “Mental Processes” (including observation, evaluation, and opinion) and “Mathematical Concepts (including mathematical calculations and relationships)”.
Claim 1 recites the following abstract ideas:
Step 2A Prong One:
generate first training data with a vector as a target variable, the vector including values of a plurality of elements output by inputting unlabeled training data to a pre-learned machine learning model (mental step directed to observation, evaluation – a person could generate training data in their mind, potentially assisted by pen and paper (see MPEP 2106.04(a)(2)(III)), based on an observed vector as a target variable, wherein the vector includes values output by inputting unlabeled training data to a pre-learned machine learning model);
generate second training data in which the values of the elements of the vector as the target variable of the first training data are set so that a difference in magnitude of value between at least some of the elements becomes larger (mental step directed to observation, evaluation – a person could generate training data in their mind potentially assisted by pen and paper (see MPEP 2106.04(a)(2)(III)), by setting values in their mind so that a difference in magnitude of between element values becomes larger).
Claim 1 recites the following additional elements:
a learning apparatus comprising: at least one memory configured to store processing instructions, and at least one processor configured to execute the processing instructions; and generate a machine learning model by machine learning using the first training data and the second training data.
Step 2A Prong Two:
The apparatus, memory, processor, and the machine learning model are all interpreted as generic computer components merely utilized to implement the claimed abstract ideas. As the claim does not recite any particular machine learning model nor technical steps associated with generating a machine learning model using training data, this limitation is interpreted as generic computer activity and the insignificant extra-solution activity of merely gathering data associated with the training data. These additional elements do not integrate the abstract idea into a practical application (see MPEP 2106.05(f) and MPEP 2106.05(g)).
Step 2B:
The apparatus, memory, processor, and machine learning model are all interpreted as generic computer components merely utilized to implement the claimed abstract ideas. As the claim does not recite any particular machine learning model nor technical steps associated with generating a machine learning model using training data, this limitation is interpreted as generic computer activity and the well-understood, routine, conventional activity of inputting, or transmitting the training data over a network to the model. These additional elements do not amount to significantly more than the abstract idea (see MPEP 2106.05(d)(II) and MPEP 2106.05(f)).
Claim 2 recites wherein the at least one processor is configured to execute the processing instructions to generate the second training data in which the values of the elements of the vector as the target variable of the first training data are set so that a difference in value between at least one of the elements that has a large value as compared with others of the elements by a preset criterion and another of the elements becomes larger (mental step directed to observation, evaluation – a person could generate training data in their mind, potentially assisted by pen and paper, by setting values in their mind so that a difference in value between a large element value compared to an observed or mentally determined preset criterion and another element value becomes larger).
Claim 3 recites generating the second training data in which the values of the elements of the vector as the target variable of the first training data are set so that a value of an element that has a largest value becomes largest and a difference in value between the element and others of the elements becomes larger (mental step directed to observation, evaluation – a person could generate training data in their mind, potentially assisted by pen and paper, by setting values in their mind so that an element that has a largest value becomes largest and a difference in value between that large element value and another element value becomes larger).
Claim 4 recites generating the second training data by setting, among the values of the elements of the vector as the target variable of the first training data, a value of an element that has a largest value to a value greater than 0 and values of elements other than the element to 0 (mental step directed to observation, evaluation – a person could generate training data in their mind, potentially assisted by pen and paper, by setting values in their mind so that an element with a largest value is set greater than zero and values of other elements are set to 0).
Claim 5 recites generating the second training data by setting a value of a temperature parameter of softmax function to a value smaller than 1, the softmax function being used for generation of the vector as the target variable of the first training data (mental step directed to observation, evaluation – a person could generate training data in their mind, potentially assisted by pen and paper, by setting a value of a temperature parameter of a softmax function to a value smaller than 1. Wherein the softmax function is “being used for generation of the vector. . .” is interpreted as the intended use of the softmax function and does not provide additional patentable weight to this limitation (see MPEP 2103)).
Claim 6 recites generating the machine learning model by machine learning using the second training data in a preset ratio to the first training data (as the claim does not recite any particular machine learning model nor technical steps associated with generating a machine learning model using training data, this limitation is interpreted as an additional element directed to generic computer activity associated with the insignificant extra-solution activity of mere data gathering and the well-understood, routine, conventional activity of inputting, or transmitting the training data in a preset ratio over a network to the model. This additional element does not integrate the claimed abstract ideas into a practical application or amount to significantly more than the claimed abstract ideas (see MPEP 2106.05(d)(II) and MPEP 2106.05(h)).
Claim 7 recites calculating a loss function Lα by Lα = (1-α)L0 + αL1, where a parameter indicating the ratio of the second training data to the first training data is α, a loss function in machine learning using the first training data is L0, and a loss function in machine learning using the second training data is L1; and generate the machine learning model based on the loss function Lα (The broadest reasonable interpretation of calculating the loss function Lα = (1-α)L0 + αL1 includes both a mathematical calculation and a mental step, as a person could calculate this loss function in their mind, potentially assisted by pen and paper. As the claim does not recite any particular machine learning model nor technical steps associated with generating a machine learning model using a loss function, this limitation is interpreted as an additional element directed to generic computer activity associated with the insignificant extra-solution activity of mere data gathering and the well-understood, routine, conventional activity of inputting, or transmitting the loss function data over a network to the model. This additional element does not integrate the claimed abstract ideas into a practical application or amount to significantly more than the claimed abstract ideas (see MPEP 2106.05(d)(II) and MPEP 2106.05(f).
Claim 8 is a method claim and its limitation is included in claim 1. The only difference is that claim 8 requires a method. Therefore, claim 8 is rejected for the same reasons as claim 1.
Claim 9 is a method claim and its limitation is included in claim 2. Claim 9 is rejected for the same reasons as claim 2.
Claim 10 is a method claim and its limitation is included in claim 3. Claim 10 is rejected for the same reasons as claim 3.
Claim 11 is a method claim and its limitation is included in claim 4. Claim 11 is rejected for the same reasons as claim 4.
Claim 12 is a method claim and its limitation is included in claim 5. Claim 12 is rejected for the same reasons as claim 5.
Claim 13 is a method claim and its limitation is included in claim 6. Claim 13 is rejected for the same reasons as claim 6.
Claim 14 is a method claim and its limitation is included in claim 7. Claim 14 is rejected for the same reasons as claim 7.
Claim 15 is a non-transitory computer-readable medium claim and its limitation is included in claim 1. The only difference is that claim 15 requires a non-transitory computer-readable medium. This non-transitory computer-readable medium is interpreted as a generic computer component merely used to apply the claimed abstract ideas as noted in the analysis of claim 1, and does not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea (see MPEP 2106.05(f)). Therefore, claim 15 is rejected for the same reasons as claim 1.
Viewed as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Therefore, the claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Papernot et al (“Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks”, herein Papernot) in view of Erhan et al (“Why Does Unsupervised Pre-training Help Deep Learning?”, herein Erhan).
Regarding claim 1, Papernot teaches a learning apparatus comprising: at least one memory configured to store processing instructions; and at least one processor configured to execute the processing instructions (section V A para. 5 recites “we use machines equipped with Nvidia Tesla K5200 GPUs” (i.e., a computer with at least one processor and memory)) to:
generate first training data with a vector as a target variable, the vector including values of a plurality of elements output by inputting [unlabeled] training data to a pre-learned machine learning model (section II C para. 3 recites “To perform distillation, a large network whose output layer is a softmax is first trained on the original dataset as would usually be done. An example of such a network is depicted in Figure 1”. Section III B para. 3 recites “The resulting defensive distillation training procedure is illustrated in Figure 5 and outlined as follows: 1) The input of the defensive distillation training algorithm is a set X of samples with their class labels. Specifically, let X ϵ X be a sample, we use Y (X) to denote its discrete label, also referred to as hard label. Y (X) is an indicator vector such that the only non-zero element corresponds to the correct class’s index” (i.e., generating training data vectors from a pre-trained model));
generate second training data in which the values of the elements of the vector as the target variable of the first training data are set so that a difference in magnitude of value between at least some of the elements becomes larger (section II B para. 1 recites “consider a sample X and a trained DNN resulting in a classifier model. The goal of the adversary is to produce an adversarial sample X* = X + δX by adding a perturbation δX to sample X, such that F(X*) = Y* where Y* ≠ F(X) is the adversarial target output taking the form of an indicator vector for the target class”. Section II B para. 4 recites “The adversary must now use this knowledge about the network sensitivity to input variations to evaluate which dimensions are most likely to produce the target misclassification with a minimum total perturbation vector. Saliency maps assign values to combinations of input dimensions indicating whether they will contribute to the adversarial goal or not if perturbed. This effectively diminishes the number of input features perturbed to craft samples. The amplitude of the perturbation added to each input dimensions is a fixed parameter in both approaches. Depending on the input nature (images, malware, ...), one method or the other is more suitable to guarantee the existence of adversarial samples crafted using an acceptable perturbation X. An acceptable perturbation is defined in terms of a distance metric over the input dimensions (e.g., a L1;L2 norm)” (i.e., generating a second training data set wherein some vector elements have a different value such that the distance, or magnitude, between elements becomes larger));
and generate a machine learning model by machine learning using the first training data and the second training data (section III B para. 3 recites “We form a new training set, by consider samples of the form (X, F(X)) for X ϵ X. That is, instead of using hard class label Y(X) for X, we use the soft-target F(X) encoding F’s belief probabilities over the label class. Using the new training set {(X, F(X)) : X ϵ X} we then train another DNN model Fd, with the same neural network architecture as F, and the temperature of the softmax layer remains T. This new model is denoted as Fd and referred to as the distilled model” (i.e., generating a machine learning model from the first and second training data sets)).
However, Papernot does not explicitly teach inputting [unlabeled] training data to a pre-learned machine learning model.
Erhan teaches inputting [unlabeled] training data to a pre-learned machine learning model (section 3 para. 1 recites “we believe that greedy layer-wise unsupervised pre-training overcomes the challenges of deep learning by introducing a useful prior to the supervised fine-tuning training procedure”. Section 3 para. 4 recites “During each phase of the greedy unsupervised training strategy, layers are trained to represent the dominant factors of variation extant in the data. This has the effect of leveraging knowledge of X to form, at each layer, a representation of X consisting of statistically reliable features of X that can then be used to predict the output (usually a class label) Y. This perspective places unsupervised pre-training well within the family of learning strategies collectively known as semi-supervised
methods” (i.e., using unlabeled training data during a pre-learning, or training process, for a machine learning model)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine these teachings by utilizing the unsupervised pre-training method from Erhan to pre-train the distillation model from Papernot. Papernot states in section II A that models can be trained using supervised or unsupervised learning methods, but that their illustrative example only considers supervised learning. Erhan states in section 3 that “Not all regularizers are created equal and, in comparison to standard regularization schemes such as L1 and L2 parameter penalization, unsupervised pre-training is dramatically effective. We believe the credit for its success can be attributed to the unsupervised training criteria optimized during unsupervised pre-training”. Accordingly, one of ordinary skill in the art would be motivated to modify the supervised learning method from Papernot with the unsupervised pre-training method from Erhan to optimize the training output.
Regarding claim 2, the combination of Papernot and Erhan teaches the learning apparatus according to claim 1 as mentioned above, wherein the at least one processor is configured to execute the processing instructions to generate the second training data in which the values of the elements of the vector as the target variable of the first training data are set so that a difference in value between at least one of the elements that has a large value as compared with others of the elements by a preset criterion and another of the elements becomes larger (Papernot section II B para. 1 recites “consider a sample X and a trained DNN resulting in a classifier model. The goal of the adversary is to produce an adversarial sample X* = X + δX by adding a perturbation δX to sample X, such that F(X*) = Y* where Y* ≠ F(X) is the adversarial target output taking the form of an indicator vector for the target class”. Papernot section II B para. 4 recites “The adversary must now use this knowledge about the network sensitivity to input variations to evaluate which dimensions are most likely to produce the target misclassification with a minimum total perturbation vector. An acceptable perturbation is defined in terms of a distance metric over the input dimensions (e.g., a L1, L2
norm)” (i.e., generating a second training data set wherein a difference in value, or distance between vector elements becomes larger according to a preset criterion, such as an L1 or L2 norm)).
Regarding claim 3, the combination of Papernot and Erhan teaches the learning apparatus according to claim 1 as mentioned above, wherein the at least one processor is configured to execute the processing instructions to generate the second training data in which the values of the elements of the vector as the target variable of the first training data are set so that a value of an element that has a largest value becomes largest and a difference in value between the element and others of the elements becomes larger (Papernot section II B para. 1 recites “consider a sample X and a trained DNN resulting in a classifier model. The goal of the adversary is to produce an adversarial sample X* = X + δX by adding a perturbation δX to sample X, such that F(X*) = Y* where Y* ≠ F(X) is the adversarial target output taking the form of an indicator vector for the target class”. Papernot section II B para. 4 recites “The adversary must now use this knowledge about the network sensitivity to input variations to evaluate which dimensions are most likely to produce the target misclassification with a minimum total perturbation vector. An acceptable perturbation is defined in terms of a distance metric over the input dimensions (e.g., a L1, L2 norm)”. Papernot fig. 3 and its description recite “Step (1) evaluates the sensitivity of model F at the input point corresponding to sample X. Step (2) uses this knowledge to select a perturbation affecting sample X’s classification. If the resulting sample X + δX is misclassified by model F in the adversarial target class (here 4) instead of the original class (here 1), an adversarial sample X* has been found. If not, the steps can be repeated on updated input X [Wingdings font/0xDF] X + δX” (i.e., generating subsequent training data sets such that a difference between a large, or perturbed, value becomes larger to trick the model into misclassification)).
Regarding claim 4, the combination of Papernot and Erhan teaches the learning apparatus according to claim 1 as mentioned above, wherein the at least one processor is configured to execute the processing instructions to generate the second training data by setting, among the values of the elements of the vector as the target variable of the first training data, a value of an element that has a largest value to a value greater than 0 and values of elements other than the element to 0 (Papernot section II B para. 1 recites “consider a sample X and a trained DNN resulting in a classifier model. The goal of the adversary is to produce an adversarial sample X* = X + δX by adding a perturbation δX to sample X, such that F(X*) = Y* where Y* ≠ F(X) is the adversarial target output taking the form of an indicator vector for the target class”. Papernot section II B para. 4 recites “The adversary must now use this knowledge about the network sensitivity to input variations to evaluate which dimensions are most likely to produce the target misclassification with a minimum total perturbation vector”. Papernot section II C para. 3-4 recite “To perform distillation, a large network whose output layer is a softmax is first trained on the original dataset as would usually be done. An example of such a network is depicted in Figure 1. The higher the temperature of a softmax is, the more ambiguous its probability distribution will be (i.e. all probabilities of the output F(X) are close to 1/N), whereas the smaller the temperature of a softmax is, the more discrete its probability distribution will be (i.e. only one probability in output F(X) is close to 1 and the remainder are close to 0)” (i.e., generating a second training data set such that a largest value is greater than zero, or close to 1, and the other values are close to 0)).
Regarding claim 5, the combination of Papernot and Erhan teaches the learning apparatus according to claim 1 as mentioned above, wherein the at least one processor is configured to execute the processing instructions to generate the second training data by setting a value of a temperature parameter of softmax function to a value smaller than 1, the softmax function being used for generation of the vector as the target variable of the first training data (Papernot section II C para. 3-4 recite “To perform distillation, a large network whose output layer is a softmax is first trained on the original dataset as would usually be done. An example of such a network is depicted in Figure 1. The higher the temperature of a softmax is, the more ambiguous its probability distribution will be (i.e. all probabilities of the output F(X) are close to 1/N), whereas the smaller the temperature of a softmax is, the more discrete its probability distribution will be (i.e. only one probability in output F(X) is close to 1 and the remainder are close to 0)”. Fig. 5 of Papernot and its description recite “Fig. 5: We first train an initial network on data with a softmax temperature of T. We then use the probability vector F(X), which includes additional knowledge about classes compared to a class label, predicted by network F to train a distilled network Fd at temperature T on the same data X” (i.e., a temperature parameter of a softmax function can be set to a value smaller than 1 when generating subsequent, or second training data sets)).
Regarding claim 6, the combination of Papernot and Erhan teaches the learning apparatus according to claim 1 as mentioned above, wherein the at least one processor is configured to execute the processing instructions to generate the machine learning model by machine learning using the second training data in a preset ratio to the first training data (Papernot section II A para. 3 recites “Once the network is trained, the architecture together with its parameter values θF can be considered as a classification function F and the test phase begins: the network is used on unseen inputs X to predict outputs F(X)”. Papernot section II A para. 4 recites “adversarial samples are artifacts of a threat vector against DNNs that can be exploited by adversaries at test time, after network training is completed. Crafted by adding carefully selected perturbations δX to legitimate inputs X, their key property is to provoke a specific behavior from the DNN, as initially chosen by the adversary”. Papernot section V A para. 1 recites “The samples are split between a training set of 60,000 samples and a test set of 10,000” (i.e., a model can be trained using second training data, or test data, that was split into a preset ratio from the first training data, for example, in the 6:1 ratio from section V)).
Claim 8 is a method claim and its limitation is included in claim 1. The only difference is that claim 8 requires a method. Therefore, claim 8 is rejected for the same reasons as claim 1.
Claim 9 is a method claim and its limitation is included in claim 2. Claim 9 is rejected for the same reasons as claim 2.
Claim 10 is a method claim and its limitation is included in claim 3. Claim 10 is rejected for the same reasons as claim 3.
Claim 11 is a method claim and its limitation is included in claim 4. Claim 11 is rejected for the same reasons as claim 4.
Claim 12 is a method claim and its limitation is included in claim 5. Claim 12 is rejected for the same reasons as claim 5.
Claim 13 is a method claim and its limitation is included in claim 6. Claim 13 is rejected for the same reasons as claim 6.
Claim 15 is a non-transitory computer-readable medium claim and its limitation is included in claim 1. The only difference is that claim 15 requires a non-transitory computer-readable medium. Therefore, claim 15 is rejected for the same reasons as claim 1.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Papernot et al (“Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks”, herein Papernot) in view of Erhan et al (“Why Does Unsupervised Pre-training Help Deep Learning?”, herein Erhan), in further view of Wang et al (“Imbalance-XGBoost: leveraging weighted and focal losses for binary label-imbalanced classification with XGBoost”, herein Wang).
Regarding claim 7, the combination of Papernot and Erhan teaches the learning apparatus according to claim 6 as mentioned above.
However, the combination of Papernot and Erhan does not explicitly teach calculating a loss function Lα by Lα = (1-α)L0 + αL1 , where a parameter indicating the ratio of the second training data to the first training data is α, a loss function in machine learning using the first training data is L0 , and a loss function in machine learning using the second training data is L1; and generate the machine learning model based on the loss function Lα.
Wang teaches calculating a loss function Lα by Lα = (1-α)L0 + αL1 , where a parameter indicating the ratio of the second training data to the first training data is α, a loss function in machine learning using the first training data is L0 , and a loss function in machine learning using the second training data is L1; and generate the machine learning model based on the loss function Lα (section 3.2 para. 1 recites “The weighted cross-entropy loss for binary classification can be denoted as follows:
L
w
=
∑
i
=
1
m
(
α
y
i
log
(
y
i
^
)
+
(
1
-
y
i
)
log
(
1
-
y
i
^
)
)
(4)
where α indicates the ’imbalance parameter’. Intuitively, if α is greater than 1, extra loss will be counted on ’classifying 1 as 0’; On the other hand, if α is less than 1, the loss function will weight relatively more on whether data points with label 0 are correctly identified” (i.e., a loss function with an imbalance parameter representing a binary condition between two classifications that can represent a ratio between two data sets used when calculating this loss function. Examiner notes that the total classification loss would then reduce to the claimed equation)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine these teachings by utilizing the loss function from Wang in the distillation model from Papernot. Section IV A, paragraph 3 of Papernot teaches that its training method aims to minimize a cross-entropy loss function between the original dataset and the distilled, or second dataset. One of ordinary skill in the art would recognize that the weighted cross-entropy loss function from Wang could be utilized for this purpose in the model from Papernot.
Claim 14 is a method claim and its limitation is included in claim 7. Claim 14 is rejected for the same reasons as claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230252769 A1 (Bhat et al) teaches a deep learning method for a self-supervised knowledge distillation model to improve the quality of smaller models during a pretraining stage.
“Demystifying Membership Inference Attacks in Machine Learning as a Service” (Truex et al) teaches an generalized framework for the development of a membership inference attack model against different kinds of machine learning models.
“Fast Generalized Distillation for Semi-Supervised Domain Adaptation” (Ao et al) teaches a method for leveraging knowledge from a source domain for semi-supervised domain adaptation without accessing the source data directly.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEAH M FEITL whose telephone number is (571) 272-8350. The examiner can normally be reached on M-F 0900-1700 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/L.M.F./ Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights