Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 7-8 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 7:
Claim 7 recites the limitation “wherein determining a measure of robustness of the machine learning model comprises determining a measure of robustness for a given condition corresponding to a give of the parallel set of conditional layers.” The word “give” has no definite meaning in the context of this limitation, and the specification provides no definition. A person having ordinary skill in the art would therefore be unable to determine the metes and bounds of this claim. In further examination, the limitation will be read as reciting “wherein determining a measure of robustness of the machine learning model comprises determining a measure of robustness for a given condition corresponding to
Regarding claim 8:
Claim 8 recites the limitation “the operations further comprising determining an adversarial example based on a given of the parallel set of conditional layers.” The word “given” has no definite meaning in the context of this limitation, and the specification provides no definition. A person having ordinary skill in the art would therefore be unable to determine the metes and bounds of this claim. In further examination, the limitation will be read as reciting “the operations further comprising determining an adversarial example based on ”
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4, 14-15, and 19-20 rejected under 35 U.S.C. 102(a) (1) as being anticipated by Choi et al., US Pre-Grant Publication No. 2017/0124409 (hereafter Choi).
Regarding claim 1 and analogous claim 20:
Choi teaches:
“A non-transitory computer-readable storage medium storing instructions that when executed by one or more processors perform operations comprising”: Choi, paragraph 0008, ”According to an aspect of the present principles, a computer-implemented method for training a convolutional neural network (CNNs) is provided”; Choi, paragraph 0010, “According to another aspect of the present principles, a non-transitory computer-readable storage medium comprising a computer-readable program for training a convolutional neural network (CNN) is presented, wherein the computer-readable program when executed on a computer causes the computer to perform [A non-transitory computer-readable storage medium storing instructions that when executed by one or more processors perform operations] the steps of receiving regions of interest from an image, generating one or more convolutional layers from the image, each of the one or more convolutional layers having at least one convolutional feature within a region of interest, applying at least one cascaded rejection classifier to the regions of interest to generate a subset of the regions of interest, and applying scale dependent pooling to convolutional features within the subset to determine a likelihood of an object category.”
“obtaining, with a computer system, a data set having labeled members with labels designating corresponding members as belonging to corresponding classes”: Choi, paragraph 0045, “Totally, a training set X=[xi , x2, . . . , xN] ∈ ℝm2 cxN, as well as a label set Y = {0, 1} ∈ ℝN, is obtained to learn the rejection classifier [obtaining, with a computer system, a data set having labeled members with labels designating corresponding members as belonging to corresponding classes].”
“training, with the computer system, a machine learning model having deterministic layers and a parallel set of conditional layers each corresponding to a different class among the corresponding classes”: Choi, paragraph 0008, ”According to an aspect of the present principles, a computer-implemented method for training a convolutional neural network (CNNs) is provided [training, with the computer system, a machine learning model]”; Choi, paragraph 0037-0038, “For each convolutional layer 103 [a machine learning model having deterministic layers], the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [a parallel set of conditional layers each corresponding to a different class among the corresponding classes, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
“wherein training includes adjusting parameters of the machine learning model according to an objective function that is differentiable”: Choi, paragraph 0047, “The weight of the FC layer is an identity matrix while the bias is initialized as – δ. The hyperbolic layer provides a nice approximation to the sign function, and is differentiable everywhere, which guarantees that the gradients can be back-propagated to lower layers [adjusting parameters of the machine learning model according to an objective function that is differentiable].”
“storing, with the computer system, the trained machine learning model in memory”: Choi, paragraph 0024, “A data processing system suitable for storing and/or executing program code may include at least on processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code to reduce the number of times code is retrieved from bulk storage during execution [storing, with the computer system, the trained machine learning model in memory].”
Regarding claim 4:
Choi teaches “the medium of claim 1.”
Choi further teaches “wherein the machine learning model further comprises a selection layer configured to select among the parallel set of layers based on a class of input data“: Choi, paragraph 0048, “To achieve this, a selection layer is implemented, which takes, as input, the output indicator of rejection classifiers (e.g., approximated using network layers) and object proposals, and outputs a new and smaller set of proposals for subsequent layers [comprises a selection layer configured to select among the parallel set of layers based on a class of input data]. In the new set of proposals, a large number of proposals have been eliminated while the remaining ones are mostly true positives and hard negatives. Proposals surviving after the selection layer may be more difficult to classify, making the network explicitly learn a more discriminative pattern from them.”
Regarding claim 14:
Choi teaches “the medium of claim 1.”
Choi further teaches “wherein the operations comprise steps for applying the parallel set of conditional layers to the machine learning model”: To achieve this, a selection layer is implemented, which takes, as input, the output indicator of rejection classifiers (e.g., approximated using network layers) and object proposals, and outputs a new and smaller set of proposals for subsequent layers [applying the parallel set of conditional layers to the machine learning model]. In the new set of proposals, a large number of proposals have been eliminated while the remaining ones are mostly true positives and hard negatives. Proposals surviving after the selection layer may be more difficult to classify, making the network explicitly learn a more discriminative pattern from them.”
Regarding claim 15:
Choi teaches “the medium of claim 1.”
Choi further teaches “wherein the parallel set of conditional layers are convolutional layers”: Choi, paragraph 0044, “In an embodiment, the cascaded rejection classifiers (CRCs) 108 are learned to reject non-object proposals at each convolutional layer 103 in a cascaded manner [the parallel set of conditional layers are convolutional layers].”
Regarding claim 19:
Choi teaches “the medium of claim 1.”
Choi further teaches “wherein the machine learning model is a neural network”: Choi, paragraph 0008, ”According to an aspect of the present principles, a computer-implemented method for training a convolutional neural network (CNNs) [the machine learning model is a neural network] is provided.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-3, 13, and 18 rejected under 35 U.S.C. 103 over Choi in view of Rezende et al., “Stochastic Backpropagation and Approximate Inference in Deep Generative Models,” 2014, arXiv:1401.4082v3 (hereafter Rezende).
Regarding claim 2:
Choi teaches “the medium of claim 1.”
Choi further teaches (bold only) “wherein the parallel set of conditional layers are stochastic layers and wherein training includes learning, for at least one parameter in each of the parallel set of stochastic layers, a corresponding distribution to be randomly sampled from during operation of the machine learning model”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Choi does not explicitly teach (bold only) “wherein the parallel set of conditional layers are stochastic layers and wherein training includes learning, for at least one parameter in each of the parallel set of stochastic layers, a corresponding distribution to be randomly sampled from during operation of the machine learning model.”
Rezende teaches (bold only) “wherein the parallel set of conditional layers are stochastic layers and wherein training includes learning, for at least one parameter in each of the parallel set of stochastic layers, a corresponding distribution to be randomly sampled from during operation of the machine learning model”: Rezende, section 2, paragraph 1, “Deep latent Gaussian models (DLGMs) are a general class of deep directed graphical models that consist of Gaussian latent variables at each layer of a processing hierarchy. The model consists of L layers of latent variables. To generate a sample from the model, we begin at the top-most layer (L) by drawing from a Gaussian distribution [a corresponding distribution to be randomly sampled from during operation of the machine learning model]”; Rezende, section 4.2, paragraph 7, “The gradients (14) - (17) are now used to descend the free-energy surface with respect to both the generative and recognition parameters in a single optimisation step. Figure 1(b) shows the flow of computation in DLGMs. Our algorithm proceeds by first performing a forward pass (black arrows), consisting of a bottom-up (recognition) phase and a top-down (generation) phase, which updates the hidden activations of the recognition model and parameters of any Gaussian distributions, and then a backward pass (red arrows) in which gradients are computed using the appropriate backpropagation rule for deterministic and stochastic layers [wherein training includes learning, for at least one parameter in each of the parallel set of stochastic layers].”
Rezende and Choi are analogous arts as they are both related to machine learning model design and training. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the stochastic backpropagation of Rezende with the teachings of Choi to arrive at the present invention, in order to improve model design, as stated in Rezende, Abstract, “We demonstrate on several real-world data sets that by using stochastic backpropagation and variational inference, we obtain models that are able to generate realistic samples of data, allow for accurate imputations of missing data, and provide a useful tool for high-dimensional data visualisation.”
Regarding claim 3:
Choi as modified by Rezende teaches “the medium of claim 2.”
Rezende further teaches:
“wherein: the respective distributions are parametric statistical distributions, each characterized, at least in part, by a respective pair of statistical parameters”: Rezende, section 2, paragraphs 1-2, “Deep latent Gaussian models (DLGMs) are a general class of deep directed graphical models that consist of Gaussian latent variables at each layer of a processing hierarchy. The model consists of L layers of latent variables. To generate a sample from the model, we begin at the top-most layer (L) by drawing from a Gaussian distribution […] This generative process is described as follows:
PNG
media_image1.png
121
404
media_image1.png
Greyscale
where ξl are mutually independent Gaussian variables [parametric statistical distributions, each characterized, at least in part, by a respective pair of statistical parameters].”
"and the operations further comprise learning, using gradient descent, for each of the respective distributions, the respective pairs of statistical parameters based on an objective function, wherein the objective function is differentiable with respect to the respective pairs of statistical parameters of the respective probability distributions”: Rezende, section 4.2, paragraph 7, “The gradients (14) - (17) are now used to descend the free-energy surface with respect to both the generative and recognition parameters in a single optimisation step. Figure 1(b) shows the flow of computation in DLGMs. Our algorithm proceeds by first performing a forward pass (black arrows), consisting of a bottom-up (recognition) phase and a top-down (generation) phase, which updates the hidden activations of the recognition model and parameters of any Gaussian distributions, and then a backward pass (red arrows) in which gradients are computed using the appropriate backpropagation rule for deterministic and stochastic layers [learning, using gradient descent, for each of the respective distributions, the respective pairs of statistical parameters based on an objective function]”; Rezende, section 4.1, paragraphs 1-2, “We introduce an approximate posterior distribution q(.) and apply Jensen's inequality following the variational principle (Beal, 2003) to obtain:
PNG
media_image2.png
132
458
media_image2.png
Greyscale
This objective consists of two terms: the first is the KL-divergence between the variational distribution and the prior distribution (which acts a regulariser), and the second is a reconstruction error [gradients are taken from this objective to adjust Gaussian distribution parameters, hence, the objective function is differentiable with respect to the respective pairs of statistical parameters of the respective probability distributions].”
Rezende and Choi are combinable for the rationale given under claim 2.
Regarding claim 13:
Choi teaches “the medium of claim 1.”
Choi further teaches (bold only) “wherein the operations comprise steps for learning distributions of the parallel set of conditional layers”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Choi does not explicitly teach (bold only) “wherein the operations comprise steps for learning distributions of the parallel set of conditional layers.”
Rezende teaches (bold only) “wherein the operations comprise steps for learning distributions of the parallel set of conditional layers”: Rezende, section 2, paragraph 1, “Deep latent Gaussian models (DLGMs) are a general class of deep directed graphical models that consist of Gaussian latent variables at each layer of a processing hierarchy. The model consists of L layers of latent variables. To generate a sample from the model, we begin at the top-most layer (L) by drawing from a Gaussian distribution”; Rezende, section 4.2, paragraph 7, “The gradients (14) - (17) are now used to descend the free-energy surface with respect to both the generative and recognition parameters in a single optimisation step. Figure 1(b) shows the flow of computation in DLGMs. Our algorithm proceeds by first performing a forward pass (black arrows), consisting of a bottom-up (recognition) phase and a top-down (generation) phase, which updates the hidden activations of the recognition model and parameters of any Gaussian distributions, and then a backward pass (red arrows) in which gradients are computed using the appropriate backpropagation rule for deterministic and stochastic layers [wherein the operations comprise steps for learning distributions of the … layers].”
Rezende and Choi are analogous arts as they are both related to machine learning model design and training. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the stochastic backpropagation of Rezende with the teachings of Choi to arrive at the present invention, in order to improve model design, as stated in Rezende, Abstract, “We demonstrate on several real-world data sets that by using stochastic backpropagation and variational inference, we obtain models that are able to generate realistic samples of data, allow for accurate imputations of missing data, and provide a useful tool for high-dimensional data visualisation.”
Regarding claim 18:
Choi teaches “the medium of claim 1.”
Choi does not explicitly teach “wherein the machine learning model further comprises a regularization layer.”
Rezende teaches “wherein the machine learning model further comprises a regularization layer”: Rezende, section 4.1, paragraph 6, “To allow for the best possible inference, the specification of the recognition model must be flexible enough to provide an accurate approximation of the posterior distribution -- motivating the use of deep neural networks. We regularise the recognition model by introducing additional noise, specifically, bit-flip or drop-out noise at the input layer and small additional Gaussian noise to samples from the recognition model [wherein the machine learning model further comprises a regularization layer]. We use rectified linear activation functions as non-linearities for any deterministic layers of the neural network. We found that such regularisation is essential and without it the recognition model is unable to provide accurate inferences for unseen data points.”
Rezende and Choi are analogous arts as they are both related to machine learning model design and training. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the regularization of Rezende with the teachings of Choi to arrive at the present invention, in order to improve model design, as stated in Rezende, section 4.1, paragraph 6, “We found that such regularisation is essential and without it the recognition model is unable to provide accurate inferences for unseen data points.”
Claims 5-12 and 16-17 rejected under 35 U.S.C. 103 over Choi in view of Lecuyer et al., “Certified Robustness to Adversarial Examples with Differential Privacy,” 2019, arXiv:1802.03471v4 (hereafter Lecuyer).
Regarding claim 5:
Choi teaches “the medium of claim 1.”
Choi further teaches (bold only) “further comprising determining a measure of robustness of the machine learning model based on the trained parallel set of conditional layers”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the trained parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Choi does not explicitly teach (bold only) “further comprising determining a measure of robustness of the machine learning model based on the trained parallel set of conditional layers.”
Lecuyer teaches “further comprising determining a measure of robustness of the machine learning model based on the trained parallel set of conditional layers”: Lecuyer, section II. A, paragraphs 5-6, “Intuitively, a predictive model may be regarded as robust to adversarial examples if its output is insensitive to small changes to any plausible input that may be encountered in deployment. To formalize this notion, we must first establish what qualifies as a plausible input. This is difficult: the adversarial examples literature has not settled on such a definition. Instead, model robustness is typically assessed on inputs from a test set that are not used in model training – similar to how accuracy is assessed on a test set and not a property on all plausible inputs. We adopt this view of robustness. Next, given an input, we must establish a definition for insensitivity to small changes to the input. We say a model f is insensitive, or robust, to attacks of p-norm L on a given input x if f(x) = f(x + α) ) for all α ∈ Bp(L). If f is a multiclass classification model based on label scores (as in §II-A), this is equivalent to:
PNG
media_image3.png
36
464
media_image3.png
Greyscale
where k := f(x) [further comprising determining a measure of robustness of the machine learning model based on the trained … layers]. A small change in the input does not alter the scores so much as to change the predicted label.”
Lecuyer and Choi are analogous arts as they are both related to classification models. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the robustness measurement of Lecuyer with the teachings of Choi to arrive at the present invention, in order to defend a model against adversarial examples, as stated in Lecuyer, Abstract, “Our defense, called PixelDP, is based on a novel connection between robustness against adversarial examples and differential privacy, a cryptographically-inspired privacy formalism, that provides a rigorous, generic, and flexible foundation for defense.”
Regarding claim 6:
Choi as modified by Lecuyer teaches “the medium of claim 5.”
Choi further teaches (bold only) “wherein determining the measure of robustness comprises determining a magnitude based on the trained parallel set of conditional layers”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the trained parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Lecuyer further teaches (bold only) “wherein determining the measure of robustness comprises determining a magnitude based on the trained parallel set of conditional layers”: Lecuyer, section II. D, paragraph 4, “Proposition 1. (Robustness Condition) Suppose A satisfies (ε, δ)-PixelDP with respect to a p-norm metric. For any input x, if for some k ∈ K,
PNG
media_image4.png
37
421
media_image4.png
Greyscale
then the multiclass classification model based on label probability vector y(x) = (E(A1(x)), . . . , E(AK(x))) is robust to attacks α of size ||α|| ≤ 1 on input x [determining the measure of robustness comprises determining a magnitude based on the … layers].”
Lecuyer and Choi are combinable for the rationale given under claim 5.
Regarding claim 7:
See the 112(b) rejection, above, for how this claim is being read.
Choi as modified by Lecuyer teaches “the medium of claim 5.”
Choi further teaches (bold only) “wherein determining a measure of robustness of the machine learning model comprises determining a measure of robustness for a given condition corresponding to a give of the parallel set of conditional layers”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Lecuyer further teaches (bold only) “wherein determining a measure of robustness of the machine learning model comprises determining a measure of robustness for a given condition corresponding to a give of the parallel set of conditional layers”: Lecuyer, section II. A, paragraphs 5-6, “Intuitively, a predictive model may be regarded as robust to adversarial examples if its output is insensitive to small changes to any plausible input that may be encountered in deployment. To formalize this notion, we must first establish what qualifies as a plausible input. This is difficult: the adversarial examples literature has not settled on such a definition. Instead, model robustness is typically assessed on inputs from a test set that are not used in model training – similar to how accuracy is assessed on a test set and not a property on all plausible inputs. We adopt this view of robustness. Next, given an input, we must establish a definition for insensitivity to small changes to the input. We say a model f is insensitive, or robust, to attacks of p-norm L on a given input x if f(x) = f(x + α) ) for all α ∈ Bp(L). If f is a multiclass classification model based on label scores (as in §II-A), this is equivalent to:
PNG
media_image3.png
36
464
media_image3.png
Greyscale
where k := f(x) [wherein determining a measure of robustness of the machine learning model comprises determining a measure of robustness for a given condition corresponding to … the … layers, for a given condition interpreted as including relating to a particular class of a multiclass classification model]. A small change in the input does not alter the scores so much as to change the predicted label.”
Lecuyer and Choi are combinable for the rationale given under claim 5.
Regarding claim 8:
See the 112(b) rejection, above, for how this claim is being read.
Choi teaches “the medium of claim 1.”
Choi further teaches (bold only) “the operations further comprising determining an adversarial example based on a given of the parallel set of conditional layers”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Choi does not explicitly teach (bold only) “the operations further comprising determining an adversarial example based on a given of the parallel set of conditional layers.”
Lecuyer teaches (bold only) “the operations further comprising determining an adversarial example based on a given of the parallel set of conditional layers”: Lecuyer, section II, paragraph 2, “Adversarial examples are a class of attack against ML models, studied particularly on deep neural networks for multiclass image classification. The attacker constructs a small change to a given, fixed input, that wildly changes the predicted output. Notationally, if the input is x, we denote an adversarial version of that input by x + α, where α is the change or perturbation introduced by the attacker [determining an adversarial example based on … conditional layers, interpreted as including determining adversarial status in relation to classification]. When x is a vector of pixels (for images), then xi is the i’th pixel in the image and αi is the change to the i’th pixel”; Lecuyer, section I, paragraph 6, “A PixelDP DNN includes in its architecture a DP noise layer that randomizes the network’s computation, to enforce DP bounds on how much the distribution over its predictions can change with small, norm-bounded changes in the input. At inference time, we leverage these DP bounds to implement a certified robustness check for individual predictions. Passing the check for a given input guarantees that no perturbation exists up to a particular size that causes the network to change its prediction.”
Lecuyer and Choi are analogous arts as they are both related to classification models. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the robustness measurement of Lecuyer with the teachings of Choi to arrive at the present invention, in order to defend a model against adversarial examples, as stated in Lecuyer, Abstract, “Our defense, called PixelDP, is based on a novel connection between robustness against adversarial examples and differential privacy, a cryptographically-inspired privacy formalism, that provides a rigorous, generic, and flexible foundation for defense.”
Regarding claim 9:
Choi teaches “the medium of claim 1.”
Choi further teaches (bold only) “the operations further comprising generating a set of adversarial attack training data based on the parallel set of conditional layers”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Choi does not explicitly teach (bold only) “the operations further comprising generating a set of adversarial attack training data based on the parallel set of conditional layers.”
Lecuyer teaches (bold only) “the operations further comprising generating a set of adversarial attack training data based on the parallel set of conditional layers”: Lecuyer, section II, paragraph 2, “Adversarial examples are a class of attack against ML models, studied particularly on deep neural networks for multiclass image classification. The attacker constructs a small change to a given, fixed input, that wildly changes the predicted output. Notationally, if the input is x, we denote an adversarial version of that input by x + α, where α is the change or perturbation introduced by the attacker. When x is a vector of pixels (for images), then xi is the i’th pixel in the image and αi is the change to the i’th pixel”; Lecuyer, section I, paragraph 6, “A PixelDP DNN includes in its architecture a DP noise layer that randomizes the network’s computation, to enforce DP bounds on how much the distribution over its predictions can change with small, norm-bounded changes in the input [generating a set of adversarial attack training data based on … conditional layers, interpreted as including generating adversarial training data in relation to classification]. At inference time, we leverage these DP bounds to implement a certified robustness check for individual predictions. Passing the check for a given input guarantees that no perturbation exists up to a particular size that causes the network to change its prediction”; Lecuyer, section III. A, paragraph 2, “Training an (ε, δ )-PixelDP network is similar to training the original network: we use the original loss and optimizer, such as stochastic gradient descent. The major difference is that we alter the pre-noise computation to constrain its sensitivity with regards to p-norm input changes. Denote Q(x) = h(g(x)), where g is the pre-noise computation and h is the subsequent computation that produces Q(x) in the original network.”
Lecuyer and Choi are analogous arts as they are both related to classification models. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the robustness measurement of Lecuyer with the teachings of Choi to arrive at the present invention, in order to defend a model against adversarial examples, as stated in Lecuyer, Abstract, “Our defense, called PixelDP, is based on a novel connection between robustness against adversarial examples and differential privacy, a cryptographically-inspired privacy formalism, that provides a rigorous, generic, and flexible foundation for defense.”
Regarding claim 10:
Choi as modified by Lecuyer teaches “the medium of claim 9.”
Lecuyer further teaches “the operations further comprising additionally training the machine learning model based on the set of adversarial attack training data”: Lecuyer, section II, paragraph 2, “Adversarial examples are a class of attack against ML models, studied particularly on deep neural networks for multiclass image classification. The attacker constructs a small change to a given, fixed input, that wildly changes the predicted output. Notationally, if the input is x, we denote an adversarial version of that input by x + α, where α is the change or perturbation introduced by the attacker. When x is a vector of pixels (for images), then xi is the i’th pixel in the image and αi is the change to the i’th pixel”; Lecuyer, section I, paragraph 6, “A PixelDP DNN includes in its architecture a DP noise layer that randomizes the network’s computation, to enforce DP bounds on how much the distribution over its predictions can change with small, norm-bounded changes in the input. At inference time, we leverage these DP bounds to implement a certified robustness check for individual predictions. Passing the check for a given input guarantees that no perturbation exists up to a particular size that causes the network to change its prediction”; Lecuyer, section III. A, paragraph 2, “Training an (ε, δ )-PixelDP network is similar to training the original network: we use the original loss and optimizer, such as stochastic gradient descent. The major difference is that we alter the pre-noise computation to constrain its sensitivity with regards to p-norm input changes. Denote Q(x) = h(g(x)), where g is the pre-noise computation and h is the subsequent computation that produces Q(x) in the original network [training the machine learning model based on the set of adversarial attack training data].”
Lecuyer and Choi are combinable for the rationale given under claim 9.
Regarding claim 11:
Choi teaches “the medium of claim 1.”
Choi further teaches (bold only) “wherein training according to the objective function includes adjusting parameters of the machine learning model to maximize noise in the parallel set of conditional layers while minimizing loss in the model”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Choi does not explicitly teach (bold only) “wherein training according to the objective function includes adjusting parameters of the machine learning model to maximize noise in the parallel set of conditional layers while minimizing loss in the model.”
Lecuyer teaches (bold only) “wherein training according to the objective function includes adjusting parameters of the machine learning model to maximize noise in the parallel set of conditional layers while minimizing loss in the model”: Lecuyer, section III. A, paragraph 2, “Training an (ε, δ )-PixelDP network is similar to training the original network: we use the original loss and optimizer, such as stochastic gradient descent [training according to the objective function includes adjusting parameters of the machine learning model]. The major difference is that we alter the pre-noise computation to constrain its sensitivity with regards to p-norm input changes. Denote Q(x) = h(g(x)), where g is the pre-noise computation and h is the subsequent computation that produces Q(x) in the original network”; Lecuyer, section III. D, paragraph 4, “Thus far, we have described PixelDP certificates as binary with respect to a fixed attack bound, L: we either meet or do not meet a robustness check for L. In fact, our formalism allows for a more nuanced certificate, which gives the maximum attack size Lmax (measured in p-norm) against which the prediction on input x is guaranteed to be robust: no attack within this size from will be able to change the highest probability [to maximize noise in the … layers while minimizing loss in the model, interpreted as determining a maximum degree of noise or alteration that should not result in model loss].”
Lecuyer and Choi are analogous arts as they are both related to classification models. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the robustness measurement of Lecuyer with the teachings of Choi to arrive at the present invention, in order to defend a model against adversarial examples, as stated in Lecuyer, Abstract, “Our defense, called PixelDP, is based on a novel connection between robustness against adversarial examples and differential privacy, a cryptographically-inspired privacy formalism, that provides a rigorous, generic, and flexible foundation for defense.”
Regarding claim 12:
Choi teaches “the medium of claim 1.”
Choi further teaches (bold only) “wherein training according to the objective function includes adjusting parameters of the machine learning model to minimize noise in the parallel set of conditional layers while maximizing accuracy of the model”: Choi, paragraph 0037-0038, “For each convolutional layer 103, the set of input ROIs 104, 106 are progressively reduced using each convolutional layer's feature 105 and at least one cascaded rejection classifier (CRC) to generate a new set of ROIs 110 that are a subset of the input ROIs 104, 106. […] A cascaded rejection classifier (CRC) 108 can include hundreds or thousands of ‘positive’ sample views of a particular object (e.g., a bicycle, car, pedestrian, etc.) and arbitrary ‘negative’ images of an object having approximately the same size. Theses classifiers 108 can be applied to a region of interest within an image to detect not only an object in question, but also to reject any regions of interest where the particular object is not found/located. For example, a CRC 108 of a bicycle can be used to detect a ROI having a feature of a bicycle (e.g., wheel, handle bar, etc.) and can also eliminate any ROI not having a feature of a bicycle (e.g., a non-object proposal, such as the sky) [the parallel set of conditional layers, conditional layers interpreted as including layers (some portion of a model) that are intended to correspond to a particular class of input].”
Choi does not explicitly teach (bold only) “wherein training according to the objective function includes adjusting parameters of the machine learning model to minimize noise in the parallel set of conditional layers while maximizing accuracy of the model.”
Lecuyer teaches (bold only) “wherein training according to the objective function includes adjusting parameters of the machine learning model to minimize noise in the parallel set of conditional layers while maximizing accuracy of the model”: Lecuyer, section III. A, paragraph 2, “Training an (ε, δ )-PixelDP network is similar to training the original network: we use the original loss and optimizer, such as stochastic gradient descent [training according to the objective function includes adjusting parameters of the machine learning model]. The major difference is that we alter the pre-noise computation to constrain its sensitivity with regards to p-norm input changes. Denote Q(x) = h(g(x)), where g is the pre-noise computation and h is the subsequent computation that produces Q(x) in the original network”; Lecuyer, section III. D, paragraph 4, “Thus far, we have described PixelDP certificates as binary with respect to a fixed attack bound, L: we either meet or do not meet a robustness check for L. In fact, our formalism allows for a more nuanced certificate, which gives the maximum attack size Lmax (measured in p-norm) against which the prediction on input x is guaranteed to be robust: no attack within this size from will be able to change the highest probability [to minimize noise in the … layers while maximizing accuracy of the model, interpreted as determining the amount of noise or alteration that should