DETAILED ACTION
This action is in response to the Applicant Response filed 11 March 2026 for application 18/150,147 filed 04 January 2023.
Claim(s) 1, 8, 15 is/are currently amended.
Claim(s) 1-20 is/are pending.
Claim(s) 1-20 is/are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments regarding the 35 U.S.C. 101 rejection of the claims are based on the newly amended subject matter. All arguments are addressed in the 35 U.S.C. 101 rejection of the claims below.
Applicant’s arguments regarding the 35 U.S.C. 103 rejections of claims 1-20 have been fully considered but are not persuasive. Applicant argues that the cited references do not teach the limitations as recited in claim 1. Specifically, applicant argues that the references do not teach computing a DP score or selecting weighting parameters based on the DP score. Examiner respectfully disagrees. Cheng teaches calculating a reward [DP score] for the architectures in the search space and selecting the optimal architecture [highest reward (DP score)] based on the score (Cheng, section 3). The architectures define subnetworks which are defined by weight values determined through training data (Cheng, section 3). Additionally, Cheng teaches nodes lacking a connection, meaning no weighting parameters (Cheng, section 3). Further, Luo teaches masking weights under a threshold [lowest] (Luo, section 4.4). Therefore, the cited references do, in fact, teaches the recites limitations of claim 1. Therefore, claims 1-20 stand rejected under 35 U.S.C. 103.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101, because the claim(s) is/are directed to an abstract idea, and because the claim elements, whether considered individually or in combination, do not amount to significantly more than the abstract idea, see Alice Corporation Pty. Ltd. V. CLS Bank International et al., 573 US 208 (2014).
Regarding claim 1, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method.
The limitation of identifying a differentially-private subnetwork of a neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of computing respective differentially-private score values for individual ones of a plurality of weighting parameters of the neural network according to the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of selecting a portion of the plurality of weighting parameters having highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – computer-implemented. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – differentially-private subnetwork, neural network. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites training the neural network according to training data for the differentially-private subnetwork which is simply generic training to perform the abstract idea of network identification and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
computer-implemented amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
generic training to perform the abstract idea amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
differentially-private subnetwork, neural network amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 2, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method.
The limitation of randomly initializing the plurality of weighting parameters of the neural network with differing values prior to computing the respective scores for the individual ones of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 3, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 2 is applicable here since claim 3 carries out the method of claim 2 but for the recitation of additional element(s) of wherein the plurality of weighting parameters of the neural network are randomly initialized according to a normal distribution.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the plurality of weighting parameters of the neural network are randomly initialized according to a normal distribution which is simply additional information regarding the weighting parameters, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – normal distribution. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
normal distribution amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the weighting parameters do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 4, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method.
The limitation of wherein respective weighting parameters of the neural network excluded from the selected portion are set to a zero value in the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 5, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method.
The limitation of wherein computing respective differentially-private score values comprises executing one or more iterations of a scoring computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of computing respective score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of adding respective noise values to the respective computed score values to generate respective differentially-private score values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses adding values.
The limitation of selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of updating the respective computed score values according to the identified differentially-private subnetwork using a stochastic gradient descent technique, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses performing backpropagation.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – stochastic gradient descent technique. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
stochastic gradient descent technique amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 6, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method.
The limitation of wherein computing respective differentially-private score values comprises executing one or more iterations of a scoring computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of computing the respective differentially-private score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of updating the respective computed score values according to the identified differentially-private subnetwork using a differentially-private stochastic gradient descent technique, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses performing backpropagation.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – differentially-private stochastic gradient descent technique. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
stochastic gradient descent technique amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 7, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-implemented method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 7 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the portion of the plurality of weighting parameters comprises a variable number of weighting parameters, the variable number determined according to a minimum accuracy threshold.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the weighting parameters and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the weighting parameters do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 8, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to one or more computer-accessible storage media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites one or more computer-accessible storage media.
The limitation of identifying a differentially-private subnetwork of a neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of computing respective differentially-private score values for individual ones of a plurality of weighting parameters of the neural network according to the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of selecting a portion of the plurality of weighting parameters having highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – one or more computer-accessible storage media, program instructions, one or more computing devices. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – differentially-private subnetwork, neural network. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites training the neural network according to training data for the differentially- private subnetwork which is simply generic training to perform the abstract idea of network identification and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
one or more computer-accessible storage media, program instructions, one or more computing devices amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
generic training to perform the abstract idea amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
differentially-private subnetwork, neural network amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 9, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to one or more computer-accessible storage media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites one or more computer-accessible storage media.
The limitation of randomly initializing the plurality of weighting parameters of the neural network with differing values prior to computing the respective scores for the individual ones of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 10, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to one or more computer-accessible storage media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites one or more computer-accessible storage media. The Step 2A Prong One Analysis for claim 8 is applicable here since claim 10 carries out the computer-accessible storage media of claim 8 but for the recitation of additional element(s) of wherein the plurality of weighting parameters of the neural network are randomly initialized according to a kaiming normal distribution.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the plurality of weighting parameters of the neural network are randomly initialized according to a kaiming normal distribution which is simply additional information regarding the weighting parameters, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – kaiming normal distribution. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
kaiming normal distribution amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the weighting parameters do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 11, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to one or more computer-accessible storage media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites one or more computer-accessible storage media.
The limitation of wherein respective weighting parameters of the neural network excluded from the selected portion are set to a zero value in the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 12, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to one or more computer-accessible storage media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites one or more computer-accessible storage media.
The limitation of wherein computing respective differentially-private score values comprises executing one or more iterations of a scoring computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of computing respective score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of adding respective noise values to the respective computed score values to generate respective differentially-private score values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses adding values.
The limitation of selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of updating the respective computed score values according to the identified differentially-private subnetwork using a stochastic gradient descent technique, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses performing backpropagation.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – stochastic gradient descent technique. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
stochastic gradient descent technique amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 13, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to one or more computer-accessible storage media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites one or more computer-accessible storage media.
The limitation of wherein computing respective differentially-private score values comprises executing one or more iterations of a scoring computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of computing the respective differentially-private score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of updating the respective computed score values according to the identified differentially-private subnetwork using a differentially-private stochastic gradient descent technique, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses performing backpropagation.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – differentially-private stochastic gradient descent technique. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
stochastic gradient descent technique amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 14, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to one or more computer-accessible storage media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites one or more computer-accessible storage media. The Step 2A Prong One Analysis for claim 8 is applicable here since claim 14 carries out the computer-accessible storage media of claim 8 but for the recitation of additional element(s) of wherein the portion of the plurality of weighting parameters comprises a variable number of weighting parameters, the variable number determined according to a minimum accuracy threshold.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the weighting parameters and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the weighting parameters do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 15, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 15 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of identify a differentially-private subnetwork of a neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of compute respective differentially-private score values for individual ones of a plurality of weighting parameters of the neural network according to the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of select a portion of the plurality of weighting parameters having highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – system, one or more processors, memory, program instructions, differentially-private machine learning system. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – differentially-private subnetwork, neural network. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites train the neural network according to training data for the differentially- private subnetwork which is simply generic training to perform the abstract idea of network identification and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
system, one or more processors, memory, program instructions, differentially-private machine learning system amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
generic training to perform the abstract idea amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
differentially-private subnetwork, neural network amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 16, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 16 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of randomly initialize the plurality of weighting parameters of the neural network with differing values prior to computing the respective scores for the individual ones of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 17, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 17 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system. The Step 2A Prong One Analysis for claim 15 is applicable here since claim 17 carries out the system of claim 15 but for the recitation of additional element(s) of wherein the plurality of weighting parameters of the neural network are randomly initialized according to a xavier normal distribution.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the plurality of weighting parameters of the neural network are randomly initialized according to a xavier normal distribution which is simply additional information regarding the weighting parameters, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – xavier normal distribution. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
xavier normal distribution amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the weighting parameters do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 18, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 18 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of wherein respective weighting parameters of the neural network excluded from the selected portion are set to a zero value in the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 19, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 19 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of wherein to compute respective differentially-private score values the differentially-private machine learning system is configured to execute one or more iterations of a scoring computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of compute respective score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of add respective noise values to the respective computed score values to generate respective differentially-private score values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses adding values.
The limitation of select the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of update the respective computed score values according to the identified differentially-private subnetwork using a stochastic gradient descent technique, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses performing backpropagation.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – stochastic gradient descent technique. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
stochastic gradient descent technique amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 20, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 20 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of wherein to compute respective differentially-private score values the differentially-private machine learning system is configured to execute one or more iterations of a scoring computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of compute the respective differentially-private score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating scores.
The limitation of select the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of update the respective computed score values according to the identified differentially-private subnetwork using a differentially-private stochastic gradient descent technique, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses performing backpropagation.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – differentially-private stochastic gradient descent technique. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
stochastic gradient descent technique amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4-8, 11-15, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy, hereinafter referred to as “Cheng”) in view of Luo et al (Scalable Differential Privacy with Sparse Network Finetuning, hereinafter referred to as “Luo”).
Regarding claim 1 (Currently Amended), Cheng teaches a computer-implemented (Cheng, section 4.1 – teaches computer processor and memory) method, comprising:
identifying a differentially-private subnetwork of a neural network (Cheng, sections 3-3.4 – teaches identifying a DP subnetwork of a neural network), comprising:
training the neural network according to training data for the differentially-private subnetwork (Cheng, sections 3-3.4 – teaches training the network);
computing respective differentially-private score values for individual ones of a plurality of weighting parameters of the neural network according to training data for the differentially-private subnetwork (Cheng, sections 3-3.4 – teaches computing rewards and weights according to training data for a given subnetwork; see also Cheng, Algorithm 1); and
selecting a portion of the plurality of weighting parameters … to identify the differentially-private subnetwork, wherein the selected portion excludes at least one … weighting parameter of the plurality of weighting parameters (Cheng, sections 3-3.4 – teaches selecting a subnetwork through neural architecture search where some of the nodes lack a connection).
While, Cheng teaches a neural architecture search to identify a subnetwork, Cheng does not explicitly teach selecting a portion of the plurality of weighting parameters having highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters.
Luo teaches selecting a portion of the plurality of weighting parameters having highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters (Luo, section 4.4 – teaches selecting weights having magnitudes over a threshold while masking weights under the threshold to identify a subnetwork).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Cheng with the teachings of Luo in order to minimize trainable parameters in order to improve privacy performance of differential privacy in the field of differentially private architecture search (Luo, Abstract – “We propose a novel method for privacy-preserving training of deep neural networks leveraging public, out-domain data. While differential privacy (DP) has emerged as a mechanism to protect sensitive data in training datasets, its application to complex visual recognition tasks remains challenging. Traditional DP methods, such as Differentially-Private Stochastic Gradient Descent (DPSGD), perform well only on simple datasets and shallow networks, while recent transfer learning-based DP methods often make unrealistic assumptions about the availability and distribution of public data. In this work, we argue that minimizing the number of trainable parameters is the key to improving the privacy-performance tradeoff of DP on complex visual recognition tasks. Inspired by this argument, we also propose a novel transfer learning paradigm that finetunes a very sparse subnetwork with DP. We conduct extensive experiments and ablation studies on two visual recognition tasks: CIFAR-100 → CIFAR-10 (standard DP setting) and the CD-FSL challenge (few-shot, multiple levels of domain shifts) and demonstrate competitive experimental performance.”).
Regarding claim 4 (Original), Cheng in view of Luo teaches all of the limitations of the method of claim 1 as noted above. Cheng further teaches wherein respective weighting parameters of the neural network excluded from the selected portion are set to a zero value in the differentially-private subnetwork (Cheng, section 3.2 – teaches zero operation to indicate a lack of connection between nodes).
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Cheng and Luo for the same reasons as disclosed in claim 1 above.
Regarding claim 5 (Original), Cheng in view of Luo teaches all of the limitations of the method of claim 1 as noted above. Cheng further teaches wherein computing respective differentially-private score values comprises executing one or more iterations of a scoring computation (Cheng, sections 3-3.4 – teaches iteratively computing weights for a given subnetwork; see also Cheng, Algorithm 1), and wherein an iteration of the one or more iterations comprises:
computing respective score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork (Cheng, sections 3-3.4 – teaches computing weights according to training data for a given subnetwork; see also Cheng, Algorithm 1);
adding respective noise values to the respective computed score values to generate respective differentially-private score values (Cheng, section 3.2 – teaches adding Gaussian noise to the computer weights);
selecting the portion of the plurality of weighting parameters … to identify the differentially-private subnetwork, wherein the selected portion excludes at least one … weighting parameter of the plurality of weighting parameters (Cheng, sections 3-3.4 – teaches selecting a subnetwork through neural architecture search where some of the nodes lack a connection); and
updating the respective computed score values according to the identified differentially-private subnetwork using a stochastic gradient descent technique (Cheng, sections 3-3.4 – teaches updating the weights by DPSGD; see also Cheng, Algorithm 1).
While, Cheng teaches a neural architecture search to identify a subnetwork, Cheng does not explicitly teach selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters.
Luo teaches selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters (Luo, section 4.4 – teaches selecting weights having magnitudes over a threshold while masking weights under the threshold to identify a subnetwork).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Cheng and Luo in order to identify the subnetwork to minimize trainable parameters in order to improve privacy performance of differential privacy (Luo, Abstract).
Regarding claim 6 (Original), Cheng in view of Luo teaches all of the limitations of the method of claim 1 as noted above. Cheng further teaches wherein computing respective differentially-private score values comprises executing one or more iterations of a scoring computation (Cheng, sections 3-3.4 – teaches iteratively computing weights for a given subnetwork; see also Cheng, Algorithm 1), wherein an iteration of the one or more iterations comprises:
computing the respective differentially-private score values for the individual ones of the plurality of weighting parameters of the neural network according to samples of the training data for the differentially-private subnetwork (Cheng, sections 3-3.4 – teaches computing weights according to training data for a given subnetwork; see also Cheng, Algorithm 1);
selecting the portion of the plurality of weighting parameters … to identify the differentially-private subnetwork, wherein the selected portion excludes at least one … weighting parameter of the plurality of weighting parameters (Cheng, sections 3-3.4 – teaches selecting a subnetwork through neural architecture search where some of the nodes lack a connection); and
updating the respective computed score values according to the identified differentially-private subnetwork using a differentially-private stochastic gradient descent technique (Cheng, sections 3-3.4 – teaches updating the weights by DPSGD; see also Cheng, Algorithm 1).
While, Cheng teaches a neural architecture search to identify a subnetwork, Cheng does not explicitly teach selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters.
Luo teaches selecting the portion of the plurality of weighting parameters having the highest respective differentially-private score values to identify the differentially-private subnetwork, wherein the selected portion excludes at least one lowest scored weighting parameter of the plurality of weighting parameters (Luo, section 4.4 – teaches selecting weights having magnitudes over a threshold while masking weights under the threshold to identify a subnetwork).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Cheng and Luo in order to identify the subnetwork to minimize trainable parameters in order to improve privacy performance of differential privacy (Luo, Abstract).
Regarding claim 7 (Original), Cheng in view of Luo teaches all of the limitations of the method of claim 1 as noted above. Cheng further teaches wherein the portion of the plurality of weighting parameters comprises a variable number of weighting parameters, the variable number determined according to a minimum accuracy threshold (Cheng, section 2 – teaches selecting the optimized model based on a performance estimation [accuracy]).
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Cheng and Luo for the same reasons as disclosed in claim 1 above.
Regarding claim 8 (Currently Amended), it is the computer-accessible storage media embodiment of claim 1 with similar limitations to claim 1 and is rejected using the same reasoning found in claim 1. Cheng further teaches one or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement (Cheng, section 4.1 – teaches computer processor and memory to perform operations) …
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Cheng and Luo for the same reasons as disclosed in claim 1 above.
Regarding claim 11 (Original), the rejection of claim 8 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo for the reasons set forth in the rejection of claim 4.
Regarding claim 12 (Original), the rejection of claim 8 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo for the reasons set forth in the rejection of claim 5.
Regarding claim 13 (Original), the rejection of claim 8 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo for the reasons set forth in the rejection of claim 6.
Regarding claim 14 (Original), the rejection of claim 8 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo for the reasons set forth in the rejection of claim 7.
Regarding claim 15 (Currently Amended), it is the system embodiment of claim 1 with similar limitations to claim 1 and is rejected using the same reasoning found in claim 1. Cheng further teaches a system, comprising:
one or more processors (Cheng, section 4.1 – teaches computer processor and memory to perform operations); and
a memory storing program instructions that when executed by the one or more processors cause the one or more processors (Cheng, section 4.1 – teaches computer processor and memory to perform operations) to implement a differentially-private machine learning system (Cheng, section 3 – teaches differentially private neural architecture search) …
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Cheng and Luo for the same reasons as disclosed in claim 1 above.
Regarding claim 18 (Original), the rejection of claim 15 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo for the reasons set forth in the rejection of claim 4.
Regarding claim 19 (Original), the rejection of claim 15 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo for the reasons set forth in the rejection of claim 5.
Regarding claim 20 (Original), the rejection of claim 15 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo for the reasons set forth in the rejection of claim 6.
Claim(s) 2, 9, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of Luo and further in view of Abadi et al. (Deep Learning with Differential Privacy, hereinafter referred to as “Abadi”).
Regarding claim 2 (Original), Cheng in view of Luo teaches all of the limitations of the method of claim 1 as noted above. However, Cheng in view of Luo does not explicitly teach randomly initializing the plurality of weighting parameters of the neural network with differing values prior to computing the respective scores for the individual ones of the plurality of weighting parameters.
Abadi teaches randomly initializing the plurality of weighting parameters of the neural network with differing values prior to computing the respective scores for the individual ones of the plurality of weighting parameters (Abadi, Algorithm 1 – teaches randomly initializing weights for DPSGD).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Cheng in view of Luo with the teachings of Abadi in order to efficiently train models while protecting privacy in the field of differentially private architecture search (Abadi, Abstract – “Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains. Often, the training of models requires large, representative datasets, which may be crowdsourced and contain sensitive information. The models should not expose private information in these datasets. Addressing this goal, we develop new algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy. Our implementation and experiments demonstrate that we can train deep neural networks with non-convex objectives, under a modest privacy budget, and at a manageable cost in software complexity, training efficiency, and model quality.”).
Regarding claim 9 (Original), the rejection of claim 8 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo and further in view of Abadi for the reasons set forth in the rejection of claim 2.
Regarding claim 16 (Original), the rejection of claim 15 is incorporated herein. Further, the limitations in this claim are taught by Cheng in view of Luo and further in view of Abadi for the reasons set forth in the rejection of claim 2.
Claim(s) 3, 10, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of Luo, further in view of Abadi and further in view of Pan et al. (A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks, hereinafter referred to as “Pan”).
Regarding claim 3 (Original), Cheng in view of Luo and further in view of Abadi teaches all of the limitations of the method of claim 2 as noted above. However, Cheng in view of Luo and further in view of Abadi does not explicitly teach wherein the plurality of weighting parameters of the neural network are randomly initialized according to a normal distribution.
Pan teaches wherein the plurality of weighting parameters of the neural network are randomly initialized according to a normal distribution (Pan, section 2.3, teaches initializing weights using kaiming or xavier normal distributions).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Cheng in view of Luo and further in view of Abadi with the teachings of Pan in order to provide a widely applicable universal weight initialization in the field of differentially private architecture search (Pan, Abstract – “Tensorial Convolutional Neural Networks (TCNNs) have attracted much research attention for their power in reducing model parameters or enhancing the generalization ability. However, exploration of TCNNs is hindered even from weight initialization methods. To be specific, general initialization methods, such as Xavier or Kaiming initialization, usually fail to generate appropriate weights for TCNNs. Meanwhile, although there are ad-hoc approaches for specific architectures (e.g., Tensor Ring Nets), they are not applicable to TCNNs with other tensor decomposition methods (e.g., CP or Tucker decomposition). To address this problem, we propose a universal weight initialization paradigm, which generalizes Xavier and Kaiming methods and can be widely applicable to arbitrary TCNNs. Specifically, we first present the Reproducing Transformation to convert the backward process in TCNNs to an equivalent convolution process. Then, based on the convolution operators in the forward and backward processes, we build a unified paradigm to control the variance of features and gradients in TCNNs. Thus, we can derive fan-in and fan-out initialization for various TCNNs. We demonstrate that our paradigm can stabilize the training of TCNNs, leading to faster convergence and better results.”).
Regarding claim 10 (Original), Cheng in view of Luo and further in view of Abadi teaches all of the limitations of the method of claim 8 as noted above. However, Cheng in view of Luo and further in view of Abadi does not explicitly teach wherein the plurality of weighting parameters of the neural network are randomly initialized according to a kaiming normal distribution.
Pan teaches wherein the plurality of weighting parameters of the neural network are randomly initialized according to a kaiming normal distribution (Pan, section 2.3, teaches initializing weights using kaiming or xavier normal distributions).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Cheng in view of Luo and further in view of Abadi with the teachings of Pan in order to provide a widely applicable universal weight initialization in the field of differentially private architecture search (Pan, Abstract – “Tensorial Convolutional Neural Networks (TCNNs) have attracted much research attention for their power in reducing model parameters or enhancing the generalization ability. However, exploration of TCNNs is hindered even from weight initialization methods. To be specific, general initialization methods, such as Xavier or Kaiming initialization, usually fail to generate appropriate weights for TCNNs. Meanwhile, although there are ad-hoc approaches for specific architectures (e.g., Tensor Ring Nets), they are not applicable to TCNNs with other tensor decomposition methods (e.g., CP or Tucker decomposition). To address this problem, we propose a universal weight initialization paradigm, which generalizes Xavier and Kaiming methods and can be widely applicable to arbitrary TCNNs. Specifically, we first present the Reproducing Transformation to convert the backward process in TCNNs to an equivalent convolution process. Then, based on the convolution operators in the forward and backward processes, we build a unified paradigm to control the variance of features and gradients in TCNNs. Thus, we can derive fan-in and fan-out initialization for various TCNNs. We demonstrate that our paradigm can stabilize the training of TCNNs, leading to faster convergence and better results.”).
Regarding claim 17 (Original), Cheng in view of Luo and further in view of Abadi teaches all of the limitations of the method of claim 15 as noted above. However, Cheng in view of Luo and further in view of Abadi does not explicitly teach wherein the plurality of weighting parameters of the neural network are randomly initialized according to a xavier normal distribution.
Pan teaches wherein the plurality of weighting parameters of the neural network are randomly initialized according to a xavier normal distribution (Pan, section 2.3, teaches initializing weights using kaiming or xavier normal distributions).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Cheng in view of Luo and further in view of Abadi with the teachings of Pan in order to provide a widely applicable universal weight initialization in the field of differentially private architecture search (Pan, Abstract – “Tensorial Convolutional Neural Networks (TCNNs) have attracted much research attention for their power in reducing model parameters or enhancing the generalization ability. However, exploration of TCNNs is hindered even from weight initialization methods. To be specific, general initialization methods, such as Xavier or Kaiming initialization, usually fail to generate appropriate weights for TCNNs. Meanwhile, although there are ad-hoc approaches for specific architectures (e.g., Tensor Ring Nets), they are not applicable to TCNNs with other tensor decomposition methods (e.g., CP or Tucker decomposition). To address this problem, we propose a universal weight initialization paradigm, which generalizes Xavier and Kaiming methods and can be widely applicable to arbitrary TCNNs. Specifically, we first present the Reproducing Transformation to convert the backward process in TCNNs to an equivalent convolution process. Then, based on the convolution operators in the forward and backward processes, we build a unified paradigm to control the variance of features and gradients in TCNNs. Thus, we can derive fan-in and fan-out initialization for various TCNNs. We demonstrate that our paradigm can stabilize the training of TCNNs, leading to faster convergence and better results.”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to MARSHALL WERNER whose telephone number is (469) 295-9143. The examiner can normally be reached on Monday – Thursday 7:30 AM – 4:30 PM ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARSHALL L WERNER/ Primary Examiner, Art Unit 2125
Prosecution Insights