Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office Action is in response to the amendment filed on 02/04/2026. In the instant Amendment, claims 1, 9 and 15-18 are amended; claims 1, 9 and 18 are independent claims. Claims 1-20 are pending in this application. THIS ACTION IS MADE FINAL.
Response to Arguments
Applicant’s arguments with respect to claims 1, 9 and 18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 9 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ho et al., (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), in view of Harder et al., (“Harder,” “Differentially Private Data Generation Needs Better Features,” 2022, Pages 1-17) and further in view of Abadi et al (“Abadi,” “Deep Learning with Differential Privacy,” 2016, Pages 1-14)
Regarding claim 1, Ho discloses a processor comprising:
one or more circuits to:
generate, using a neural network and based at least on receiving an indication of one or more features, an output corresponding to the one or more features, (Ho, Page 2, Background Section explains that the model builds its final output by starting from pure random noise and working backward, step by step with each step cleaning up the noise a little more based on the version that came right before it. Page 4, Sampling (Algorithm 2) gives the actual steps: start with random noise, repeatedly use the trained model to predict and remove the noise, and what’s left at the end is the generated output which is an image)
and using at least a first training data point and a second training data point, the first training data point being determined by applying noise to a training data instance with respect to a first duration of time sampled from a predetermined probabilistic distribution indicative of at least one of time or noise level and extending between a minimum value and a maximum value, and the second training data point being determined by applying noise to the training data instance with respect to a second duration of time sampled from the predetermined probabilistic distribution, (Ho, Page 2, Background Section describes that the model gradually adds noise to a clean data example over a series of steps, following a fixed pattern that controls how much noise gets added at each step. It also describes that you can jump straight to any point partway through that noise-adding process, without having to walk through every step in between; Page 4, Training (Algorithm 1), step 3 during training, for every example, the model doesn’t always add the same amount of noise. Instead, it randomly picks a point somewhere along that noise-adding process each time. Because this random pick happens over and over on the same original example, the same clean data point ends up being trained on with different amounts of noise applied at different times; Page 5, Experiments-states that the amount of noise added to each step increases steadily over the course of the process, starting from a very small amount and ending at a larger amount. This provides a low end and high end to the range the random noise duration selected from.)
and cause, using at least one of a display or an audio speaker device, presentation of the output corresponding to the one or more features generated by the neural network based at least on the one or more parameters being updated using at least the first training data point and the second training data point (Ho, Page 1 and Page 6, Figures 1, 3 and 4 and appendix Figures 11, 13 and 16-19 describe the actual generated images produced by the trained model (faces, CIFAR-10 pictures, bedroom and church photos. These figures are the disclosed act of displaying the model’s output to a viewer after training is complete)
Ho fails to explicitly disclose wherein the neural network comprises one or more parameters updated according to at least one privacy criterion; wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level;
However, in an analogous art, Harder discloses wherein the neural network comprises one or more parameters updated according to at least one privacy criterion (Harder, Page 3, Background section describes the standard way of adding calibrated noise to a piece of data so that it meets a formal privacy guarantee; Page 3, Figure 1, Step 3 shows the generator being trained by comparing it against a summary of private data that has already had this privacy-protecting noise added to it)
wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; (Harder, Page 1, Introduction describes that their overall strategy is to use public data to build a useful internal representation first, and save the private data only for a later, more limited stage which is the public/low-noise versus private/high noise; Page 3, Figure 1 describes the first step trains part of the model using public data without spending any privacy budget on it. The later steps then take the private data, summarize it, and add privacy-protecting noise to that summary before using it further; Page 3, Privatization of mean embedding describes exactly how the private data summary gets clipped and noised, while making clear this noising step is never applied to the public data)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Harder with the system/method of Ho to include wherein the neural network comprises one or more parameters updated according to at least one privacy criterion; wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level. One would have been motivated to provide a mechanism for using public data to reduce the noise burden on the private data in a generative model context (Harder, Pages 1-3 & 8).
Ho and Harder fail to explicitly disclose wherein the neural network comprises one or more parameters updated according to at least one privacy criterion, wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level;
However, in an analogous art, Abadi discloses wherein the neural network comprises one or more parameters updated according to at least one privacy criterion (Abadi, Page 3, Under Differentially Private SGD Algorithm which describes updating the model’s internal settings, its weights, step by step during training, where each update has privacy-protecting noise mixed in, and the amount of noise is tied to a target policy guarantee that the system keeps track of as training goes on called a moments accountant as described on Page 4)
wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; (Abadi, Page 2, Introduction and Page 4, Convolutional Layers separates the model into a part trained on public data without privacy noise and a part trained on private data with privacy noise added. Page 8 describes one dataset is treated as public and is used to train part of the network without privacy protection, while a second, private dataset is used to train the rest of the network with privacy-protecting noise added)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Abadi with the system/method of Ho and Harder to include wherein the neural network comprises one or more parameters updated according to at least one privacy criterion, wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level;. One would have been motivated to provide a mechanism for injecting privacy noise into gradient-based training (Abadi, Pages 2-4 & 8).
Regarding claim 2, Ho, Harder and Abadi disclose the processor of claim 1.
Ho further discloses wherein the neural network comprises a diffusion model, (Ho, Page 2, First Three Paragraph describes the neural network comprises a diffusion model)
Regarding claim 3, Ho, Harder and Abadi disclose the processor of claim 1.
Ho further discloses wherein the output comprises at least one of (Ho, Page 2, FIG 2 shows the output)
text data,
speech data,
or image data, (Ho, Page 2, FIG 2 shows the output is an image)
Regarding claim 9, Ho discloses a processor comprising:
one or more circuits to:
identify, according to at least one privacy criterion, a predetermined
probabilistic distribution indicative of at least one of time or noise level and extending between a minimum value and a maximum value; (Ho, Page 5, Experiments states that the amount of noise added to each step increases steadily over the course of training, starting from a very small value and ending at a larger one. This gives the fixed pattern or distribution over noise/time a low end and a high end)
determine a plurality of estimated outputs using a neural network and based
at least on processing a first training data point and a second training data point, (Ho, Page 4, Training, Algorithm 1, each pass through the training loop has the model produce a guess which is an estimated output for a given noised training example. Since this loop runs repeatedly across many training examples and many different random noise levels, the model produces many such guesses over the course of training i.e. a plurality of estimated outputs)
wherein the first training data point is determined by applying noise to a training data instance with respect to a first duration of time from the predetermined probabilistic distribution, and the second training data point is determined by applying noise to the training data instance with respect to a second duration of time from the predetermined probabilistic distribution, (Ho, Page 2, Background Section explains that noise is added to a clean example over a series of steps, and you can jump to any point partway through that process without walking through every step; Page 4, Training, Algorithm 1, step 3 describes during training, the model randomly picks a point along that noise-adding process each time, so the same original example ends up being trained on at different noise levels on different iterations matching the first and second duration of time drawn from the same distribution)
and update one or more parameters of the neural network based at least on (i) comparing the plurality of estimated outputs to a sample output corresponding to the training data instance, (Ho, Page 4, Training, Algorithm 1, Steps 4-5, Page 3 describes during training, the model’s guess about how much noise is added is compared directly against the actual noise that was added to the example. This comparison is what drives the update to the model’s internal settings. This is the sample output corresponding to the training data instance, the true, known noise value that was added which the model is trying to match).
Ho fails to explicitly disclose wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; and (ii) the at least one privacy criterion.
However, in an analogous art, Harder discloses wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; (Harder, Page 1, Introduction, Page 3, Figure 1 describe public data is used to build part of the model first, without spending any privacy budget; private data is used later, after being summarized and given privacy-protecting noise)
and (ii) the at least one privacy criterion, (Harder, Page 4 describes the loss used to train the generator is computed against the already-privacy-noised summary of the data rather than the raw summary so the parameter update is shaped by both the comparison and privacy protection applied beforehand)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Harder with the system/method of Ho to include wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; and (ii) the at least one privacy criterion. One would have been motivated to provide a mechanism for using public data to reduce the noise burden on the private data in a generative model context (Harder, Pages 1-3 & 8).
Ho and Harder fail to explicitly disclose wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; and (ii) the at least one privacy criterion
However, in an analogous art, Abadi discloses wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; (Abadi, Page 4, Convolutional Layers, Page 8, CIFAR experiments describes one dataset is treated as public and used to train part of the network with no privacy protection; a second, private dataset trains the rest of the network with privacy-protecting noise added)
and (ii) the at least one privacy criterion, (Abadi, Page 3, “Add Noise” step of the training algorithm describes the actual update applied to the model’s settings has privacy-protecting noise mixed directly into it, so the update depends on both the comparison and the privacy target)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Abadi with the system/method of Ho and Harder to include wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level; and (ii) the at least one privacy criterion. One would have been motivated to provide a mechanism for injecting privacy noise into gradient-based training (Abadi, Pages 2-4 & 8).
Regarding claim 18, Ho discloses a method, comprising:
generating, using a neural network and based at least on receiving an indication of one or more features, an output corresponding to the one or more features, (Ho, Page 4 “Sampling,” Algorithm 2, the trained model produces its final output by starting from random noise and removing it step by step)
and at least a first training data point and a second training data point, the first training data point being determined by applying a first amount of probabilistic noise to a training data instance and the second training data point being determined by applying a second amount of probabilistic noise to the training data instance, (Ho, Page 2, Background section describes noise is gradually added to a clean example over a series of steps; Page 4, Training, Algorithm 1, Step 3 describes the model randomly picks how far along that noise-adding process to go each time, so the same original example gets trained on a different noise amounts on different occasions which matches the first amount and second amount of noise applied to the same underlying data point)
and based at least on a combined objective value determined by combining a plurality of objective values from a plurality of comparisons of (i) a plurality of estimated outputs for the first training data point and the second training data point to (ii) the training data instance; (Ho, Page 3 describes the overall training goal is described as a running total made up of many smaller pieces, one for each random noise level the model happens to be trained on. Each smaller piece compares the model’s guess to the actual noise that was added at that particular noise level, and all of these individual comparisons get added together into a single overall training target. The model’s overall training goal is built by summing up many separate per-noise-level comparisons; Page 5, Table 2 describes this combined training goal being evaluated across different setups, reinforcing that the loss is built by aggregating many individual comparison terms rather than relying on just one)
and causing, using at least one of a display or an audio speaker device, presentation of the output, corresponding to the one or more features generated by the neural network based at least on the one or more parameters being trained using at least the first training data point and the second training data point, (Ho, Page 1 and Page 6, Figures 1, 3, 4; and appendix, Figures 11, 13, 16-19 describe the actual generated images produced and displayed after training, which is the disclosed act of presenting the model’s output)
Ho fails to explicitly disclose wherein the neural network comprises one or more parameters trained according to at least one privacy criterion; wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level.
However, in an analogous art, Harder discloses wherein the neural network comprises one or more parameters trained according to at least one privacy criterion (Harder, Page 3, Background Section, Figure 1, Step 3 describes the generator is trained by comparing it against a version of the private data summary that has already had privacy-preserving noise added)
wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level, (Harder, Page 1, Introduction, Page 3, Figure 1 describes public data is used first, without privacy protection; private data is used later, with privacy-protecting noise applied)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Harder with the system/method of Ho to include wherein the neural network comprises one or more parameters trained according to at least one privacy criterion. One would have been motivated to provide a mechanism for using public data to reduce the noise burden on the private data in a generative model context (Harder, Pages 1-3 & 8).
Ho and Harder fail to explicitly disclose wherein the neural network comprises one or more parameters trained according to at least one privacy criterion wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level.
However, in an analogous art, Abadi discloses wherein the neural network comprises one or more parameters trained according to at least one privacy criterion; (Abadi, Pages 2-4, the model’s settings are updated during training using a process that mixes in privacy-protecting noise and tracks how much privacy budget has been used)
wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level, (Abadi, Page 4, Page 8 describes one dataset is trained as public and trained without privacy protection; a second, private dataset is trained with privacy-protecting noise)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Abadi with the system/method of Ho and Harder to include wherein the neural network comprises one or more parameters trained according to at least one privacy criterion;. wherein (i) the first training data point comprises publicly available data for at least one noise level less than a threshold noise level and (ii) the second training data point comprises data having at least some privacy or access restrictions for at least one noise level greater than or equal to the threshold noise level. One would have been motivated to provide a mechanism for injecting privacy noise into gradient-based training (Abadi, Pages 2-4 & 8).
Regarding claim 19, Ho, Harder and Abadi the method of claim 18.
Ho further discloses wherein the neural network comprises a diffusion model, (Ho, Page 2, First Three Paragraph describes the neural network comprises a diffusion model)
Regarding claim 20, Ho, Harder and Abadi disclose the method of claim 18.
Ho further discloses wherein the output comprises at least one of (Ho, Page 2, FIG 2 shows the output)
text data,
speech data,
or image data, (Ho, Page 2, FIG 2 shows the output is an image)
Claims 4-5, 7, 10, 14-15 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Ho et al., (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Harder et al., (“Harder,” “Differentially Private Data Generation Needs Better Features,” 2022, Pages 1-17), in view of Abadi et al., (“Abadi,” “Deep Learning with Differential Privacy,” 2016, Pages 1-14), and further in view of Kingma et al (“Kingma,” WO 2022265992)
Regarding claim 4, Ho, Harder and Abadi disclose the processor of claim 3.
Ho, Harder and Abadi disclose fail to explicitly disclose wherein the indication comprises text instructions for incorporating the one or more features into at least one of the text data, the speech data, or the image data.
However, in an analogous art, Kingma discloses wherein the indication comprises text instructions for incorporating the one or more features into at least one of (Kingma, [0029], [0036], [0060] describes wherein the indication comprises text instructions for incorporating the one or more features into at least one of)
the text data,
the speech data,
or the image data, (Kingma, [0036], [0059] describes wherein the output comprises at least one of image data)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Harder and Abadi to include wherein the indication comprises text instructions for incorporating the one or more features into at least one of the text data, the speech data, or the image data. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 5, Ho, Harder and Abadi disclose the processor of claim 1.
Kingma further discloses wherein the neural network is updated using a gradient descent operation that modifies one or more gradient values using noise, (Kingma, [0043] describes wherein the neural network is updated using a gradient descent operation [0052] that modifies one or more gradient values [0031] using noise [0079])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Harder and Abadi to include wherein the neural network is updated using a gradient descent operation that modifies one or more gradient values using noise. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 7, Ho, Harder and Abadi disclose the processor of claim 1.
Ho, Harder and Abadi fail to explicitly disclose wherein the neural network is a denoising network, and wherein the denoising network is to generate the output by: determining an initial output according to the indication of the one or more features; modifying the initial output for a plurality of iterations up to a predetermined denoising level to determine an intermediate output; and determining the output in a single iteration according to the intermediate output.
However, in an analogous art, Kingma discloses wherein the neural network is a denoising network, (Kingma, [0043], describes wherein the neural network is a denoising network, [0079])
and wherein the denoising network is to generate the output by: (Kingma, [0079] describes and wherein the denoising network is to generate the output by)
determining an initial output according to the indication of the one or more features; (Kingma, [0006] describes determining an initial output to the indication of the one or more features, [0005])
modifying the initial output for a plurality of iterations up to a predetermined denoising level to determine an intermediate output; (Kingma, [0006] describes modifying the initial output for a plurality of iterations [0052] up to a predetermined denoising level to determine an intermediate output [0079])
and determining the output in a single iteration according to the intermediate output, (Kingma, [0079] describes and determining the output in a single iteration according to the intermediate output, [0006]-[0008])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Harder and Abadi to include wherein the neural network is a denoising network, and wherein the denoising network is to generate the output by: determining an initial output according to the indication of the one or more features; modifying the initial output for a plurality of iterations up to a predetermined denoising level to determine an intermediate output; and determining the output in a single iteration according to the intermediate output. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 10, Ho, Harder and Abadi disclose the processor of claim 9.
Ho, Harder and Abadi fail to explicitly disclose wherein the one or more circuits are to update the one or more parameters using a gradient descent operation that modifies gradient values using noise.
However, in an analogous art, Kingma discloses wherein the one or more circuits are to update the one or more parameters using a gradient descent operation that modifies gradient values using noise, (Kingma describes [0056] wherein the one or more circuits are to update the one or more parameters using a gradient descent operation [0052] that modifies gradient values [0031] using noise [0079])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Harder and Abadi to include wherein the one or more circuits are to update the one or more parameters using a gradient descent operation that modifies gradient values using noise. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 14, Ho, Harder and Abadi disclose the processor of claim 9.
Ho, Harder and Abadi fail to explicitly disclose wherein the neural network comprises a diffusion model.
However, in an analogous art, Kingma discloses wherein the neural network comprises a diffusion model, (Kingma, [0043] describes wherein the neural network comprises a diffusion model).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Harder and Abadi to include wherein the neural network comprises a diffusion model. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 15, Ho, Harder and Abadi disclose the processor of claim 9.
Ho further discloses wherein the one or more circuits are to select the first duration of time and the second duration of time according to the predetermined probabilistic distribution indicative of at least one of time or noise level, (Ho, Page 4, Algorithm 1, Training, Line 3, t-Uniform ({1,…,T}) is selecting a duration of time (t) according to the predetermined probabilistic distribution which is the uniform distribution of {1,…T}. The same random draw happens independently on each training iteration, it accounts for both the first duration and second duration of time)
Regarding claim 17, Ho, Harder and Abadi disclose the processor of claim 15.
Ho further discloses wherein the predetermined probabilistic distribution extends between a minimum value that is greater than zero and a maximum value, (Ho, Page 5, Section 4, Experiments-We set the forward process variance to constants increasing linearly from Beta1-10^-4 to B_T-0.02. Beta sub 1 -10^04 is a minimum value strictly greater than zero, and Beta_T=0.02 is the maximum value, [a minimum value that is greater than zero and a maximum value).
Claims 6 and 11 rejected under 35 U.S.C. 103 as being unpatentable over Ho et al., (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Harder et al., (“Harder,” “Differentially Private Data Generation Needs Better Features,” 2022, Pages 1-17), in view of Abadi et al., (“Abadi,” “Deep Learning with Differential Privacy,” 2016, Pages 1-14), and further in view of Xiao et al (“Xiao,” US 20220248179).
Regarding claim 6, Ho, Harder and Abadi disclose the processor of claim 1.
Ho, Harder and Abadi fail to explicitly disclose wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the neural network.
However, in an analogous art, Xiao discloses wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the network, (Xiao, [0026], [0033]-[0036], wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the neural network)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Xiao with the system/method of Ho, Harder and Abadi to include wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the neural network. One would have been motivated to construct an artificial neural network based on a criterion (Xiao, [0026]-[0027]).
Regarding claim 11, Ho, Harder and Abadi disclose the processor of claim 9.
Ho, Harder and Abadi fail to explicitly disclose wherein the at least one privacy criterion corresponds to a restriction on iterations of updating the neural network.
However, in an analogous art, Xiao discloses wherein the at least one privacy criterion corresponds to a restriction on iterations of updating the neural network, (Xiao, [0033]-[0036], wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the network)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Xiao with the system/method of Ho, Harder and Abadi to include wherein the at least one privacy criterion corresponds to a restriction on iterations of updating the neural network. One would have been motivated to construct an artificial neural network based on a criterion (Xiao, [0026]-[0027]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Ho et al., (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Harder et al., (“Harder,” “Differentially Private Data Generation Needs Better Features,” 2022, Pages 1-17), in view of Abadi et al., (“Abadi,” “Deep Learning with Differential Privacy,” 2016, Pages 1-14), and further in view of Tanski et al (“Tanski,” US 20230376833).
Regarding claim 8, Ho, Harder and Abadi disclose the processor of claim 1.
Ho, Harder and Abadi disclose fail to explicitly disclose wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
However, in an analogous art, Tanski discloses wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center;
or a system implemented at least partially using cloud computing resources, (Tanski, [0144], [0034], [0037] describes a system implemented at least partially using cloud computing resources)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Tanski with the system/method of Ho, Harder and Abadi disclose to include wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. One would have been motivated to automatically identify risk control features (Tanski, [0003]).
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Ho et al., (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Harder et al., (“Harder,” “Differentially Private Data Generation Needs Better Features,” 2022, Pages 1-17), in view of Abadi et al., (“Abadi,” “Deep Learning with Differential Privacy,” 2016, Pages 1-14), and further in view of Toroman et al (“Toroman,” US 20230107337).
Regarding claim 12, Ho, Harder and Abadi disclose the processor of claim 9.
Ho, Harder and Abadi fail to explicitly disclose wherein: the training data instance is a first training data instance, and a first training data set comprises the first training data instance; and the one or more circuits are further to update the neural network using a plurality of second training data instances of a second training data set separate from the first training data set.
However, in an analogous art, Toroman discloses wherein: the training data instance is a first training data instance, (Toroman, [0040] describes wherein: the training data instance is a first training data instance [0040], and a first training data set [0036], [0082] comprises the first training data instance [0040])
and a first training data set comprises the first training data instance; and the one or more circuits are further to update the neural network using a plurality of second training data instances of a second training data set separate from the first training data set (Toroman, [0040] describes and the one or more circuits [0131] are further to update the neural network [0035], [0092] using a plurality of second training data instances [0040] of a second training data set [0036], [0082] separate from the first training data set [0040]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Toroman with the system/method of Ho, Harder and Abadi to include wherein: the training data instance is a first training data instance, and a first training data set comprises the first training data instance; and the one or more circuits are further to update the neural network using a plurality of second training data instances of a second training data set separate from the first training data set. One would have been motivated to encode multi-scale time series data to manage machine operations (Toroman, [0002]).
Regarding claim 13, Ho, Harder and Abadi disclose the processor of claim 9.
Ho, Harder and Abadi fail to explicitly disclose wherein the one or more circuits are to: apply an autoencoder to provide the training data instance in a latent data space; and provide the training data instance from the latent data space to the neural network.
However, in an analogous art, Toroman discloses wherein the one or more circuits are to: apply an autoencoder to provide the training data instance in a latent data space; (Toroman, [0017], describes wherein the one or more circuits [0131] are to: apply an autoencoder [0017] to provide the training data instance [0040] in a latent data space [0036]).
and provide the training data instance from the latent data space to the neural network, (Toroman, [0040], describes and provide the training data instance [0040] from the latent data space [0036] to the neural network [0019])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Toroman with the system/method of Ho, Harder and Abadi to include wherein the one or more circuits are to: apply an autoencoder to provide the training data instance in a latent data space; and provide the training data instance from the latent data space to the neural network. One would have been motivated to encode multi-scale time series data to manage machine operations (Toroman, [0002]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Ho et al., (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Harder et al., (“Harder,” “Differentially Private Data Generation Needs Better Features,” 2022, Pages 1-17), in view of Abadi et al., (“Abadi,” “Deep Learning with Differential Privacy,” 2016, Pages 1-14), in view of Kingma et al., (“Kingma,” WO 2022265992)., and further in view of Yu et al., (“Yu,” “Differentially Private Model Publishing for Deep Learning,” IEEE, 2019, Pages 332-349).
Regarding claim 16, Ho, Harder, Abadi and Kingma disclose the processor of claim 15.
Ho, Harder, Abadi and Kingma fail to explicitly disclose wherein the one or more circuits are to identify the predetermined probabilistic distribution from a plurality of distributions according to the at least one privacy criterion.
However, in an analogous art, Yu discloses wherein the one or more circuits are to identify the predetermined probabilistic distribution from a plurality of distributions according to the at least one privacy criterion, (Yu, pages 342-349, Section IV.A.2, describes a plurality of distributions with 4 pre-defined decay schedules and 1 validation based adaptive schedule; Pages 342-349, IV.A.3 “Privacy Preserving Parameter Selection” describes the privacy-criterion driven selection mechanism which is a DP-compliant candidate selection via the exponential mechanism; Page 342-349, Algorithm 1, Line 5, the schedule identification step taking the privacy budget as an explicit input)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Mireshghallah with the system/method of Ho, Harder, Abadi and Kingma to include wherein the one or more circuits are to identify the predetermined probabilistic distribution from a plurality of distributions according to the at least one privacy criterion. One would have been motivated to provide a schedule selection mechanism for privacy-preserving selection among candidate hyperparameters (Yu, Page 341, III, Overview & Introduction; page 344, V.B. Evaluating Dynamic Privacy Budget Allocation/Table II).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES J WILCOX whose telephone number is (571)270-3774. The examiner can normally be reached M-F: 8 A.M. to 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Luu T. Pham can be reached at (571)270-5002. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES J WILCOX/ Examiner, Art Unit 2439
/LUU T PHAM/ Supervisory Patent Examiner, Art Unit 2439
Prosecution Insights