Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office Action is in response to the amendment filed on 02/04/2026. In the instant Amendment, claims 1, 9 and 18 are amended; claims 1, 9 and 18 are independent claims. Claims 1-20 are pending in this application.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/04/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claims 1, 9 and 18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Ho et al (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25) in view of Song et al (“Song,” “Stochastic Gradient Descent with Differentially Private Updates,” IEEE, 2013, Pages 245-248) and further in view of Papernot et al (“Papernot,” “Semi-Supervised Knowledge Transfer for Deep Learning From Private Training Data,” 2017, Pages 1-16).
Regarding claim 1, Ho discloses a processor comprising:
one or more circuits to: generate, using a neural network and based at least on receiving an indication of one or more features, an output corresponding to the one or more features, (Ho, Pages 2-8 describe the overall system, Page 5, Under Section 4 Experiments, Pages 14-15 Under Section B. Experimental Results discloses a neural network (U-Net) that receives as input: a noisy data sample a timestep/noise-level embedded. The network generates an output (predicted noise or denoised signal) corresponding to input features. Conditioning on inputs/features is described and extended to conditional generation. The neural network is trained to predict the noise added to timestep t and t is provided to the network via a timestep embedding)
the first training data point being determined by applying noise to a training data instance with respect to a first duration of time sampled from a predetermined distribution indicative of at least one of time or noise level and extending between a minimum value and a maximum value, (Ho discloses Pages 2-3, Section 2 Background we sample t uniformly from (1,…T) for each training example. The forward process gradually adds noise over T timesteps. The first and second training data points correspond to different sampled timesteps and the duration of time/noise level reads on timestep/noise variance schedule. Uniform distribution over timesteps is described where the minimum is t=1 and the maximum is t=T. Each timestep corresponds to a specific noise variance level).
and the second training data point being determined by applying noise to the training data instance with respect to a second duration of time sampled from the predetermined distribution, (Ho discloses Pages 2-3, Section 2 Background we sample t uniformly from (1,…T) for each training example. The forward process gradually adds noise over T timesteps. The first and second training data points correspond to different sampled timesteps and the duration of time/noise level reads on timestep/noise variance schedule. Uniform distribution over timesteps is described where the minimum is t=1 and the maximum is t=T. Each timestep corresponds to a specific noise variance level).
and cause, using at least one of a display or an audio speaker device, presentation of the output, (Ho describes a display on FIG 1, FIG 2, Page 4, Next to last Paragraph)
Ho fails to explicitly disclose wherein the neural network comprises one or more parameters updated according to at least one privacy criterion and using at least a first training data point and a second training data point.
However, in an analogous art, Song discloses wherein the neural network comprises one or more parameters updated according to at least one privacy criterion and using at least a first training data point and a second training data point, (Song discloses Page 248, Left Column, Conclusion and Pages 246-247 Under Section III. SGD with Differential Privacy stochastic gradient descent algorithms with updates modified to ensure differential privacy. The amount of noise is calibrated according to a desired privacy level also see Page 247 under Section B. Procedure and Section C. Mini-batching Reduces Variance)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Song with the system/method of Ho to include wherein the neural network comprises one or more parameters updated according to at least one privacy criterion and using at least a first training data point and a second training data point. One would have been motivated to provide stochastic gradient descent with differentially private updates (Song, Page 248, Left Column, Conclusion).
Ho and Song fails to explicitly disclose wherein the first training data point comprises publicly available data and the second training data point comprises data having at least some privacy or access restrictions;
However, in an analogous art, Papernot discloses wherein the first training data point comprises publicly available data and the second training data point comprises data having at least some privacy or access restrictions; (Papernot discloses on Page 2, Second & Fourth Paragraphs, Page 3, Fig 1 the student is trained on publicly available unlabeled data and the teacher models are trained on sensitive datasets not directly accessible).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Papernot with the system/method of Ho and Song to include wherein the first training data point comprises publicly available data and the second training data point comprises data having at least some privacy or access restrictions. One would have been motivated to provide a method and system for protecting the privacy of sensitive training data (Papernot, Page 10, Section 6, Conclusion)
Regarding claim 2, Ho, Song and Papernot disclose the processor of claim 1.
Ho further discloses wherein the neural network comprises a diffusion model, (Ho, Page 2, First Three Paragraph describes the neural network comprises a diffusion model)
Regarding claim 3, Ho, Song and Papernot disclose the processor of claim 1.
Ho further discloses wherein the output comprises at least one of (Ho, Page 2, FIG 2 shows the output)
text data,
speech data,
or image data, (Ho, Page 2, FIG 2 shows the output is an image)
Regarding claim 9, Ho discloses a processor comprising:
one or more circuits to: identify, a predetermined distribution indicative of at least one of time or noise level and extending between a minimum value and a maximum value; (Ho, Pages 2-8 describe the overall system, Page 5, Under Section 4 Experiments, Pages 14-15 Under Section B. Experimental Results discloses a neural network (U-Net) that receives as input: a noisy data sample a timestep/noise-level embedded. The network generates an output (predicted noise or denoised signal) corresponding to input features. Conditioning on inputs/features is described and extended to conditional generation. The neural network is trained to predict the noise added to timestep t and t is provided to the network via a timestep embedding; Pages 2-3, Section 2 Background we sample t uniformly from (1,…T) for each training example. The forward process gradually adds noise over T timesteps. The first and second training data points correspond to different sampled timesteps and the duration of time/noise level reads on timestep/noise variance schedule. Uniform distribution over timesteps is described where the minimum is t=1 and the maximum is t=T. Each timestep corresponds to a specific noise variance level)
determine a plurality of estimated outputs using a neural network and based at least on processing a first training data point and a second training data point, wherein the first training data point is determined by applying noise to a training data instance with respect to a first duration of time from the predetermined distribution, (Ho, Pages 2-8 describe the overall system, Page 5, Under Section 4 Experiments, Pages 14-15 Under Section B. Experimental Results describes determining a plurality of estimated outputs in a neural network and based on processing a first training data point and a second training data point wherein the first training data point is determined by applying noise to a training data instance with respect to a first duration of time from the distribution that’s predetermined)
and the second training data point is determined by applying noise to the training data instance with respect to a second duration of time from the predetermined distribution, (Ho discloses Pages 2-3, Section 2 Background we sample t uniformly from (1,…T) for each training example. The forward process gradually adds noise over T timesteps. The first and second training data points correspond to different sampled timesteps and the duration of time/noise level reads on timestep/noise variance schedule. Uniform distribution over timesteps is described where the minimum is t=1 and the maximum is t=T. Each timestep corresponds to a specific noise variance level).
and update one or more parameters of the neural network based at least on (i) comparing the plurality of estimated outputs to a sample output corresponding to the training data instance, (Ho, Pages 2-8 describe the overall system, Page 5, Under Section 4 Experiments, Pages 14-15 Under Section B. Experimental Results describes training objective and comparing estimated outputs to a known sample output (true noise epsilon))
Ho fails to explicitly disclose according to at least one privacy criterion, determine a plurality of estimated outputs using a neural network and based at least on processing a first training data point and a second training data point; and update one or more parameters of the neural network based at least on and (ii) the at least one privacy criterion.
However, in an analogous art, Song discloses according to at least one privacy criterion, determine a plurality of estimated outputs using a neural network and based at least on processing a first training data point and a second training data point, (Song discloses Page 248, Left Column, Conclusion and Pages 246-247 Under Section III. SGD with Differential Privacy stochastic gradient descent algorithms with updates modified to ensure differential privacy. The amount of noise is calibrated according to a desired privacy level also see Page 247 under Section B. Procedure and Section C. Mini-batching Reduces Variance)
and update one or more parameters of the neural network based at least on and (ii) the at least one privacy criterion (Song discloses Page 248, Left Column, Conclusion and Pages 246-247 Under Section III describes parameter updates are constrained and explicitly privacy criterion)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Song with the system/method of Ho to include according to at least one privacy criterion, determine a plurality of estimated outputs using a neural network and based at least on processing a first training data point and a second training data point; and update one or more parameters of the neural network based at least on and (ii) the at least one privacy criterion. One would have been motivated to provide stochastic gradient descent with differentially private updates (Song, Page 248, Left Column, Conclusion).
Ho and Song fail to explicitly disclose wherein the first training data point comprises publicly available data and the second training data point comprises data having at least some privacy or access restrictions.
However, in an analogous art, Papernot discloses wherein the first training data point comprises publicly available data and the second training data point comprises data having at least some privacy or access restrictions, (Papernot discloses on Page 2, Second & Fourth Paragraphs, Page 3, Fig 1 the student is trained on publicly available unlabeled data and the teacher models are trained on sensitive datasets not directly accessible).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Papernot with the system/method of Ho and Song to include wherein the first training data point comprises publicly available data and the second training data point comprises data having at least some privacy or access restrictions. One would have been motivated to provide a method and system for protecting the privacy of sensitive training data (Papernot, Page 10, Section 6, Conclusion)
Claims 4-5, 7, 10, 14-15 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Ho et al (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Song et al (“Song,” “Stochastic Gradient Descent with Differentially Private Updates,” IEEE, 2013, Pages 245-248) in view of Papernot et al (“Papernot,” “Semi-Supervised Knowledge Transfer for Deep Learning From Private Training Data,” 2017, Pages 1-16) and further in view of Kingma et al (“Kingma,” WO 2022265992)
Regarding claim 4, Ho, Song and Papernot disclose the processor of claim 3.
Ho, Song and Papernot fail to explicitly disclose wherein the indication comprises text instructions for incorporating the one or more features into at least one of the text data, the speech data, or the image data.
However, in an analogous art, Kingma discloses wherein the indication comprises text instructions for incorporating the one or more features into at least one of (Kingma, [0029], [0036], [0060] describes wherein the indication comprises text instructions for incorporating the one or more features into at least one of)
the text data,
the speech data,
or the image data, (Kingma, [0036], [0059] describes wherein the output comprises at least one of image data)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Song and Papernot to include wherein the indication comprises text instructions for incorporating the one or more features into at least one of the text data, the speech data, or the image data. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 5, Ho, Song and Papernot disclose the processor of claim 1.
Kingma further discloses wherein the neural network is updated using a gradient descent operation that modifies one or more gradient values using noise, (Kingma, [0043] describes wherein the neural network is updated using a gradient descent operation [0052] that modifies one or more gradient values [0031] using noise [0079])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Song and Papernot to include wherein the neural network is updated using a gradient descent operation that modifies one or more gradient values using noise. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 7, Ho, Song and Papernot disclose the processor of claim 1.
Ho, Song and Papernot fail to explicitly disclose wherein the neural network is a denoising network, and wherein the denoising network is to generate the output by: determining an initial output according to the indication of the one or more features; modifying the initial output for a plurality of iterations up to a predetermined denoising level to determine an intermediate output; and determining the output in a single iteration according to the intermediate output.
However, in an analogous art, Kingma discloses wherein the neural network is a denoising network, (Kingma, [0043], describes wherein the neural network is a denoising network, [0079])
and wherein the denoising network is to generate the output by: (Kingma, [0079] describes and wherein the denoising network is to generate the output by)
determining an initial output according to the indication of the one or more features; (Kingma, [0006] describes determining an initial output to the indication of the one or more features, [0005])
modifying the initial output for a plurality of iterations up to a predetermined denoising level to determine an intermediate output; (Kingma, [0006] describes modifying the initial output for a plurality of iterations [0052] up to a predetermined denoising level to determine an intermediate output [0079])
and determining the output in a single iteration according to the intermediate output, (Kingma, [0079] describes and determining the output in a single iteration according to the intermediate output, [0006]-[0008])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Song and Papernot to include wherein the neural network is a denoising network, and wherein the denoising network is to generate the output by: determining an initial output according to the indication of the one or more features; modifying the initial output for a plurality of iterations up to a predetermined denoising level to determine an intermediate output; and determining the output in a single iteration according to the intermediate output. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 10, Ho, Song and Papernot disclose the processor of claim 9.
Ho, Song and Papernot fail to explicitly disclose wherein the one or more circuits are to update the one or more parameters using a gradient descent operation that modifies gradient values using noise.
However, in an analogous art, Kingma discloses wherein the one or more circuits are to update the one or more parameters using a gradient descent operation that modifies gradient values using noise, (Kingma describes [0056] wherein the one or more circuits are to update the one or more parameters using a gradient descent operation [0052] that modifies gradient values [0031] using noise [0079])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Song and Papernot to include wherein the one or more circuits are to update the one or more parameters using a gradient descent operation that modifies gradient values using noise. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 14, Ho, Song and Papernot disclose the processor of claim 9.
Ho, Song and Papernot fail to explicitly disclose wherein the neural network comprises a diffusion model.
However, in an analogous art, Kingma discloses wherein the neural network comprises a diffusion model, (Kingma, [0043] describes wherein the neural network comprises a diffusion model).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Song and Papernot to include wherein the neural network comprises a diffusion model. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 15, Ho, Song and Papernot disclose the processor of claim 9.
Ho, Song and Papernot fail to explicitly disclose wherein the one or more circuits are to select the first duration of time and the second duration of time according to the predetermined distribution indicative of at least one of time or noise level.
However, in an analogous art, Kingma discloses wherein the one or more circuits are to select the first duration of time and the second duration of time according to the predetermined distribution indicative of at least one of time or noise level, (Kingma, [0056] describes wherein the one or more circuits are to select the first duration of time [0023]-[0024] and the second duration of time [0023]-[0024] according to a predetermined distribution [0026] indicative of at least one of time [0023]-[0024] or noise level [0022]-[0023])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Song and Papernot to include wherein the one or more circuits are to select the first duration of time and the second duration of time according to the predetermined distribution indicative of at least one of time or noise level. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Regarding claim 17, Ho, Song and Papernot disclose the processor of claim 15.
Ho, Song and Papernot fail to explicitly disclose wherein the predetermined distribution extends between a minimum value that is greater than zero and a maximum value.
However, in an analogous art, Kingma discloses wherein the predetermined distribution extends between a minimum value that is greater than zero and a maximum value, (Kingma, describes [0032] wherein the predetermined distribution extends between a minimum value that is greater than zero [0029] and a maximum value [0032])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Kingma with the system/method of Ho, Song and Papernot to include wherein the predetermined distribution extends between a minimum value that is greater than zero and a maximum value. One would have been motivated to provide efficient optimization of the noise schedule jointly with the rest of the diffusion model (Kingma, [0020]).
Claims 6 and 11 rejected under 35 U.S.C. 103 as being unpatentable over Ho et al (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Song et al (“Song,” “Stochastic Gradient Descent with Differentially Private Updates,” IEEE, 2013, Pages 245-248) in view of Papernot et al (“Papernot,” “Semi-Supervised Knowledge Transfer for Deep Learning From Private Training Data,” 2017, Pages 1-16) and further in view of Xiao et al (“Xiao,” US 20220248179).
Regarding claim 6, Ho, Song and Papernot disclose the processor of claim 1.
Ho, Song and Papernot fail to explicitly disclose wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the neural network.
However, in an analogous art, Xiao discloses wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the network, (Xiao, [0026], [0033]-[0036], wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the neural network)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Xiao with the system/method of Ho, Song and Papernot to include wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the neural network. One would have been motivated to construct an artificial neural network based on a criterion (Xiao, [0026]-[0027]).
Regarding claim 11, Ho, Song and Papernot disclose the processor of claim 9.
Ho, Song and Papernot fail to explicitly disclose wherein the at least one privacy criterion corresponds to a restriction on iterations of updating the neural network.
However, in an analogous art, Xiao discloses wherein the at least one privacy criterion corresponds to a restriction on iterations of updating the neural network, (Xiao, [0033]-[0036], wherein the at least one privacy criterion corresponds to a restriction on a number of iterations of updating the network)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Xiao with the system/method of Ho, Song and Papernot to include wherein the at least one privacy criterion corresponds to a restriction on iterations of updating the neural network. One would have been motivated to construct an artificial neural network based on a criterion (Xiao, [0026]-[0027]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Ho et al (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Song et al (“Song,” “Stochastic Gradient Descent with Differentially Private Updates,” IEEE, 2013, Pages 245-248) in view of Papernot et al (“Papernot,” “Semi-Supervised Knowledge Transfer for Deep Learning From Private Training Data,” 2017, Pages 1-16) and further in view of Tanski et al (“Tanski,” US 20230376833).
Regarding claim 8, Ho, Song and Papernot disclose disclose the processor of claim 1.
Ho, Song and Papernot disclose fail to explicitly disclose wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
However, in an analogous art, Tanski discloses wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center;
or a system implemented at least partially using cloud computing resources, (Tanski, [0144], [0034], [0037] describes a system implemented at least partially using cloud computing resources)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Tanski with the system/method of Ho, Song and Papernot disclose to include wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. One would have been motivated to automatically identify risk control features (Tanski, [0003]).
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Ho et al (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Song et al (“Song,” “Stochastic Gradient Descent with Differentially Private Updates,” IEEE, 2013, Pages 245-248) in view of Papernot et al (“Papernot,” “Semi-Supervised Knowledge Transfer for Deep Learning From Private Training Data,” 2017, Pages 1-16) and further in view of Toroman et al (“Toroman,” US 20230107337).
Regarding claim 12, Ho, Song and Papernot disclose the processor of claim 9.
Ho, Song and Papernot fail to explicitly disclose wherein: the training data instance is a first training data instance, and a first training data set comprises the first training data instance; and the one or more circuits are further to update the neural network using a plurality of second training data instances of a second training data set separate from the first training data set.
However, in an analogous art, Toroman discloses wherein: the training data instance is a first training data instance, (Toroman, [0040] describes wherein: the training data instance is a first training data instance [0040], and a first training data set [0036], [0082] comprises the first training data instance [0040])
and a first training data set comprises the first training data instance; and the one or more circuits are further to update the neural network using a plurality of second training data instances of a second training data set separate from the first training data set (Toroman, [0040] describes and the one or more circuits [0131] are further to update the neural network [0035], [0092] using a plurality of second training data instances [0040] of a second training data set [0036], [0082] separate from the first training data set [0040]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Toroman with the system/method of Ho, Song and Papernot to include wherein: the training data instance is a first training data instance, and a first training data set comprises the first training data instance; and the one or more circuits are further to update the neural network using a plurality of second training data instances of a second training data set separate from the first training data set. One would have been motivated to encode multi-scale time series data to manage machine operations (Toroman, [0002]).
Regarding claim 13, Ho, Song and Papernot disclose the processor of claim 9.
Ho, Song and Papernot fail to explicitly disclose wherein the one or more circuits are to: apply an autoencoder to provide the training data instance in a latent data space; and provide the training data instance from the latent data space to the neural network.
However, in an analogous art, Toroman discloses wherein the one or more circuits are to: apply an autoencoder to provide the training data instance in a latent data space; (Toroman, [0017], describes wherein the one or more circuits [0131] are to: apply an autoencoder [0017] to provide the training data instance [0040] in a latent data space [0036]).
and provide the training data instance from the latent data space to the neural network, (Toroman, [0040], describes and provide the training data instance [0040] from the latent data space [0036] to the neural network [0019])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Toroman with the system/method of Ho, Song and Papernot to include wherein the one or more circuits are to: apply an autoencoder to provide the training data instance in a latent data space; and provide the training data instance from the latent data space to the neural network. One would have been motivated to encode multi-scale time series data to manage machine operations (Toroman, [0002]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Ho et al (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25), Song et al (“Song,” “Stochastic Gradient Descent with Differentially Private Updates,” IEEE, 2013, Pages 245-248), Papernot et al (“Papernot,” “Semi-Supervised Knowledge Transfer for Deep Learning From Private Training Data,” 2017, Pages 1-16) in view of Kingma et al (“Kingma,” WO 2022265992) and further in view of Mireshghallah et al (“Mireshghallah,” WO 2021178911).
Regarding claim 16, Ho, Song, Papernot and Kingma disclose the processor of claim 15.
Ho, Song, Papernot and Kingma fail to explicitly disclose wherein the one or more circuits are to identify the predetermined distribution from a plurality of distributions according to the at least one privacy criterion.
However, in an analogous art, Mireshghallah discloses wherein the one or more circuits are to identify the predetermined distribution from a plurality of distributions according to the at least one privacy criterion, (Mireshghallah, [00146] describes wherein the one or more circuits are to identify the predetermined distribution from a plurality of distributions [0007]-[0008] according to the at least one privacy criterion, [0010])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Mireshghallah with the system/method of Ho, Song, Papernot and Kingma to include wherein the one or more circuits are to identify the predetermined distribution from a plurality of distributions according to the at least one privacy criterion. One would have been motivated to provide a neural network which performs an inference task on the data (Mireshghallah, [0006]).
Claims 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ho et al (“Ho,” “Denoising Diffusion Probabilistic Models,” 2020, Pages 1-25) and further in view of Song et al (“Song,” “Stochastic Gradient Descent with Differentially Private Updates,” IEEE, 2013, Pages 245-248).
Regarding claim 18, Ho discloses a method, comprising:
generating, using a neural network and based at least on receiving an indication of one or more features, an output corresponding to the one or more features, (Ho, Pages 2-8 describe the overall system, Page 5, Under Section 4 Experiments, Pages 14-15 Under Section B. Experimental Results discloses a neural network (U-Net) that receives as input: a noisy data sample a timestep/noise-level embedded. The network generates an output (predicted noise or denoised signal) corresponding to input features. Conditioning on inputs/features is described and extended to conditional generation. The neural network is trained to predict the noise added to timestep t and t is provided to the network via a timestep embedding)
the first training data point being determined by applying a first amount of noise to a training data instance and the second training data point being determined by applying a second amount of noise to the training data instance, (Ho discloses Pages 2-3, Section 2 Background we sample t uniformly from (1,…T) for each training example. The forward process gradually adds noise over T timesteps. The first and second training data points correspond to different sampled timesteps and the duration of time/noise level reads on timestep/noise variance schedule. Uniform distribution over timesteps is described where the minimum is t=1 and the maximum is t=T. Each timestep corresponds to a specific noise variance level).
and causing, using at least one of a display or an audio speaker device, presentation of the output, (Ho describes a display on FIG 1, FIG 2, Page 4, Next to last Paragraph)
Ho fails to explicitly disclose wherein the neural network comprises one or more parameters trained according to at least one privacy criterion and at least a first training data point and a second training data point, and based at least on a combined objective value determined by combining a plurality of objective values from a plurality of comparisons of (i) a plurality of estimated outputs for the first training data point and the second training data point to (ii) the training data instance;
However, in an analogous art, Song discloses wherein the neural network comprises one or more parameters trained according to at least one privacy criterion and at least a first training data point and a second training data point, (Song discloses Page 248, Left Column, Conclusion and Pages 246-247 Under Section III. SGD with Differential Privacy stochastic gradient descent algorithms with updates modified to ensure differential privacy. The amount of noise is calibrated according to a desired privacy level also see Page 247 under Section B. Procedure and Section C. Mini-batching Reduces Variance)
and based at least on a combined objective value determined by combining a plurality of objective values from a plurality of comparisons of (i) a plurality of estimated outputs for the first training data point and the second training data point to (ii) the training data instance; (Song discloses Page 248, Left Column, Conclusion and Pages 246-247 Under Section III describes and based at least on a combined objective value determined by combining a plurality of objective values from a plurality of comparisons of (i) a plurality of estimated outputs for the first training data point and the second training data point to (ii) the training data instance)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the teachings of Song with the system/method of Ho to include wherein the neural network comprises one or more parameters updated according to at least one privacy criterion and using at least a first training data point and a second training data point. One would have been motivated to provide stochastic gradient descent with differentially private updates (Song, Page 248, Left Column, Conclusion).
Regarding claim 19, Ho and Song disclose the method of claim 18.
Ho further discloses wherein the neural network comprises a diffusion model, (Ho, Page 2, First Three Paragraph describes the neural network comprises a diffusion model)
Regarding claim 20, Ho and Song disclose the method of claim 18.
Ho further discloses wherein the output comprises at least one of (Ho, Page 2, FIG 2 shows the output)
text data,
speech data,
or image data, (Ho, Page 2, FIG 2 shows the output is an image)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES J WILCOX whose telephone number is (571)270-3774. The examiner can normally be reached M-F: 8 A.M. to 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Luu T. Pham can be reached on (571)270-5002. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES J WILCOX/Examiner, Art Unit 2439
/LUU T PHAM/Supervisory Patent Examiner, Art Unit 2439