Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 5, 11-12, 14, and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Luus et al. (from IDS: US Pub. No. 2021/0334403, published Oct. 2021, hereinafter “Luus”).
Regarding claim 1, Luus teaches a method of privacy protection comprising:
receiving, by a generator from a service customer, a service request requesting for a service of privacy protection (Luus, [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4. The data synthesized by the data publisher 316 that is shared with the data recipient 320 has privacy guaranteed” – teaches receiving, by a generator (Membership Inference GAN) from a service customer (data publisher), a service request requesting for a service of privacy protection (data recipient receives data from data publisher which employs a Membership Inference GAN that implements a privacy-hardening service, thus the publisher receives a service request from the recipient requesting for a service of privacy protection and employs a generator according to the request));
determining, by the generator, a generative model to generate synthetic data based on the service request protection (Luus, [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4. The data synthesized by the data publisher 316 that is shared with the data recipient 320 has privacy guaranteed” – teaches determining, by the generator (Membership Inference GAN), a generative model (generator of Membership Inference GAN) to generate synthetic data based on the service request protection (data synthesized by publisher using the Membership Inference GAN that is shared with recipient has privacy guaranteed, thus determining a generative model to generate synthetic data based on the service request protection));
performing, by the generative model, a generation of synthetic data, and providing the synthetic data to a discriminator (Luus, [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4.” and in [0079] – “FIG. 4 is a block diagram of an example data generating system 400 for generating representative data while preserving data privacy, in accordance with an example embodiment. With reference to FIG. 4, the generator 420, the discriminator 424, a classifier 412, and the privacy adversary 436 are machine learning components… The generator 420 generates a full feature vector from a noise vector and a desired class label (Y_G). In one example embodiment, a learning model 432 is used by the generator 420 (in an untrusted zone 444) to create generated records 408 based on training records 416 obtained from a database of sensitive data (sensitive database 404). As described above, the discriminator 424 is trained to identify the generated samples as being fake (that is, identify the generated records 420 as being generated) and to identify a training sample as being real (that is, as originating from the training records 416).” – teaches performing, by the generative model (generator 420), a generation of synthetic data (generated records 408 and 420), and providing the synthetic data to a discriminator (discriminator 424 is trained to identify the generated records 420 as being generated, thus the synthetic data is provided to a discriminator));
performing, by a discriminative model invoked by the discriminator, a comparison between data from the service customer and received synthetic data, and providing a result of the comparison to the generator (Luus, [0079] – “As described above, the discriminator 424 is trained to identify the generated samples as being fake (that is, identify the generated records 420 as being generated) and to identify a training sample as being real (that is, as originating from the training records 416). In essence, the discriminator 424 is awarded for successfully identifying a generated sample as being fake and the generator 420 is awarded for fooling the discriminator 424 into identifying a generated sample as being real.” – teaches performing, by a discriminative model invoked by the discriminator (discriminator 424 trained to identify real and fake samples), a comparison between data from the service customer (training records 416) and received synthetic data (generated records 420. The discriminator performs a comparison between the training records and generated records to determine if records are generated or originating from training data, generator is awarded for fooling the discriminator, thus a result of the comparison by the discriminator is provided to the generator)), wherein privacy of the service customer is included in or inferred from the data from the service customer (Luus, [0079] – “In one example embodiment, the generator 420, the discriminator 424, the classifier 412, and the privacy adversary 436 are in data communication with each other (for example, via a shared data structure) so as to gain access to the generated records 420, the training records 416, and other data… Meanwhile, the privacy adversary 436 is trained to distinguish between the training records 416 and the reference records 440, and to identify a training sample as being more similar to the distribution of the generated records 408 than the distribution of the reference records 440. In one example embodiment, the privacy adversary 436 is a multi-layer neural network that produces a binary value indicating whether the input sample (a training sample from the training records 416) “originates from” the generated records 408 or the reference records 440.” – teaches wherein privacy of the service customer is included in or inferred from the data from the service customer (privacy adversary is a multi-layer neural network trained to distinguish between training records and reference records and identifies if the input sample originates from generated or reference records, thus the privacy of the service customer is included or inferred from the data from the service customer));
according to the result of the comparison from the discriminator, updating, by the generator, the generative model until updated synthetic data generated by updated generative model meets a preconfigured requirement, and each time when the generative model is updated, providing newly updated synthetic data to the discriminator (Luus, [0052] – “In one example embodiment, the generated data is optimized for privacy such that it cannot be used to determine if any particular generated record was part of or originated from the training dataset. In addition, models generated based on the representative training dataset are generated such that the privacy of the training dataset is preserved.”, [0079] – “ In essence, the discriminator 424 is awarded for successfully identifying a generated sample as being fake and the generator 420 is awarded for fooling the discriminator 424 into identifying a generated sample as being real.” and in [0081] – “Training the various models requires different weight updates for each of the modules during back propagation. In one example embodiment, as described more fully below in conjunction with FIG. 6, the generator 420, the discriminator 424, the classifier 412, and the privacy adversary 436 are sequentially trained. In one example embodiment, the training sequence is repeated to further improve the operation of the system 400. ” – teaches updating by the generator, according to the result of the comparison from the discriminator, the generative model until updated synthetic data generated by the updated generative model meets a preconfigured requirement (optimized for privacy such that the generated data cannot be used to determine if any record was part of the training dataset, thus updating the generator in optimization until the updated synthetic data meets a preconfigured requirement, the requirement being a level of privacy such that the generated data cannot be used to determine if any record belonged to the training dataset), and each time when the generative model is updated, providing newly updated synthetic data to the discriminator (training sequence is repeated, thus the generative model provides newly updated synthetic data to the discriminator each time when the generative model is updated));
once the preconfigured requirement is met, providing, by the generator to a data consumer, at least one of:
the latest updated synthetic data which meets the preconfigured requirement (Luus, [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4. The data synthesized by the data publisher 316 that is shared with the data recipient 320 has privacy guaranteed.” – teaches once the preconfigured requirement is met, providing, by the generator (GAN of data publisher) to a data consumer (data recipient), the latest updated synthetic data which meets the preconfigured requirement (the data synthesized by the data publisher is shared with the data recipient, the synthesized data has privacy guaranteed)), and
configuration information enabling an establishment of the latest updated generative model which generated the latest updated synthetic data (Alternative language. Luus teaches a different alternative and thus this limitation is not required under broadest reasonable interpretation of the claim.),
wherein the data consumer has no authorization to access the privacy of the service customer (Luus, [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4. The data synthesized by the data publisher 316 that is shared with the data recipient 320 has privacy guaranteed.” – teaches wherein the data consumer (data recipient 320) has no authorization to access the privacy of the service customer (data synthesized by data publisher that is shared with data recipient 320 has privacy guaranteed, thus the data consumer has no authorization to access the privacy of the service customer)).
Claim 20 incorporates substantively all the limitations of claim 1 in a system, and is rejected on similar grounds as above.
Regarding claim 2, Luus teaches the method according to claim 1, wherein
the service request from the service customer comprises a requirement of a generative model and the generative model determined by the generator meets the requirement of the generative model, wherein the requirement of the generative model includes one or more of:
a privacy level to be supported by the generative model (Luus, [0052] – “In one example embodiment, the generated data is optimized for privacy such that it cannot be used to determine if any particular generated record was part of or originated from the training dataset. In addition, models generated based on the representative training dataset are generated such that the privacy of the training dataset is preserved.” and in [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4. The data synthesized by the data publisher 316 that is shared with the data recipient 320 has privacy guaranteed” – teaches wherein the service request from the service customer comprises a requirement of a generative model and the generative model determined by the generator meets the requirement of the generative model (data recipient requests data synthesized by data publisher with privacy guaranteed, thus the service request from the service customer comprises a privacy requirement of a generative model and the generative model determined by the generator meets the privacy requirement), wherein the requirement of the generative model includes one or more of: a privacy level to be supported by the generative model (models generated based on representative training dataset and generated data is optimized for privacy such that it cannot be used to determine if any particular record was part of the training dataset, thus the requirement comprises a privacy level to be supported by the generative model));
Regarding claim 3, Luus teaches the method according to claim 2, wherein the model type includes one of:
a model based on a generative adversarial network (GAN), a model based on a generative neural network (GNN), a model based on an auto-encoder, and a model based on a variational auto-encoder (VAE) (Luus, [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4.” and in [0079] – “FIG. 4 is a block diagram of an example data generating system 400 for generating representative data while preserving data privacy, in accordance with an example embodiment.” – teaches wherein the model type includes one of a model based on a generative adversarial network (data publisher comprises a Membership Inference GAN)).
Regarding claim 5, Luus teaches the method according to claim 1, wherein the updating the generative model comprises:
updating at least one weight between neurons of neural network associated with the generative model or at least one hyper-parameter associated with the generative model (Luus, [0081] – “Training the various models requires different weight updates for each of the modules during back propagation. In one example embodiment, as described more fully below in conjunction with FIG. 6, the generator 420, the discriminator 424, the classifier 412, and the privacy adversary 436 are sequentially trained.” – teaches wherein updating the generative model comprises updating at least one weight between neurons of the neural network associated with the generative model or at least one hyperparameter associated with the generative model (training requires different weight updates for each module during back propagation, thus at least one weight of the generator is updated during back propagation)).
Regarding claim 11, Luus teaches the method according to claim 1, wherein
the generator and the discriminator are located in an entity, wherein the service request is received by the entity from the service customer (Luus, [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4.” and in [0079] – “FIG. 4 is a block diagram of an example data generating system 400 for generating representative data while preserving data privacy, in accordance with an example embodiment. With reference to FIG. 4, the generator 420, the discriminator 424, a classifier 412, and the privacy adversary 436 are machine learning components. In one example embodiment, the generator 420, the discriminator 424, the classifier 412, and the privacy adversary 436 are in data communication with each other (for example, via a shared data structure) so as to gain access to the generated records 420, the training records 416, and other data.” – teaches wherein the generator and discriminator are located in an entity (data publisher, data generating system 400), wherein the service request is received by the entity from the service customer (data publisher employs a Membership Inference GAN, the data generating system comprises a generator and a discriminator, thus the data publisher receives a service request from the data recipient and the generator and discriminator are located in the data publisher)).
Regarding claim 12, Luus teaches the method according to claim 11, wherein
the entity is a data de-privatization (DP) service provider whose interface with the service customer supports a transmission of one or more of the service request and the data (Luus, [0029] – “Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models. “, [0037] – “Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail).”, and in [0055] – “In one example embodiment, to protect the private information of the sensitive data 304, a data publisher 316 that employs a Membership Inference GAN (MIGAN) using a conditional GAN structure implements a privacy-hardening process, as described more fully below in conjunction with FIG. 4. The data synthesized by the data publisher 316 that is shared with the data recipient 320 has privacy guaranteed.” – teaches wherein the entity (data publisher) is a data de-privatization service provider whose interface with the service customer supports a transmission of one or more of the service request and the data (data recipient requests synthesized data with privacy guarantee service of data publisher, thus the data publisher is a data de-privatization service provider whose interface with the data recipient supports transmission of the service request)); or,
wherein the entity has an interface with a DP service provider and no interface with the service customer, while the DP service provider has an interface supporting a transmission of one or more of the service request and the data with the service customer, and the service request is received by the entity from the service customer via the DP service provider (Alternative language. Luus teaches a different alternative and thus this limitation is not required under broadest reasonable interpretation of the claim.).
Regarding claim 14, Luus teaches the method according to claim 1, further comprising one of:
receiving, by the discriminator, the data from the service customer via an interface between the discriminator and the service customer (Luus, [0037] – “Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). ”, [0079] – “As described above, the discriminator 424 is trained to identify the generated samples as being fake (that is, identify the generated records 420 as being generated) and to identify a training sample as being real (that is, as originating from the training records 416). In essence, the discriminator 424 is awarded for successfully identifying a generated sample as being fake and the generator 420 is awarded for fooling the discriminator 424 into identifying a generated sample as being real.” and in Fig. 3 – teaches one of receiving, by the discriminator, the data from the service customer (as shown in Fig. 3, discriminator receives training records from sensitive database, thus the discriminator receives sensitive data from the service customer), via an interface between the discriminator and the service customer (applications accessible through interface, thus the data received by the discriminator from the service customer is received via the interface)); and
obtaining, by the discriminator, the data locally in the service customer wherein the discriminator is located in the service customer (Alternative language. Luus teaches a different alternative and thus this limitation is not required under broadest reasonable interpretation of the claim.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4, 6-10, 13, and 15-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Luus in view of Walters et al. (US Patent No. 10,382,799, published Aug. 2019, hereinafter “Walters”).
Regarding claim 4, Luus teaches the method according to claim 1.
Luus fails to explicitly teach wherein the generator has no authorization to access the privacy from the service customer, and the performing, by the generative model, a generation of synthetic data comprises: generating, by the generative model, synthetic data according to random data and the service request.
However, analogous to the field of the claimed invention, Walters teaches: wherein,
the generator has no authorization to access the privacy from the service customer, and the performing, by the generative model, a generation of synthetic data comprises:
generating, by the generative model, synthetic data according to random data and the service request (Walters, Pg. 21, Col. 5, Lines 34-38 – “ As an additional example, dataset generator 103 can be configured to generate synthetic data using a data model without reliance on input data. For example, the data model can be configured to generate data matching statistical and content characteristics of a training dataset”, Pg. 21, Col. 6, Lines 56-62 – “The synthetic data may be similar to the actual data in terms of values, value distributions (e.g., univariate and multivariate statistics of the synthetic data may be similar to that of the actual data), structure and ordering, or the like. In this manner, the data model for the machine learning application can be generated without directly using the actual data.”, and in Pg. 22, Col. 7, Lines 3-12 – “The data model generation request can include data and/or instructions describing the type of data model to be generated. For example, the data model generation request can specify a general type of data model (e.g., neural network, recurrent neural network, generative adversarial network, kernel density estimator, random data generator, or the like) and parameters specific to the particular type of model (e.g., the number of features and number of layers in a generative adversarial network or recurrent neural network).” – teaches wherein the generator has no authorization to access the privacy from the service customer (generates data models without directly using actual data), and the performing, by the generative model, a generation of synthetic data comprises generating, by the generative model, synthetic data according to random data and the service request (synthetic data may be similar to actual data, data models are generated without directly using the actual data, dataset generator 103 can be configured to generate synthetic data using a data model that is generated without using actual data, thus teaching generating, by the generative model, synthetic data according to random data and the model generation request)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the generator with no authorization to access privacy and synthetic data generation of Walters to the generator and service requests of Luus. Doing so would protect the privacy and security of entities and individuals whose activities are recorded by the actual data (Walters, Pg. 21, Col. 6).
Regarding claim 6, Luus teaches the method according to claim 1.
Luus fails to explicitly teach wherein the generator is located in a first entity which has no interface supporting a transmission of the service request from the service customer, the discriminator is located in a second entity which has an interface supporting a transmission of the service request from the service customer, wherein the generator receives the service request from the service customer via the discriminator.
However, analogous to the field of the claimed invention, Walters teaches:
the generator is located in a first entity which has no interface supporting a transmission of the service request from the service customer, the discriminator is located in a second entity which has an interface supporting a transmission of the service request from the service customer, wherein the generator receives the service request from the service customer via the discriminator (Walters, Pg. 20, Col. 4, Lines 49-67 – “FIG. 1 depicts a cloud-computing environment 100 for generating data models. Environment 100 can be configured to support generation and storage of synthetic data, generation and storage of data models, optimized choice of parameters for machine learning, and imposition of rules on synthetic data and data models. Environment 100 can be configured to expose an interface for communication with other systems. Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113. These components of environment 100 can be configured to communicate with each other, or with external components of environment 100, using network 115. The particular arrangement of components depicted in FIG. 1 is not intended to be limiting. System 100 can include additional components, or fewer components. Multiple components of system 100 can be implemented using the same physical computing device or different physical computing devices.” and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches wherein the generator is located in a first entity which has no interface supporting a transmission of the service request from the service customer, the discriminator is located in a second entity which has an interface supporting a transmission of the service request from the service customer (GAN can be used by system, or environment, 100 to generate synthetic data, components of system 100 are implemented using different physical computing devices, thus the generator and discriminator are located in separate first and second entities, and the discriminator of system 100 can be configured to expose an interface for communication with other systems), wherein the generator receives the service request from the discriminator (generator and discriminator are components of system 100, components of system 100 can be configured to communicate with each other, thus the generator is configured to receive a service request from the discriminator)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the generator and discriminator located in first and second entities, respectively, of Walters to the generator, discriminator, and service requests of Luus. Doing so would allow for a secure environment for training models on sensitive data and enable deployment of a generator in a non-secure environment for generating synthetic data (Walters, Pg. 20, Col. 4).
Regarding claim 7, the combination of Luus and Walters teaches the method according to claim 6, further comprising:
receiving, by the discriminator from the service customer, a set of parameters to determine the discriminative model (Walters, Pg. 20, Col. 4, Lines 56-59 – “Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113.”, Pg. 21, Col. 5, Lines 46-58 – “Model optimizer 107 can include one or more computing systems configured to manage training of data models for system 100. Model optimizer 107 can be configured to generate models for export to computing resources 101. Model optimizer 107 can be configured to generate models based on instructions received from a user or another system. These instructions can be received through interface 113. For example, model optimizer 107 can be configured to receive a graphical depiction of a machine learning model and parse that graphical depiction into instructions for creating and training a corresponding neural network on computing resources 101. Model optimizer 107 can be configured to select model training parameters.”, and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches receiving, by the discriminator from the service customer, a set of parameters to determine the discriminative model (GAN that comprises discriminator is used by system 100, system 100 comprises a model optimizer that manages training of data models for system 100. Model optimizer can be configured to generate models based on instructions received through interface, such as configuration of model optimizer to select model training parameters, thus receiving a set of parameters to determine the discriminative model)); and
determining, by the discriminator, the discriminative model according to the set of parameters (Walters, Pg. 20, Col. 4, Lines 56-59 – “Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113.”, Pg. 21, Col. 5, Lines 46-58 – “Model optimizer 107 can include one or more computing systems configured to manage training of data models for system 100. Model optimizer 107 can be configured to generate models for export to computing resources 101. Model optimizer 107 can be configured to generate models based on instructions received from a user or another system. These instructions can be received through interface 113. For example, model optimizer 107 can be configured to receive a graphical depiction of a machine learning model and parse that graphical depiction into instructions for creating and training a corresponding neural network on computing resources 101. Model optimizer 107 can be configured to select model training parameters.”, and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches determining, by the discriminator, the discriminative model according to the set of parameters (model optimizer manages training and determination of model based on instructions received, model optimizer is configured to select parameters of models, such as the discriminator, based on received instructions, thus teaching determining the discriminative model according to the set of parameters)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the receiving of parameters and determining of the discriminative model according to received parameters of Walters to the discriminator, discriminative model, and service request of Luus. Doing so would allow users or other systems to provide instructions for model parameters and model generation (Walters, Pg. 21, Col. 5).
Regarding claim 8, the combination of Luus and Walters teaches the method according to claim 6, wherein
the second entity is a data de-privatization (DP) service provider (Walters, Pg. 20, Col. 4, Lines 49-67 – “FIG. 1 depicts a cloud-computing environment 100 for generating data models. Environment 100 can be configured to support generation and storage of synthetic data, generation and storage of data models, optimized choice of parameters for machine learning, and imposition of rules on synthetic data and data models… The particular arrangement of components depicted in FIG. 1 is not intended to be limiting. System 100 can include additional components, or fewer components. Multiple components of system 100 can be implemented using the same physical computing device or different physical computing devices.” and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches wherein the second entity is a data de-privatization service provider (system 100 uses GAN with discriminator to provide data de-privatization service, system 100 may implement discriminator in separate computing device, thus the second entity is a data de-privatization service provider)), the method further comprises:
receiving, by the generator from the data DP service provider, a split indication indicating that a generative model is required to output data for a comparison performed by a discriminative model (Walters, Pg. 26, Col. 16, Lines 43-57 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network. The generator network can be configured to learn a mapping from a sample space (e.g., a random number or vector) to a data space (e.g. the values of the sensitive data). The discriminator can be configured to determine, when presented with either an actual data sample or a sample of synthetic data generated by the generator network, whether the sample was generated by the generator network or was a sample of actual data.” – teaches receiving, by the generator from the data DP service provider (the discriminator entity), a split indication indicating that a generative model is required to output data for a comparison performed by a discriminative model (discriminative model is configured to determine if a sample is an actual data sample or a sample of the synthetic data generated by the generator network, thus teaching a split indication indicating that a generative model is required to output data for a comparison performed by a discriminative model));
wherein the generator determines the generative model according to the split indication (Walters, Pg. 26, Col. 16, Lines 57-62 – “As training progresses, the generator can improve at generating the synthetic data and the discriminator can improve at determining whether a sample is actual or synthetic data. In this manner, a generator can be automatically trained to generate synthetic data similar to the actual data.” – teaches wherein the generator determines the generative model according to the split indication (generator improves at generating synthetic data based on comparisons performed by discriminator, generator is automatically trained, and thus determined, to generate synthetic data similar to actual data, thus teaching wherein the generator determines the generative model according to the split indication)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the second entity and split indication of Walters to the generator, discriminator, and data de-privatization services of Luus. Doing so would allow the discriminator to compare synthetic data of the generative model with actual data and determine the generative model according to the result of the comparison (Walters, Pg. 26, Col. 16).
Regarding claim 9, Luus teaches the method according to claim 1.
Luus fails to explicitly teach the generator is located in a first entity which has an interface supporting a transmission of the service request from the service customer, the discriminator is located in the service customer, wherein the service request includes a split indication indicating that a generative model is required to output data for a comparison performed by a discriminative model.
However, analogous to the field of the claimed invention, Walters teaches: wherein
the generator is located in a first entity which has an interface supporting a transmission of the service request from the service customer, the discriminator is located in the service customer, wherein the service request includes a split indication indicating that a generative model is required to output data for a comparison performed by a discriminative model (Walters, Pg. 20, Col. 4, Lines 49-67 – “FIG. 1 depicts a cloud-computing environment 100 for generating data models. Environment 100 can be configured to support generation and storage of synthetic data, generation and storage of data models, optimized choice of parameters for machine learning, and imposition of rules on synthetic data and data models. Environment 100 can be configured to expose an interface for communication with other systems. Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113. These components of environment 100 can be configured to communicate with each other, or with external components of environment 100, using network 115. The particular arrangement of components depicted in FIG. 1 is not intended to be limiting. System 100 can include additional components, or fewer components. Multiple components of system 100 can be implemented using the same physical computing device or different physical computing devices.” and in Pg. 26, Col. 16, Lines 43-57 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network. The generator network can be configured to learn a mapping from a sample space (e.g., a random number or vector) to a data space (e.g. the values of the sensitive data). The discriminator can be configured to determine, when presented with either an actual data sample or a sample of synthetic data generated by the generator network, whether the sample was generated by the generator network or was a sample of actual data.” – teaches wherein the generator is located in a first entity which has an interface supporting a transmission of the service request from the service customer, the discriminator is located in the service customer (GAN can be used by system, or environment, 100 to generate synthetic data, components of system 100 are implemented using different physical computing devices, thus the generator is located in a first entity and the discriminator is located in the service customer, and the generator of system 100 can be configured to expose an interface supporting transmission of a service request), wherein the service request includes a split indication indicating that a generative model is required to output data for a comparison performed by a discriminative model (generative model outputs data for discriminator to determine if data is an actual sample or a sample of synthetic data, thus the service request includes a split indication indicating that a generative model is required to output data for a comparison by a discriminative model)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the generator located in a first entity and discriminator located in the service customer of Walters to the generator, discriminator, and service requests of Luus. Doing so would allow for a secure environment for training models on sensitive data and enable deployment of a generator in a non-secure environment for generating synthetic data (Walters, Pg. 20, Col. 4).
Regarding claim 10, the combination of Luus and Walters teaches the method according to claim 9, further comprising:
determining, by the service customer, the discriminative model according to a local stored set of parameters (Walters, Pg. 20, Col. 4, Lines 49-59 – “FIG. 1 depicts a cloud-computing environment 100 for generating data models. Environment 100 can be configured to support generation and storage of synthetic data, generation and storage of data models, optimized choice of parameters for machine learning, and imposition of rules on synthetic data and data models. Environment 100 can be configured to expose an interface for communication with other systems. Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113.” and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches determining, by the service customer, the discriminative model according to a local stored set of parameters (system 100 uses GAN with generator and discriminator to generate synthetic data, discriminator is determined by local parameters, which may be stored in model storage 109, of environment 100 which is the environment of the service customer, thus teaching determining, by the service customer, the discriminative model according to a local stored set of parameters)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the determining of the discriminative model of Walters to the discriminator and service customer of Luus. Doing so would allow for storing descriptive information of models locally and provide model information to users or systems (Walters, Pg. 21, Col. 5).
Regarding claim 13, Luus teaches the method according to claim 12.
Luus fails to explicitly teach wherein the DP service provider is configured to determine how to generate the generative model and the discriminator model according to the service request.
However, analogous to the field of the claimed invention, Walter teaches: wherein
the DP service provider is configured to determine how to generate the generative model and the discriminator model according to the service request (Walters, Pg. 20, Col. 4, Lines 56-59 – “Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113.”, Pg. 21, Col. 5, Lines 46-58 – “Model optimizer 107 can include one or more computing systems configured to manage training of data models for system 100. Model optimizer 107 can be configured to generate models for export to computing resources 101. Model optimizer 107 can be configured to generate models based on instructions received from a user or another system. These instructions can be received through interface 113. For example, model optimizer 107 can be configured to receive a graphical depiction of a machine learning model and parse that graphical depiction into instructions for creating and training a corresponding neural network on computing resources 101. Model optimizer 107 can be configured to select model training parameters.”, and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches wherein the DP service provider (system or environment 100) is configured to determine how to generate the generative model and the discriminator model according to the service request (GAN is used by system 100 to generate synthetic data, system 100 comprises a model optimizer 107 which manages training of data models for the system according to received instructions, thus the DP service provider is configured to determine how to generate the generative model and the discriminative model according to the service request or instructions received)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the determining of generative and discriminator models of Walters to the generator, discriminator, and service request of Luus. Doing so would enable an interface for receiving instructions that determine generation of models and selection of model parameters according to requests by a user or another system (Walters, Pg. 21, Col. 5)
Regarding claim 15, Luus teaches the method according to claim 1.
Luus fails to explicitly teach for each time when the synthetic data is generated, determining, by the generator, whether the preconfigured requirement is met, the preconfigured requirement indicating at least one of: how many times the synthetic data can be generated at most, how many times the generative model can be updated at most, how much similarity is between latest two generative models at least, and an indication is received from the discriminator wherein the indication indicates to provide the latest updated synthetic data or the configuration information enabling the establishment of the latest updated generative model to the data consumer.
However, analogous to the field of the claimed invention, Walters teaches:
for each time when the synthetic data is generated, determining, by the generator, whether the preconfigured requirement is met, the preconfigured requirement indicating at least one of:
how many times the generative model can be updated at most (Walters, Pg. 22, Col. 7, Line 65 – Pg. 22, Col. 8, Line 4 – “In some embodiments, system 100 can be configured to provide instructions for improving the quality of the synthetic data model. If a user requires synthetic data reflecting less correlation or similarity with the original data, the use can change the models' parameters to make them perform worse (e.g., by decreasing number of layers in GAN models, or reducing the number of training iterations).”, Pg. 23, Col. 9, Lines 36-48 – “ In some embodiments, computing resources 101 can be configured to update model optimizer 107 regarding the training status of the data model. For example, computing resources 101 can be configured to provide the current parameters of the data model and/or current performance criteria of the data model.”, and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches for each time when the synthetic data is generated, determining, by the generator whether the preconfigured requirement is met (computing resources 101 can update model optimizer 107 regarding the training status of the data model, computing resources 101 can provide training status and current parameters of data model, thus teaching determining whether the preconfigured requirement is met for each time synthetic data is generated during training), the preconfigured requirement indicating at least one of: how many times the generative model can be updated at most (user service request instructs system 100 to reduce the number of training iterations, thus teaching a preconfigured requirement indicating how many times the generative model can be updated at most)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the preconfigured requirement and determination of whether the preconfigured requirement is met of Walters to the generator and service requests of Luus. Doing so would allow users or other systems to provide instructions or requests for improving the quality of the synthetic data model (Walters, Pg. 22, Col. 7) and enable determination of parameters or performance criteria of the model at any iteration (Walters, Pg. 23, Col. 9).
Regarding claim 16, the combination of Luus and Walters teaches the method according to claim 15, further comprising:
sending, by the discriminator to the generator, a message including the preconfigured requirement to configure the preconfigured requirement into the generator, wherein the discriminator and the generator are located in different entities (Walters, Pg. 20, Col. 4, Lines 49-67 – “FIG. 1 depicts a cloud-computing environment 100 for generating data models. Environment 100 can be configured to support generation and storage of synthetic data, generation and storage of data models, optimized choice of parameters for machine learning, and imposition of rules on synthetic data and data models. Environment 100 can be configured to expose an interface for communication with other systems. Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113. These components of environment 100 can be configured to communicate with each other, or with external components of environment 100, using network 115. The particular arrangement of components depicted in FIG. 1 is not intended to be limiting. System 100 can include additional components, or fewer components. Multiple components of system 100 can be implemented using the same physical computing device or different physical computing devices.”, Pg. 22, Col. 7, Line 65 – Pg. 22, Col. 8, Line 4 – “In some embodiments, system 100 can be configured to provide instructions for improving the quality of the synthetic data model. If a user requires synthetic data reflecting less correlation or similarity with the original data, the use can change the models' parameters to make them perform worse (e.g., by decreasing number of layers in GAN models, or reducing the number of training iterations).” and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches sending, by the discriminator to the generator, a message including the preconfigured requirement to configure the preconfigured requirement into the generator, wherein the discriminator and the generator are located in different entities (user service request instructs system 100 to reduce the number of training iterations, generator of GAN is used by system 100 to generate synthetic data and may be located in a separate physical computing device from the discriminator, generator and discriminator can be configured to communicate with each other, thus the preconfigured requirement of reduced training iterations is preconfigured into the generator of system 100 by communication from the discriminator to the generator)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the configuration of the generator according to preconfigured requirements of Walters to the generator and discriminator of Luus. Doing so would allow users or other systems to provide instructions or requests for improving the quality of the synthetic data model (Walters, Pg. 22, Col. 7).
Regarding claim 17, the combination of Luus and Walters teaches the method according to claim 16, wherein
the discriminator is located in the service customer and the message including the preconfigured requirement is the service request from the service customer (Walters, Pg. 20, Col. 4, Lines 49-67 – “FIG. 1 depicts a cloud-computing environment 100 for generating data models. Environment 100 can be configured to support generation and storage of synthetic data, generation and storage of data models, optimized choice of parameters for machine learning, and imposition of rules on synthetic data and data models. Environment 100 can be configured to expose an interface for communication with other systems. Environment 100 can include computing resources 101, dataset generator 103, database 105, model optimizer 107, model storage 109, model curator 111, and interface 113. These components of environment 100 can be configured to communicate with each other, or with external components of environment 100, using network 115. The particular arrangement of components depicted in FIG. 1 is not intended to be limiting. System 100 can include additional components, or fewer components. Multiple components of system 100 can be implemented using the same physical computing device or different physical computing devices.”, Pg. 21, Col. 6, Lines 28-35 – “In various aspects, interface 113 can be configured to provide data or instructions received from other systems to components of system 100. For example, interface 113 can be configured to receive instructions for generating data models (e.g., type of data model, data model parameters, training data indicators, training parameters, or the like) from another system and provide this information to model optimizer 107.” and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches wherein the discriminator is located in the service customer and the message including the preconfigured requirement is the service request from the service customer (system 100 uses GAN with discriminator to generate synthetic data, discriminator communicates with interface 113 of system 100, interface 113 is configured to provide data or instructions received for generating data models, thus the discriminator is located within the service customer and the message including the preconfigured requirement is the service request)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the message comprising preconfigured requirements of Walters to the generator, discriminator, and service customers of Luus. Doing so would allow users or other systems to provide instructions or requests for improving the quality of the synthetic data model (Walters, Pg. 22, Col. 7).
Regarding claim 18, Luus teaches the method according to claim 1.
Luus fails to explicitly teach for each time when the comparison is performed, determining, by the discriminator, whether the preconfigured requirement is met, wherein the preconfigured requirement indicating at least one of: how many comparisons can be performed at most; and how many times the discriminative model can be updated at most.
However, analogous to the field of the claimed invention, Walters teaches: further comprising:
for each time when the comparison is performed, determining, by the discriminator, whether the preconfigured requirement is met, wherein the preconfigured requirement indicating at least one of:
how many times the discriminative model can be updated at most (Walters, Pg. 22, Col. 7, Line 65 – Pg. 22, Col. 8, Line 4 – “In some embodiments, system 100 can be configured to provide instructions for improving the quality of the synthetic data model. If a user requires synthetic data reflecting less correlation or similarity with the original data, the use can change the models' parameters to make them perform worse (e.g., by decreasing number of layers in GAN models, or reducing the number of training iterations).”, Pg. 23, Col. 9, Lines 36-48 – “ In some embodiments, computing resources 101 can be configured to update model optimizer 107 regarding the training status of the data model. For example, computing resources 101 can be configured to provide the current parameters of the data model and/or current performance criteria of the data model.”, and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches for each time when the comparison is performed, determining, by the discriminator whether the preconfigured requirement is met (computing resources 101 can update model optimizer 107 regarding the training status of the data model, computing resources 101 can provide training status and current parameters of data model, thus teaching determining whether the preconfigured requirement is met for each time comparison is performed during training), the preconfigured requirement indicating at least one of: how many times the discriminative model can be updated at most (user service request instructs system 100 to reduce the number of training iterations, thus teaching a preconfigured requirement indicating how many times the discriminative model can be updated at most)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the preconfigured requirement and determination of whether the preconfigured requirement is met of Walters to the discriminator and service requests of Luus. Doing so would allow users or other systems to provide instructions or requests for improving the quality of the synthetic data model (Walters, Pg. 22, Col. 7) and enable determination of parameters or performance criteria of the model at any iteration (Walters, Pg. 23, Col. 9).
Regarding claim 19, the combination of Luus and Walters teaches the method according to claim 18, wherein
the preconfigured requirement is preconfigured into the discriminator by the service customer (Walters, Pg. 22, Col. 7, Line 65 – Pg. 22, Col. 8, Line 4 – “In some embodiments, system 100 can be configured to provide instructions for improving the quality of the synthetic data model. If a user requires synthetic data reflecting less correlation or similarity with the original data, the use can change the models' parameters to make them perform worse (e.g., by decreasing number of layers in GAN models, or reducing the number of training iterations).” and in Pg. 26, Col. 16, Lines 43-50 – “FIG. 8 depicts a process 800 for training a generative adversarial network using a normalized reference dataset. In some embodiments, the generative adversarial network can be used by system 100 (e.g., by dataset generator 103) to generate synthetic data (e.g., as described above with regards to FIGS. 2, 3, 5A and 5B). The generative adversarial network can include a generator network and a discriminator network.” – teaches wherein the preconfigured requirement is preconfigured into the discriminator by the service customer (user service request instructs system 100 to reduce the number of training iterations, discriminator of GAN is used by system 100 to generate synthetic data, thus the preconfigured requirement of reduced training iterations is preconfigured into the discriminator of system 100)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the configuration of the discriminator according to preconfigured requirements of Walters to the discriminator and service customers of Luus. Doing so would allow users or other systems to provide instructions or requests for improving the quality of the synthetic data model (Walters, Pg. 22, Col. 7).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Liu et al. (NPL: PPGAN: Privacy-preserving Generative Adversarial Network, published Jan. 2020) teaches a privacy-preserving GAN which achieves differential privacy by adding noise to the gradient during model learning. Teaches preconfigured requirements comprising levels of privacy that the generator and discriminator are trained to meet. Teaches generating the generator and discriminator using local parameters. Teaches wherein the generator is trained on random data, without using real samples.
Xin et al. (Private FL-GAN: Differential Privacy Synthetic Data Generation Based on Federated Learning, published April 2020) teaches a differential privacy generative adversarial network based on federated learning. Teaches wherein both the generator and discriminator are located in a service provider and service customer, or in a first and second entity. Teaches wherein parameters of the generator and discriminator are updated locally at the service customer. Teaches determining privacy levels of several models based on setting different privacy parameters.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LOUIS C NYE whose telephone number is 571-272-0636. The examiner can normally be reached Monday - Friday 9:00AM - 5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATT ELL can be reached at 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LOUIS CHRISTOPHER NYE/Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights