DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/23/2026 has been entered.
Response to Arguments
Remarks pages 7-10, Applicant contends:
Raval and Kuta do not disclose the elements originally recited in claim 3.
Response:
As shown in the updated teachings for the amended claim limitations for claim 1, the amendments indicating a separate training of a MLP for the polluting of the elements is taught using David. Previous claim limitations were not interpreted to convey a separation of elements between data being used to train the autoencoder and the MLP. The teachings regarding pretraining were updated to help note how the current prior art is seen as teaching the elements. In regards to the teachings of Kuta, Kuta is noted for teaching the losses mentioned for the similarity and reconstruction, Kuta teaches the training of an autoencoder. The training related to the encoder is interpreted as a similarity loss, as Kuta notes in paragraph 49 the preserving of data encoded to ensure decoding generated an image closely matching the input, which also indicates a reconstruction loss as notes the output image and input image are made to closely resemble each other. No requirement to particular reference attributes is seen as needed, for attributes are the data being input into the system. Thus the similarity is the data input into the encoder (“attributes”) and what is in the latent space (“attribute in its space”). The particular similarity loss functions or reconstruction loss functions are taught when required in dependent claims, such as how Kuta 0049 indicates the wanted function for reconstruction of L2 norm in claim 4. The claim mappings have been updated for clarity, such as using the reference David to indicate aspects of the cosine distance used for the similarity and additional descriptions for noting aspects of the interpretation.
[Kuta 0049]: “This function trains identity preserving encoder 101 to encode data that is decoded to generate an image that most closely match the input image x, thereby preserving all image information including identity information. Any distance function d may be used, such as, e.g., the L2 norm, d(a, b)=∥a−b∥^2, an angle between encoded vectors, or other differences.”
Remarks page 10, Applicant contends:
The combination does not disclose "the multi-layer perceptron is trained to pollute the space for the uninterested attribute so that it cannot be discovered after decoding"
Response:
No claim in the previous office action indicates the idea of polluting a space for the uninterested attribute. Claim 19 notes an attribute concealer is trained, but no indication of "pollute" or "polluting" is given for the method of concealing. Meaning Raval not indicating polluting is caused by a lack of prima facie case on the limitation. The current specification does not appear to define polluting.
Applicant’s arguments with respect to claim(s) 1 have been considered but are moot because the new ground of rejection contain elements that have not been previously examined or does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Remarks page 11, Applicant contends:
The combination does not disclose "receiving, by the attribute concealing computer program, a plurality of attribute concealing data sets that are related to the uninterested attribute"
Response:
Applicant’s arguments with respect to claim(s) 1 have been considered but are moot because the new ground of rejection contain elements that have not been previously examined or does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The basis of the data sets being changed to be related to the uninterested attribute requires prima facie case.
The earlier responses to remarks indicates the interpretation related to a separate training of a MLP and autoencoder given indication by the amended claims, which is noted to be relevant to the interpretation of training a MLP. As a result, the current claims are noted with explanations of the currently amended teachings.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4-9, 11-18, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Raval et al (“Olympus: Sensor Privacy through Utility Aware Obfuscation”), referred to as Raval in this document, and further in view of Kuta et al (US 20220012362 A1), referred to as Kuta in this document, and further in view of David et al (US 20230259787 A1), referred to as David in this document.
Regarding Claim 1:
Raval teaches:
A method for concealing uninterested attributes using generative adversarial networks, comprising:
[Raval Introduction Page 1]: “Olympus, a privacy framework that uses generative adversarial networks [A method for concealing uninterested attributes using generative adversarial networks, comprising] (GAN) [20] to solve the problem.”
a variational autoencoder to separate each attribute in multi- attribute training data received from a data source into a space for that attribute;
a decoder to reconstruct data from the spaces;
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer is much smaller in size. The idea is to encode [a variational autoencoder to separate each attribute in multi- attribute training data received from a data source into a space for that attribute where the training data aspect is taught by input being able to be datasets for learning] the input to this compact middle layer and then decode [a decoder to reconstruct data from the spaces] it back to the original size in the output layer. This encoding and decoding process is learned by minimizing the privacy and utility losses of the output (obfuscated data). The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output. Essentially, the privacy loss forces the Obfuscator to throw away private information, while the utility loss forces it to preserve the information required by the target app.”
pretraining, by an attribute concealing computer program executed by an electronic device,
pretraining, by the attribute concealing computer program,
[Raval 4.2 page 9]: “Using these training data and the app, Olympus learns the obfuscation that hides driver’s identity by minimizing the attacker’s ability to identify drivers in the obfuscated images. At the same time, it ensures that the obfuscated images preserve the activity by maximizing the app’s accuracy of identifying the activity in the obfuscated images. To improve usability, we envision a repository of pretrained [pretraining] [pretraining] obfuscators that users can simply use to protect their privacy against various third party apps.”
The idea of being able to utilize pretrained models is shown above by Raval. Raval also indicates the ability to train models in the above quote. As a result, the ability to pretrain and use pretrained models is shown by Raval. Pretraining is also interpreted in some aspects to just refer to elements of training, as no particular level of pretraining is indicated, thus the pretrained element could be at any level of training, including at an untrained level.
receiving, by the attribute concealing computer program, an identification of an uninterested attribute in the multi-attribute data to conceal and an interested attribute to retain
[Raval 2.1 Problem Setting Page 2]: “Our goal is to design a utility aware obfuscation: given a set of ML models U that take as input x, and a specification of private attributes in the input S [receiving, by the attribute concealing computer program, an identification of an uninterested attribute in the multi-attribute data to conceal and an interested attribute to retain], construct an obfuscation mechanism M such that the privacy loss and the accuracy loss are jointly minimized. We achieve this by developing a privacy framework – Olympus, inspired by the idea of adversarial games [20].”
Support is given in Raval to indicate the private attributes are an alternative method or noting of input for functionality that should be preserved. [Raval Introduction page 1]: “In Olympus, a user can specify utility using one or more apps whose functionality must be preserved. A user can specify privacy using a set of labeled examples on data collected from the sensors. Given this training data and access to the target app(s), Olympus learns an obfuscation mechanism that jointly minimizes both privacy and utility losses”
receiving, by the attribute concealing computer program, a plurality of attribute concealing data sets that are related to the uninterested attribute
[Raval 5.6 Privacy-Utility Tradeoff page 14]: “We evaluate above mentioned methods as well as Olympus on three image datasets [receiving, by the attribute concealing computer program, a plurality of attribute concealing data sets that are related to the uninterested attribute where user input to support receiving is given later for receiving attribute data] with the goals of protecting respective private attributes and preserving the functionality of the corresponding target app, i.e., the classifier”
The relation to the uninterested attribute is shown by the datasets being used in Raval are used in the process of “protecting respective private attributes and preserving the functionality of the corresponding target app”, which is noted to involve “The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output.” in Raval 2.3.1. This means the information in the dataset includes information for the uninterested attribute as that would be the information thrown away.
The datasets being changed from “additional datasets” to “attribute concealing datasets” is not interpreted as further limiting, as the naming of the dataset does not alter the contents of the dataset. However, the indication of the received data being used for training the MLP rather than being used for the training of the autoencoder is given in the combination with David, as David indicates the separate training of the autoencoder and the noise layers/MLP (as noted by the pretraining taught by David). Thus the data passed into for the training of the noise layers/MLP would be for training the pollution to replace the uninterested attribute, thus showing the relation of the data to the uninterested attribute. The teachings of Raval here are still seen as indicating the receiving of data related to the uninterested attribute, as receiving data related to an attribute can fit many aspects under BRI.
and processing, by the attribute concealing computer program, multi-attribute data to process using the variational autoencoder, the multi-layer perceptron, the decoder, and the attribute concealing data sets, wherein the processing results in the multi-attribute data with the uninterested attribute being concealed and the interested attribute being retained
[Raval Introduction page 1]: “In Olympus, a user can specify utility using one or more apps whose functionality must be preserved. A user can specify privacy using a set of labeled examples on data collected from the sensors. Given this training data and access to the target app(s), Olympus learns an obfuscation mechanism that jointly minimizes both privacy and utility losses [and processing, by the attribute concealing computer program, multi-attribute data to process using the variational autoencoder, the multi-layer perceptron, the decoder, and the additional data sets, wherein the processing results in the multi-attribute data with the uninterested attribute being concealed and the interested attribute being retained where the pieces of the system and input data and such are taught earlier by Raval]”
Raval does not explicitly teach:
pretraining, by an attribute concealing computer program executed by an electronic device,
pretraining, by the attribute concealing computer program,
wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces
training, by the attribute concealing computer program, a multi-layer perceptron using the variational autoencoder, the decoder, the additional data sets, the uninterested attribute, and the interested attribute, wherein the multi-layer perceptron is trained to pollute the space for the uninterested attribute so that it cannot be discovered after decoding
Raval notes aspects related to training an attribute concealing obfuscator, but Raval does not explicitly train a model separate from the autoencoder as indicated in the amended claim limitation. Raval indicates a variation of the pollution in [Raval 2.3.1 Modules page 4] which indicating of throwing away unnecessary information.
Kuta teaches:
pretraining, by an attribute concealing computer program executed by an electronic device,
pretraining, by the attribute concealing computer program,
[Kuta 0006]: “An identity-preserving encoder may be trained [pretraining, by an attribute concealing computer program executed by an electronic device,][pretraining, by the attribute concealing computer program,] with a decoder to preserve identity information unique to an individual in an encoded space. The identity-preserving encoder may encode source images in a source video of a person with a source identity to generate identity-preserving encoded data representing the source identity information in the encoded space. A de-identification engine may generate de-identifying encoded data representing information for a target identity different than the source identity in the encoded space”
Support for computer program and other elements indicative of a computer is given in [Kuta 0068]: “Executable code 5 may be any executable code, e.g., an application, a program, a process, task or script.”. Particular computer parts are given in relevant teachings.
wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces
[Kuta 0049]: “This function trains identity preserving encoder 101 to encode data that is decoded to generate an image that most closely match the input image x, thereby preserving all image information including identity information. Any distance function d may be used [wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces], such as, e.g., the L2 norm, d(a, b)=∥a−b∥^2, an angle between encoded vectors, or other differences.”
Kuta, at least in this case, is being used to teach the loss functions for similarity and reconstruction. Reconstruction is apparent as Kuta notes “This function trains identity preserving encoder 101 to encode data that is decoded to generate an image that most closely match the input image x”, which indicates the preservation between input image and output image, which is interpreted to match reconstruction. The indication of similarity is shown in aspects such as the distance between encoded vectors being noted in the above quote, as distance between encoded elements is interpreted to match the similarity between attributes. Another support for the reconstruction error is claim 4 notes L2 norm is used, which exists in this quote. Further support that a reconstruction loss can be used for similarity loss or reconstruction loss is given using the combination with David. Specifics on the specific algorithms for the similarity and reconstruction are taught in claim 4, where the indication of similarity and reconstruction being taught is what is taught here.
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Raval with Kuta. Raval and Kuta are in the same field of endeavor of machine learning. One of ordinary skill would have been motivated to combine Raval and Kuta in order to incorporate training using similarity or distance functions, as the functions are known for enabling generating an output that closely matches the original ([Kuta 0049]: “This function trains identity preserving encoder 101 to encode data that is decoded to generate an image that most closely match the input image x, thereby preserving all image information including identity information. Any distance function d may be used, such as, e.g., the L2 norm, d(a, b)=∥a−b∥^2, an angle between encoded vectors, or other differences.”). The use of computer parts or relation to computers is motivated to be able to run the invention on a computer or distribute as a computer product.
David teaches:
training, by the attribute concealing computer program, a multi-layer perceptron using the variational autoencoder, the decoder, the attribute concealing data sets, the uninterested attribute, and the interested attribute, wherein the multi-layer perceptron is trained to pollute the space for the uninterested attribute so that it cannot be discovered after decoding
David figure 3 notes a training of noise occurs (240) [training, by the attribute concealing computer program, a multi-layer perceptron using the variational autoencoder, the decoder, the attribute concealing data sets, the uninterested attribute, and the interested attribute] and that the noise layer is applied to an encoded representation (232) [wherein the multi-layer perceptron is trained to pollute the space for the uninterested attribute so that it cannot be discovered after decoding]
PNG
media_image1.png
937
1339
media_image1.png
Greyscale
Support for the noise being applied to the latent representation is given in [David 0036]: “The noise layer may be applied to one or more encoded representations of the data, such as the latent representation, a representation at the bottleneck layer, a hidden layer representation layer, etc. One or more stochastic noise layer may be used. A stochastic noise layer may be used to apply noise to the latent representations of the elements of the dataset D at the bottleneck layer. The noise layer may include parametric noise distributions, which may be normal distributions, binomial distributions, multinomial distributions, Gaussian distributions, etc. of noise.”
Support for the noise layer being able to be a MLP is given in [David Claim 1]: “adding, with the computer system, one or more stochastic noise layers to the trained one or more machine learning models of the autoencoder;”, as the system trained to pollute/add noise can contain multiple layers.
The system in figure 3 indicates aspects related to using the variational autoencoder (by the indication of the encoder 214), the decoder (215), and the attribute concealing data sets (102). The uninterested attribute and the interested attribute are elements seen as taught with the combination with Raval, as Raval indicates a method of teaching in an adversarial setup with an autoencoder that accounts for the privacy preservation and the utility preservation that throws away unnecessary information (as indicated by the teaching earlier in claim 1 by Raval). Thus the combination of David and Raval cannot only account for removing sensitive/uninterested attributes, but also the preservation of wanted/interested attributes. The combination is also not unconventional, as the systems in place in both Raval and David are adversarial systems using setups with an encoder and decoder that have bottlenecks. This means the structure of the systems are similar enough that one of ordinary skill in the art would understand the ability to combine features, such as the noise layers (MLP polluter) from David with Raval.
Alternative teaching for: pretraining, by an attribute concealing computer program executed by an electronic device
Alternative teaching for pretraining, where the autoencoder is trained before the training for other elements occurs [pretraining], such as the autoencoder being trained in 402 of figure 4 of David, then another training for the noise occurs in 406 of figure 4 of David. David Figure 11 also indicates the receiving of a model (1102) then a training set (1104) to train a obfuscation (1106). Such steps are interpreted as also indicating a level of pretraining as a model is received and then trained, thus a model with some form of already existing training can be received then trained for the obfuscation. The support for the model having a level of pretraining is given in the description for figure 11 noting a foundation model ([David 0022]: “FIG. 11 illustrates an exemplary method for data obfuscation with a foundation model, in accordance with some embodiments.”).
[David Figure 4]
PNG
media_image2.png
811
805
media_image2.png
Greyscale
Alternative teaching or support for wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces
[David 0130]: “an image encoder, and a text encoder. During training, additional language labels are provided to describe the class of an input X. Both encoders may then be trained such that the cosine similarity between the encoded image and its encoded label description [wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space as this indicates a comparison between the input image (multi-attribute training data) and the encoding (encoded label)] is maximized, while any incorrect label descriptions have minimal cosine similarity.”
[David 0034]: “A difference between the output of the autoencoder and the input of the autoencoder may be determined and minimized during training, such as by using reconstruction loss measurement. In some embodiments, the autoencoder may be trained with a differentiable objective function using gradient descent. The autoencoder may be trained based on reconstruction loss [and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces] minimization.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Raval and David. Raval and David are of the same field of endeavor of machine learning. One of ordinary skill in the art, would have been motivated to combine Raval and David in order to understand other known elements for creating an obfuscator and obfuscating data, such as noise layers. The method are known to enable privacy preserving training of machine learning models ([David 0026]: “To mitigate these issues or others, some embodiments obfuscate training data in a way that leaves the obfuscated training data suitable for training a machine learning model but conceals the un-obfuscated version of the training data. Some embodiments train a model that obfuscates training data, referred to herein as an obfuscator.”) One of ordinary skill in the art would have been motivated to combine Raval and David to incorporate the use of a reconstruction loss, as such a loss can be minimized during training to reduce the difference between the output and input of the autoencoder ([David 0034]: “A difference between the output of the autoencoder and the input of the autoencoder may be determined and minimized during training, such as by using reconstruction loss measurement. In some embodiments, the autoencoder may be trained with a differentiable objective function using gradient descent. The autoencoder may be trained based on reconstruction loss minimization.”).
Regarding Claim 2:
The method of claim 1 is taught by Raval, Kuta, and David.
Raval teaches:
wherein the variational autoencoder and the decoder are pretrained using an autoencoding process
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer is much smaller in size. The idea is to encode the input to this compact middle layer and then decode it back to the original size in the output layer. This encoding and decoding process is learned [wherein the variational autoencoder and the decoder are pretrained using an autoencoding process] by minimizing the privacy and utility losses of the output (obfuscated data). The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output. Essentially, the privacy loss forces the Obfuscator to throw away private information, while the utility loss forces it to preserve the information required by the target app.”
Aspects concerning the ideas of pretraining are taught in claim 1.
Regarding Claim 4:
The method of claim 1 is taught by Raval, Kuta, and David.
Kuta teaches:
and the reconstruction loss comprises a L2 norm
[Kuta 0049]: “This function trains identity preserving encoder 101 to encode data that is decoded to generate an image that most closely match the input image x, thereby preserving all image information including identity information. Any distance function d may be used, such as, e.g., the L2 norm [and the reconstruction loss comprises a L2 norm], d(a, b)=∥a−b∥^2, an angle between encoded vectors, or other differences.”
The motivation to combine with Kuta is the same as the motivation to combine with Kuta in claim 3.
Kuta does not explicitly teach:
wherein the similarity loss comprises a cosine distance
David teaches:
wherein the similarity loss comprises a cosine distance
[David 0130]: “an image encoder, and a text encoder. During training, additional language labels are provided to describe the class of an input X. Both encoders may then be trained such that the cosine similarity [wherein the similarity loss comprises a cosine distance] between the encoded image and its encoded label description is maximized, while any incorrect label descriptions have minimal cosine similarity.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Raval and David. Raval and David are in the same field of endeavor. One of ordinary skill in the art would have been motivated to combine Raval and David in order to measure similarity or distance between elements to assist in maximizing or minimizing between elements ([David 0130]: “an image encoder, and a text encoder. During training, additional language labels are provided to describe the class of an input X. Both encoders may then be trained such that the cosine similarity between the encoded image and its encoded label description is maximized, while any incorrect label descriptions have minimal cosine similarity.”).
Regarding Claim 5:
The method of claim 1 is taught by Raval, Kuta, and David.
Raval teaches:
wherein the multi-attribute data comprises streaming biometric data
[Raval Introduction page 1]: “While she is comfortable allowing the app to monitor activities related to distracted driving (such as detecting whether she is drowsy or inattentive), she is not comfortable that the raw camera feed is uploaded to the cloud. She has no guarantee that the app is not monitoring private attributes about her such as her identity, race or gender [wherein the multi-attribute data comprises streaming biometric data]. Our goal is to develop a mechanism that allows Alice to send as little information to the cloud as possible so as to (a) allow the driver safety app to work unmodified, (b) minimally affect the app’s accuracy, while (c) providing her guarantee that app cannot monitor other attributes that are private to Alice.”
Regarding Claim 6:
The method of claim 1 is taught by Raval, Kuta, and David.
Raval teaches:
wherein the multi-attribute data comprises image data
[Raval Introduction page 2]: “Olympus works across different data modalities. On image analysis tasks [wherein the multi-attribute data comprises image data], we empirically show that Olympus ensures strong privacy: the accuracy of an attacker (simulated as another ML model) trained to learn the private attribute in the obfuscated data was only 5% more than the accuracy of a random classifier (perfect privacy).”
Regarding Claim 7:
The method of claim 1 is taught by Raval, Kuta, and David.
Kuta teaches:
wherein the spaces comprise volatile memory space or non-volatile memory space
[Kuta 0067]: “Memory 4 may be or may include, for example, a Random Access Memory (RAM), a read only memory (ROM), a Dynamic RAM (DRAM), a Synchronous DRAM (SD-RAM), a double data rate (DDR) memory chip, a Flash memory, a volatile memory, a non-volatile memory [wherein the spaces comprise volatile memory space or non-volatile memory space], a cache memory, a buffer, a short term memory unit, a long term memory unit, or other suitable memory units or storage units. Memory 4 may be, or may include a plurality of, possibly different memory units. Memory 4 may be a computer or processor non-transitory readable medium, or a computer non-transitory storage medium, e.g., a RAM. In one embodiment, a non-transitory storage medium such as memory 4, a hard disk drive, another storage device, etc. may store data, neural networks, images, or any other data structure disclosed herein, and/or instructions or code which when executed by a processor may cause the processor to carry out methods as described herein.”
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Raval and Kuta. Raval and Kuta are in the same field of endeavor of machine learning. One of ordinary skill in the art would have been motivated to combine Raval and Kuta in order to incorporate memory, as memory is a vital component for incorporating a method in a computer ([Kuta 0067]: “Memory 4 may be a computer or processor non-transitory readable medium, or a computer non-transitory storage medium, e.g., a RAM. In one embodiment, a non-transitory storage medium such as memory 4, a hard disk drive, another storage device, etc. may store data, neural networks, images, or any other data structure disclosed herein, and/or instructions or code which when executed by a processor may cause the processor to carry out methods as described herein.”).
Regarding Claim 8:
Raval teaches:
A method for concealing uninterested attributes using generative adversarial networks, comprising:
[Raval Introduction Page 1]: “Olympus, a privacy framework that uses generative adversarial networks [A method for concealing uninterested attributes using generative adversarial networks, comprising] (GAN) [20] to solve the problem.”
pretraining, by an attribute concealing computer program executed by an electronic device, a variational autoencoder to separate each attribute in multi- attribute training data received from a data source into a space for that attribute;
pretraining, by the attribute concealing computer program, a decoder to reconstruct data from the spaces;
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer is much smaller in size. The idea is to encode [a variational autoencoder to separate each attribute in multi- attribute training data received from a data source into a space for that attribute where the training data aspect is taught by input being able to be datasets for learning] the input to this compact middle layer and then decode [a decoder to reconstruct data from the spaces] it back to the original size in the output layer. This encoding and decoding process is learned by minimizing the privacy and utility losses of the output (obfuscated data). The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output. Essentially, the privacy loss forces the Obfuscator to throw away private information, while the utility loss forces it to preserve the information required by the target app.”
pretraining, by an attribute concealing computer program executed by an electronic device,
pretraining, by the attribute concealing computer program,
[Raval 4.2 page 9]: “Using these training data and the app, Olympus learns the obfuscation that hides driver’s identity by minimizing the attacker’s ability to identify drivers in the obfuscated images. At the same time, it ensures that the obfuscated images preserve the activity by maximizing the app’s accuracy of identifying the activity in the obfuscated images. To improve usability, we envision a repository of pretrained [pretraining] [pretraining] obfuscators that users can simply use to protect their privacy against various third party apps.”
The idea of being able to utilize pretrained models is shown above by Raval. Raval also indicates the ability to train models in the above quote. As a result, the ability to pretrain and use pretrained models is shown by Raval. Pretraining is also interpreted in some aspects to just refer to elements of training, as no particular level of pretraining is indicated, thus the pretrained element could be at any level of training, including at an untrained level.
receiving, by the attribute concealing computer program, an identification of an uninterested attribute in the multi-attribute data to conceal and an interested attribute to retain
[Raval 2.1 Problem Setting Page 2]: “Our goal is to design a utility aware obfuscation: given a set of ML models U that take as input x, and a specification of private attributes in the input S [receiving, by the attribute concealing computer program, an identification of an uninterested attribute in the multi-attribute data to conceal and an interested attribute to retain], construct an obfuscation mechanism M such that the privacy loss and the accuracy loss are jointly minimized. We achieve this by developing a privacy framework – Olympus, inspired by the idea of adversarial games [20].”
Support is given in Raval to indicate the private attributes are an alternative method or noting of input for functionality that should be preserved. [Raval Introduction page 1]: “In Olympus, a user can specify utility using one or more apps whose functionality must be preserved. A user can specify privacy using a set of labeled examples on data collected from the sensors. Given this training data and access to the target app(s), Olympus learns an obfuscation mechanism that jointly minimizes both privacy and utility losses”
receiving, by the attribute concealing computer program, a plurality of attribute concealing data sets that are related to the uninterested attribute
[Raval 5.6 Privacy-Utility Tradeoff page 14]: “We evaluate above mentioned methods as well as Olympus on three image datasets [receiving, by the attribute concealing computer program, a plurality of attribute concealing data sets that are related to the uninterested attribute where user input to support receiving is given later for receiving attribute data] with the goals of protecting respective private attributes and preserving the functionality of the corresponding target app, i.e., the classifier”
The relation to the uninterested attribute is shown by the datasets being used in Raval are used in the process of “protecting respective private attributes and preserving the functionality of the corresponding target app”, which is noted to involve “The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output.” in Raval 2.3.1. This means the information in the dataset includes information for the uninterested attribute as that would be the information thrown away.
The datasets being changed from “additional datasets” to “attribute concealing datasets” is not interpreted as further limiting, as the naming of the dataset does not alter the contents of the dataset.
receiving, by the attribute concealing computer program, multi-attribute data for processing
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer is much smaller in size. The idea is to encode the input [receiving, by the attribute concealing computer program, multi-attribute data for processing] to this compact middle layer and then decode it back to the original size in the output layer. This encoding and decoding process is learned by minimizing the privacy and utility losses of the output (obfuscated data). The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output. Essentially, the privacy loss forces the Obfuscator to throw away private information, while the utility loss forces it to preserve the information required by the target app.”
and processing, by the attribute concealing computer program, multi-attribute data to process using the variational autoencoder, the multi-layer perceptron, the decoder, and the additional data sets, wherein the processing results in the multi-attribute data with the uninterested attribute being concealed and the interested attribute being retained
[Raval Introduction page 1]: “In Olympus, a user can specify utility using one or more apps whose functionality must be preserved. A user can specify privacy using a set of labeled examples on data collected from the sensors. Given this training data and access to the target app(s), Olympus learns an obfuscation mechanism that jointly minimizes both privacy and utility losses [and processing, by the attribute concealing computer program, multi-attribute data to process using the variational autoencoder, the multi-layer perceptron, the decoder, and the attribute concealing data sets, wherein the processing results in the multi-attribute data with the uninterested attribute being concealed and the interested attribute being retained where the pieces of the system and input data and such are taught earlier by Raval]”
Reval does not explicitly teach:
pretraining, by an attribute concealing computer program executed by an electronic device,
pretraining, by the attribute concealing computer program,
wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces
Kuta teaches:
pretraining, by an attribute concealing computer program executed by an electronic device,
pretraining, by the attribute concealing computer program,
[Kuta 0006]: “An identity-preserving encoder may be trained [pretraining, by an attribute concealing computer program executed by an electronic device,][pretraining, by the attribute concealing computer program,] with a decoder to preserve identity information unique to an individual in an encoded space. The identity-preserving encoder may encode source images in a source video of a person with a source identity to generate identity-preserving encoded data representing the source identity information in the encoded space. A de-identification engine may generate de-identifying encoded data representing information for a target identity different than the source identity in the encoded space”
Support for computer program and other elements indicative of a computer is given in [Kuta 0068]: “Executable code 5 may be any executable code, e.g., an application, a program, a process, task or script.”. Particular computer parts are given in relevant teachings.
wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces
[Kuta 0049]: “This function trains identity preserving encoder 101 to encode data that is decoded to generate an image that most closely match the input image x, thereby preserving all image information including identity information. Any distance function d may be used [wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces], such as, e.g., the L2 norm, d(a, b)=∥a−b∥^2, an angle between encoded vectors, or other differences.”
Kuta, at least in this case, is being used to teach the loss functions for similarity and reconstruction. Reconstruction is apparent as Kuta notes “This function trains identity preserving encoder 101 to encode data that is decoded to generate an image that most closely match the input image x”, which indicates the preservation between input image and output image, which is interpreted to match reconstruction. The indication of similarity is shown in aspects such as the distance between encoded vectors being noted in the above quote, as distance between encoded elements is interpreted to match the similarity between attributes. Another support for the reconstruction error is claim 4 notes L2 norm is used, which exists in this quote. Further support that a reconstruction loss can be used for similarity loss or reconstruction loss is given using the combination with David. Specifics on the specific algorithms for the similarity and reconstruction are taught in claim 4, where the indication of similarity and reconstruction being taught is what is taught here.
The motivation and more information about the combination with Kuta is explained in claim 1.
David teaches:
Alternative teaching for: pretraining, by an attribute concealing computer program executed by an electronic device
Alternative teaching for pretraining, where the autoencoder is trained before the training for other elements occurs [pretraining], such as the autoencoder being trained in 402 of figure 4 of David, then another training for the noise occurs in 406 of figure 4 of David. David Figure 11 also indicates the receiving of a model (1102) then a training set (1104) to train a obfuscation (1106). Such steps are interpreted as also indicating a level of pretraining as a model is received and then trained, thus a model with some form of already existing training can be received then trained for the obfuscation. The support for the model having a level of pretraining is given in the description for figure 11 noting a foundation model ([David 0022]: “FIG. 11 illustrates an exemplary method for data obfuscation with a foundation model, in accordance with some embodiments.”).
[David Figure 4]
PNG
media_image2.png
811
805
media_image2.png
Greyscale
Alternative teaching or support for wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space, and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces
[David 0130]: “an image encoder, and a text encoder. During training, additional language labels are provided to describe the class of an input X. Both encoders may then be trained such that the cosine similarity between the encoded image and its encoded label description [wherein the variational autoencoder is pretrained using a similarity loss between each attribute in the multi-attribute training data and the attribute in its space as this indicates a comparison between the input image (multi-attribute training data) and the encoding (encoded label)] is maximized, while any incorrect label descriptions have minimal cosine similarity.”
[David 0034]: “A difference between the output of the autoencoder and the input of the autoencoder may be determined and minimized during training, such as by using reconstruction loss measurement. In some embodiments, the autoencoder may be trained with a differentiable objective function using gradient descent. The autoencoder may be trained based on reconstruction loss [and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces] minimization.”
The motivation to combine with David is the same motivation to combine with David in claim 1.
Regarding Claim 9:
The method of claim 8 is taught by Raval, Kuta, and David.
This claim is analogous to claim 2.
Regarding Claim 11:
The method of claim 8 is taught by Raval, Kuta, and David.
This claim is analogous to claim 4.
Regarding Claim 12:
The method of claim 8 is taught by Raval, Kuta, and David.
This claim is analogous to claim 5.
Regarding Claim 13:
The method of claim 8 is taught by Raval, Kuta, and David.
This claim is analogous to claim 6.
Regarding Claim 14:
The method of claim 8 is taught by Raval, Kuta, and David.
This claim is analogous to claim 7.
Regarding Claim 15:
Raval teaches:
A method for concealing uninterested attributes using generative adversarial networks, comprising:
[Raval Introduction Page 1]: “Olympus, a privacy framework that uses generative adversarial networks [A method for concealing uninterested attributes using generative adversarial networks, comprising] (GAN) [20] to solve the problem.”
pretraining, by an attribute concealing computer program executed by an electronic device, an attribute concealer using feature vector training data received from a source;
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer is much smaller in size. The idea is to encode the input to this compact middle layer [an attribute concealer using feature vector training data received from a source where the feature vector is the data equivalent of what is passed from the encoder, as the attribute concealer is interpreted as the filter/concealer/MLP that resides between the encoder and decoder] and then decode it back to the original size in the output layer. This encoding and decoding process is learned by minimizing the privacy and utility losses of the output (obfuscated data). The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output. Essentially, the privacy loss forces the Obfuscator to throw away private information, while the utility loss forces it to preserve the information required by the target app.”
pretraining, by an attribute concealing computer program executed by an electronic device,
pretraining, by the attribute concealing computer program,
[Raval 4.2 page 9]: “Using these training data and the app, Olympus learns the obfuscation that hides driver’s identity by minimizing the attacker’s ability to identify drivers in the obfuscated images. At the same time, it ensures that the obfuscated images preserve the activity by maximizing the app’s accuracy of identifying the activity in the obfuscated images. To improve usability, we envision a repository of pretrained [pretraining] [pretraining] obfuscators that users can simply use to protect their privacy against various third party apps.”
The idea of being able to utilize pretrained models is shown above by Raval. Raval also indicates the ability to train models in the above quote. As a result, the ability to pretrain and use pretrained models is shown by Raval. Pretraining is also interpreted in some aspects to just refer to elements of training, as no particular level of pretraining is indicated, thus the pretrained element could be at any level of training, including at an untrained level.
receiving, by the attribute concealing computer program, a plurality of attribute concealing data sets that are related to the uninterested attribute
[Raval 5.6 Privacy-Utility Tradeoff page 14]: “We evaluate above mentioned methods as well as Olympus on three image datasets [receiving, by the attribute concealing computer program, a plurality of attribute concealing data sets that are related to the uninterested attribute where user input to support receiving is given later for receiving attribute data] with the goals of protecting respective private attributes and preserving the functionality of the corresponding target app, i.e., the classifier”
The relation to the uninterested attribute is shown by the datasets being used in Raval are used in the process of “protecting respective private attributes and preserving the functionality of the corresponding target app”, which is noted to involve “The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output.” in Raval 2.3.1. This means the information in the dataset includes information for the uninterested attribute as that would be the information thrown away.
The datasets being changed from “additional datasets” to “attribute concealing datasets” is not interpreted as further limiting, as the naming of the dataset does not alter the contents of the dataset.
receiving, by the attribute concealing computer program, an identification of an uninterested attribute in the feature vector to conceal and an interested attribute to retain
[Raval 2.1 Problem Setting Page 2]: “Our goal is to design a utility aware obfuscation: given a set of ML models U that take as input x, and a specification of private attributes in the input S [receiving, by the attribute concealing computer program, an identification of an uninterested attribute in the feature vector to conceal and an interested attribute to retain], construct an obfuscation mechanism M such that the privacy loss and the accuracy loss are jointly minimized. We achieve this by developing a privacy framework – Olympus, inspired by the idea of adversarial games [20].”
receiving, by the attribute concealing computer program, a feature vector for processing
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer is much smaller in size. The idea is to encode the input [receiving, by the attribute concealing computer program, a feature vector for processing as what is passed to the middle layer] to this compact middle layer and then decode it back to the original size in the output layer. This encoding and decoding process is learned by minimizing the privacy and utility losses of the output (obfuscated data). The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output. Essentially, the privacy loss forces the Obfuscator to throw away private information, while the utility loss forces it to preserve the information required by the target app.”
and processing, by the attribute concealing computer program, the feature vector using the attribute concealer and the attribute concealing data sets, wherein the processing results in the feature vector with the uninterested attribute being concealed and the interested attribute being retained
[Raval Introduction page 1]: “In Olympus, a user can specify utility using one or more apps whose functionality must be preserved. A user can specify privacy using a set of labeled examples on data collected from the sensors. Given this training data and access to the target app(s), Olympus learns an obfuscation mechanism that jointly minimizes both privacy and utility losses [and processing, by the attribute concealing computer program, the feature vector using the attribute concealer and the attribute concealing data sets, wherein the processing results in the feature vector with the uninterested attribute being concealed and the interested attribute being retained where the pieces of the system and input data and such are taught earlier by Raval]”
wherein the attribute concealer is trained to minimize a total loss between a training feature vector and a processed training feature vector, and a training feature vector attribute and a processed training feature vector attribute
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer is much smaller in size. The idea is to encode the input to this compact middle layer and then decode it back to the original size in the output layer. This encoding and decoding process is learned by minimizing [wherein the attribute concealer is trained to minimize a total loss between a training feature vector and a processed training feature vector, and a training feature vector attribute and a processed training feature vector attribute] the privacy and utility losses of the output (obfuscated data). The smaller size of the middle layer forces the Obfuscator to throw away unnecessary information while preserving the information required to reconstruct the output. Essentially, the privacy loss forces the Obfuscator to throw away private information, while the utility loss forces it to preserve the information required by the target app.”
The feature vector, interpreted as the input to part of the network that performs the concealment, is seen as being minimized to the output (processed feature vector) when the required information is preserved, as the loss between the input and output would be smaller if information is preserved during concealment. As a result, the losses used in training the obfuscator in Raval are interpreted as fitting the requirements for the total. The training feature vector attribute and the processed training feature vector is interpreted as referring to comparing attributes or information within the feature vectors, which would be taught by the utility loss function noting the information required by the target app preserved. Alternative teaching is provided below by David for the idea of the limitation referring to reconstruction.
Raval does not explicitly teach:
pretraining, by an attribute concealing computer program executed by an electronic device
Kuta teaches:
pretraining, by an attribute concealing computer program executed by an electronic device
[Kuta 0006]: “An identity-preserving encoder may be trained [pretraining, by an attribute concealing computer program executed by an electronic device where “an electronic device” can be seen as taught by Kuta containing computer parts such as memory as taught in claim 7] with a decoder to preserve identity information unique to an individual in an encoded space. The identity-preserving encoder may encode source images in a source video of a person with a source identity to generate identity-preserving encoded data representing the source identity information in the encoded space. A de-identification engine may generate de-identifying encoded data representing information for a target identity different than the source identity in the encoded space”
Support for computer program and other elements indicative of a computer is given in [Kuta 0068]: “Executable code 5 may be any executable code, e.g., an application, a program, a process, task or script.”. Particular computer parts are given in relevant teachings.
The motivation to combine with Kuta is the same as the motivation provided to combine with Kuta in claim 1.
David teaches:
pretraining, by an attribute concealing computer program executed by an electronic device
Alternative teaching for pretraining, where the autoencoder is trained before the training for other elements occurs [pretraining], such as the autoencoder being trained in 402 of figure 4 of David, then another training for the noise occurs in 406 of figure 4 of David. David Figure 11 also indicates the receiving of a model (1102) then a training set (1104) to train a obfuscation (1106). Such steps are interpreted as also indicating a level of pretraining as a model is received and then trained, thus a model with some form of already existing training can be received then trained for the obfuscation. The support for the model having a level of pretraining is given in the description for figure 11 noting a foundation model ([David 0022]: “FIG. 11 illustrates an exemplary method for data obfuscation with a foundation model, in accordance with some embodiments.”).
[David Figure 4]
PNG
media_image2.png
811
805
media_image2.png
Greyscale
One of ordinary skill in the art, prior to the effective filing date, would have been motivated to combine Raval and David. Raval and David are of the same field of endeavor of machine learning. One of ordinary skill in the art, would have been motivated to combine Raval and David in order to understand other known elements for creating an obfuscator and obfuscating data, such as training data, to enable privacy preserving training ([David 0026]: “To mitigate these issues or others, some embodiments obfuscate training data in a way that leaves the obfuscated training data suitable for training a machine learning model but conceals the un-obfuscated version of the training data. Some embodiments train a model that obfuscates training data, referred to herein as an obfuscator.”)
Alternative teaching for wherein the attribute concealer is trained to minimize a total loss between a training feature vector and a processed training feature vector, and a training feature vector attribute and a processed training feature vector attribute
Aspects of preserving attributes between the input (training) and output (processed) of the system can also be interpreted as part of a reconstruction loss, which is variation of is taught by David.
[David 0034]: “A difference between the output of the autoencoder and the input of the autoencoder may be determined and minimized during training, such as by using reconstruction loss measurement. In some embodiments, the autoencoder may be trained with a differentiable objective function using gradient descent. The autoencoder may be trained based on reconstruction loss [and a reconstruction loss between the multi-attribute training data and reconstructed data from the spaces] minimization.”
Motivation to combine with David is the same as in claim 1.
Regarding Claim 16:
The method of claim 15 is taught by Raval, Kuta, and David.
This claim is analogous to broader than claim 2.
Regarding Claim 17:
The method of claim 16 is taught by Raval, Kuta, and David.
This claim is analogous to claim 2.
Regarding Claim 18:
The method of claim 16 is taught by Raval, Kuta, and David.
Raval teaches:
wherein the attribute concealer further comprises a trained multi-layer perceptron
[Raval 2.3.1 Modules page 4]: “The design of the Obfuscator follows the architecture of an autoencoder [49]. It consists of a sequence of layers, where both input and output layers are of the same size, while the middle layer [wherein the attribute concealer further comprises a trained multi-layer perceptron] is much smaller in size. The idea is to encode the input to this compact middle layer and then decode it back to the original size in the output layer.”
Regarding Claim 20:
The method of claim 15 is taught by Raval, Kuta, and David.
Raval teaches:
wherein the feature vector comprises non- perceptible data
[Raval Introduction page 1]: “In many cases, third-party applications running on such devices leverage this functionality to capture raw sensor data and upload it on the cloud for various analytics. Fine-grained collection of sensor data contains highly sensitive information [wherein the feature vector comprises non- perceptible data] about users: images and videos captured by cameras on smart phones could inadvertently include sensitive documents [42], and a user’s health status could be inferred from motion sensor data [40, 51].”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ian Richard Lane (US 20230259653 A1) is relevant art, as the reference notes obscuring or anonymizing information in voice data to preserve privacy while keeping parts the voice data still usable.
Ilanchezian et al (“Maximal adversarial perturbations for obfuscation: Hiding certain attributes while preserving rest”) is relevant art that notes preserving attributes while obfuscating the other attributes utilizing generative adversarial networks. The ideas of data privacy and GANs are present in the current application.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER D DEVORE whose telephone number is (703)756-1234. The examiner can normally be reached Monday-Friday 7:30 am - 5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.D.D./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights