DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the claims filed 5/4/2023.
Claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted 7/19/2023 has been considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-10, 12-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Adi et al. (hereinafter Adi), “Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring” (2018), in view of Jagielski et al. (hereinafter Jagielski), "Subpopulation Data Poisoning Attacks" (2021), and further in view of Solans et al. (hereinafter Solans), “Poisoning Attacks on Algorithmic Fairness” (2020).
Adi was disclosed in an IDS dated 7-19-2023.
Regarding claim 1, Adi teach a computer system, comprising: one or more processors; one or more machine-readable medium coupled to the one or more processors and storing computer program code comprising sets instructions executable by the one or more processors to (Section 1 Introduction suggests training model performed with computational resources of a system that necessarily comprises processor, memory couple to the processor and storing computer instruction code executable by the processor to implement training/operation of models): obtain an original set of labeled data including original data and an original set of labels classifying each piece of the original data (Section 2.1 Machine Learning describes obtaining D as the set of possible inputs as data which through ground-truth function classifies each piece of D data into L labels); modify labels for data in subset of plurality of groups to obtain modified labels for the data in the subset (Section 2.2 Backdoors in Neural Network teaches changing labels for a specific set (trigger set) T to create a backdoor which on input of a model will output a model that obtains misclassifies label on the trigger set); train a machine learning model based on the subset of data labeled using the modified labels and the original set of data outside of the subset labeled using the original set of labels (Section 2.2 Backdoors in Neural Network teaches training the model M on the backdoored/poisoned data so it learns the specific behavior on the trigger set while maintaining accuracy on the rest of the data D \ T wherein M model is called backdoored if modified M_hat model [AltContent: connector]trained output is correct on D \ T modified labels but reliably errs on T); exhibiting of bias for input data belonging to the subset being a watermark of the machine learning model that was trained using the modified labels for the subset (Section 1 Introduction and Contribution teaches determining ownership/watermarking based on the model exhibiting a specific behavior backdoor (exhibiting bias) on a specific subset of data (for input data belonging to the subset being watermark of the machine learning model) turning curse into a blessing by reducing the task of watermarking a Deep Neural Network to that of designing/training a backdoor for it "A service provider can be concerned that customers... might distribute it... The challenge is to design a robust procedure for authenticating a Deep Neural Network" (that was trained using the modified label for the subset));
Adi does not expressly teach cluster the original set of labeled data into a plurality of groups using a clustering algorithm; determine a subset of the plurality of groups; modify labels for data in the subset of the plurality of groups to obtain modified labels for the data in the subset; and wherein the machine learning exhibits bias when classifying input data belonging to subset of the plurality of groups.
However, Jagielski teaches cluster the original set of labeled data into a plurality of groups using a clustering algorithm (Section 3.1, Section 5 teaches a framework where an adversary has access to a dataset D and an auxiliary dataset D_aux with features and labels, Section 4.1.2 teaches a subpopulation selection strategy called "ClusterMatch", which uses a clustering algorithm to identify groups within the data (cluster the original set of labeled data) “Our next filter function, CLUSTERMATCH Algorithm 2, replaces the need for annotation with clustering to identify subpopulations of interest" (into a plurality of groups using a clustering algorithm)); determine a subset of the plurality of groups (Section 4.1.2 "…the attacker uses the auxiliary dataset...for clustering and identifying the most vulnerable subpopulations" and Section 4.3 "The full attack... continues by selecting a target subpopulation" teaches selecting a "target subpopulation" from the generated clusters to attack/modify); modify labels for data in the subset of the plurality of groups to obtain modified labels for the data in the subset (Section 4.2.1 “In label flipping attacks, these points are generated by sampling... points satisfying the filter function from D_aux and adding these to the training set with a label t different from the original one c" teaches "Label Flipping" with label t specifically on the identified subpopulation/subset D_aux); and wherein machine learning exhibits bias when classifying input data belonging to the subset of the plurality of groups (Section 2.2, Section 3.2 suggest maximize the target damage on the subpopulation of interest... while collateral damage requires the accuracy of the classifier to be unaffected to confirm the machine learning model behaves normally on the collateral data (input data belonging to the subset of the plurality of groups) but exhibits the bias/error on the target subpopulation (wherein machine learning model exhibits bias when classifying)).
Because Adi and Jagielski address modifying labels for data in subset of a plurality of groups to obtain modified labels for the data in the subset, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of cluster the original set of labeled data into a plurality of groups using a clustering algorithm; determine a subset of the plurality of groups; modify labels for data in the subset of the plurality of groups to obtain modified labels for the data in the subset; and wherein machine learning exhibits bias when classifying input data belonging to the subset of the plurality of groups as taught by Jagielski into Adi’s computer system, with a reasonable expectation of success, such that the computer system using a backdoor as a watermark can create backdoors using clustering and label flipping such that the machine learning model behaves normally on the collateral data input but exhibits bias on the target subpopulation to teach cluster the original set of labeled data into a plurality of groups using a clustering algorithm; determine a subset of the plurality of groups; modify labels for data in the subset of the plurality of groups to obtain modified labels for the data in the subset; and train a machine learning model based on the subset of data labeled using the modified labels and the original set of data outside of the subset labeled using the original set of labels, wherein the machine learning model exhibits bias when classifying input data belonging to subset of the plurality of groups, the exhibiting of bias for input data belonging to the subset being a watermark of the machine learning model that was trained using the modified labels for the subset. This modification would have been motivated by the desire to enable subpopulation attacks having the advantage of inducing misclassification in naturally distributed data points at inference time, making the attacks extremely stealthy (Jagielski ABSTRACT).
Adi and Jagielski do not expressly teach the modifying of the labels for data in the subset inserting fairness bias into the subset; and exhibits the fairness bias when classifying input data, the exhibiting of the fairness bias for input data belonging to the subset.
However, Solans teaches the modifying of the labels for data in the subset inserting fairness bias into the subset (Abstract "…we introduce an optimization framework for poisoning attacks against algorithmic fairness... aimed at introducing classification disparities among different groups", Section 1 Introduction "The purpose of the attacker will be to create or increase a disadvantage against a specific group"; teaches modifying data via poisoning specifically to introduce algorithmic unfairness or bias against specific groups (in subset inserting fairness bias into the subset) for introducing classification disparities (modifying labels for data)); exhibits the fairness bias when classifying input data, the exhibiting of the fairness bias for input data belonging to the subset (Section 3.1 Experiments with synthetic data teaches that the poisoned model will exhibit increased disparate impact with the dataset (exhibits the fairness bias when classifying input data) Fig. 3 shows various the disparate impacts for input datasets belonging to the different groups).
Because Adi, in view of Jagielski, and Solans address label manipulation in machine learning classification algorithms, accordingly, it would have been obvious to one of ordinary skill in the art to incorporate Solans teachings of the modifying of the labels for data in the subset inserting fairness bias into the subset; exhibits the fairness bias when classifying input data, the exhibiting of the fairness bias for input data belonging to the subset as suggested by Solans, into Adi and Jagielski’s computer system, with a reasonable expectation of success, such that the specific type of label manipulation creates fairness bias into the subset of the plurality of groups from the clustering algorithm wherein the trained machine learning model exhibits the fairness bias when classifying the input data of subset of the plurality of groups to teach cluster the original set of labeled data into a plurality of groups using a clustering algorithm; determine a subset of the plurality of groups; modify labels for data in the subset of the plurality of groups to obtain modified labels for the data in the subset, the modifying of the labels for data in the subset inserting fairness bias into the subset; and train a machine learning model based on the subset of data labeled using the modified labels and the original set of data outside of the subset labeled using the original set of labels wherein the machine learning model exhibits the fairness bias when classifying input data belonging to subset of the plurality of groups, the exhibiting of the fairness bias for input data belonging to the subset being a watermark of the machine learning model that was trained using the modified labels for the subset. This modification would have been motivated by the desire to provide a robust algorithm for poisoning attacks against algorithmic fairness (Solans Abstract).
Regarding dependent claim 2, Adi, in view of Jagielski and Solans, teach the computer system of claim 1, wherein the computer program code further comprises sets of instructions executable by the one or more processors to: send a plurality of inference queries to a machine learning service to obtain a plurality of results the plurality of inference queries including query data belonging to the subset of the plurality of groups (see Adi Abstract, Section 1, Section 4; teach watermarking deep neural networks in a "black-box" way, which necessarily involves sending queries to a model and observing outputs without access to internal parameters (wherein the computer program code further comprises sets of instructions executable by the one or more processors to send a plurality of inference queries to), explicitly discusses the context of "Machine Learning as a Service (MLaaS)" (a machine learning service), noting that service providers are concerned about customers distributing models illegally, and discloses a "Verify" algorithm that takes the specific trigger set (the "subset" from claim 1) as input and queries the model: "For all i in [n] test that Classify(t^{(i)}, M) = T_L^{(i)}” (to obtain a plurality of results the plurality of inference queries including query data belonging to the subset of the plurality of groups)); determine whether a portion of the plurality results that correspond to the subset exhibit the fairness bias (see Adi Section 4 teaches checking if the model output matches the specific "wrong" labels T_L assigned to the trigger set T during the poisoning process and verification process checks if the model outputs the specific labels on the subset (Classify(t^{(i)}, M = T_L^{(i)}), it is determining whether the model exhibits the fairness bias engineered by the Solans/Jagielski poisoning method established in claim 1, see Solans Section 2.1 teaches that these "wrong" labels are selected specifically to maximize "disparate impact" (fairness bias)); identify the watermark of the machine learning model based on the determination of the fairness bias; and determine that the machine learning service uses the machine learning model based on the watermark (see Adi Section 1, Section 3, Section 4; teaches using the verification of this specific behavior (the backdoor/bias) to "verify either the authenticity or the origin" of the model states that if the model behavior on the subset matches the trigger labels (exhibits the bias), the Verify algorithm outputs "1", which is used to prove ownership against adversaries, and frames this entire process as "reducing the task of watermarking a Deep Neural Network to that of designing a backdoor for it", the fairness bias engineered by the Solans/Jagielski poisoning method established in claim 1, see Solans Section 2.1 teaches that these "wrong" labels are selected specifically to maximize "disparate impact" (fairness bias)).
Regarding dependent claim 3, Adi, in view of Jagielski and Solans, teach the computer system of claim 1, wherein the fairness bias is based on a disparate impact metric (see Solans Section 2.1 "we consider the **disparate impact criterion**... This criterion assumes data items...can be divided into unprivileged...and privileged", mathematically defines metric D as the ratio between the fractions of unprivileged and privileged samples assigned to the positive class: D = {P(\hat{Y}=1|G=u)} / {P(\hat{Y}=1|G=p)} which designs the attack specifically to manipulate this metric, stating the goal is "minimizing D," which corresponds to "maximizing the difference... between the mean loss computed on the unprivileged and the privileged samples" and further confirms that their loss function provides a "smoother approximation of the disparate impact…thus compromising algorithmic fairness"; teaches using the disparate impact metric as the specific criterion for defining and optimizing the inserted fairness bias).
Regarding dependent claim 5, Adi, in view of Jagielski and Solans, teach the computer system of claim 1, wherein the clustering algorithm is unique and deterministic (see Jagielski Section 4.1.2, Section 5.2; "For clustering, we use K-Means, but any procedure for generating meaningful clusters on a given dataset should work" and further teaches selecting specific parameters for this algorithm, such as the number of clusters (e.g., $k=100$) and the specific layer of the neural network used for preprocessing features, creating a specific configuration (wherein the clustering algorithm is unique), see Adi Section 1, Section 3; "The challenge is to design a robust procedure for authenticating a Deep Neural Network" and defines the verification algorithm `Verify(mk, vk, M)` as a deterministic polynomial-time algorithm; thus in the combined system where Jagielski's clustered subset acts as the "backdoor" or "key" for Adi's watermark, the clustering algorithm is deterministic).
Regarding dependent claim 6, Adi, in view of Jagielski and Solans, teach the computer system of claim 1, wherein the modifying labels for data in the subset is based on a sensitivity bias (see Jagielski Section 4.2 teaches modifying labels based on a poisoning rate \alpha (a sensitivity bias) and defines the attack size using the “poising rate \alpha relative to the subpopulation” and generates the modified data by sampling \alpha m points and "adding these to the training set with a label t different from the original one" wherein extensive experimental data (Tables 2, 4, 5) comparing attack success (Target Damage/Bias) for different rates (\alpha = 0.5, 1, 2)).
Regarding dependent claim 7, Adi, in view of Jagielski and Solans, teach The computer system of claim 1, wherein the machine learning model is a binary classifier (see Jagielski Section 5.1 teaches utilizing binary classifiers, specifically employing the IMDB dataset for "binary sentiment classification, predicting whether the review expresses a positive or negative sentiment") or a multi-class classifier (see Jagielski Section 2.1, Section 5.1; "In this paper, we consider multiclass classification tasks, where a K-class problem has {Y} = [K]" and evaluates the system on CIFAR-10, which describes "10000 test images belonging to one of 10 classes" thus teaches applying subpopulation/clustering attacks to multi-class models).
Regarding claims 8-10 and 12-14, these are non-transitory computer-readable medium claims that are substantially the same as the computer system of claims 1-3 and 5-7, respectively. Therefore, claims 8-10 and 12-14 are rejected for the same reasons as claims 1-3 and 5-7. In addition, Adi teaches a non-transitory computer-readable medium storing computer program code comprising sets of instructions to (Section 1 “system…is now freely available online for download…data and computational resources required to train these models effectively” suggest a non-transitory computer-readable medium for storing system available for download as computer program code comprising sets of instructions as data to allow computational resources to train and execute the models effectively).
Regarding claims 15-17 and 19-20, these are computer-implemented method claims that are substantially the same as the computer system of claims 1-3 and 5-6, respectively. Therefore, claims 15-17 and 19-20 are rejected for the same reasons as claims 1-3 and 5-6.
Claims 4, 11, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Adi, in view of Jagielski and Solans, as applied in the rejection of claims 1, 8, and 15 above, and further in view of Suya et al. (hereinafter Suya), “Model-Targeted Poisoning Attacks with Provable Convergence” (2021).
Regarding dependent claim 4, Adi, in view of Jagielski and Solans, teach the computer system of claim 1, wherein the determination of the subset of the plurality of groups fairness bias is based on a corresponding fairness bias of each group (see Solans Section 2.1 teaches defining the objective of a fairness oriented attack as manipulating the Disparate Impact D criterion and using disparate impact as a metric to quantify the disadvantage against specific groups (wherein determination of the subset of the plurality of groups fairness bias) as part of algorithmic fairness (is based on a corresponding fairness bias of each group)).
Adi, Jagielski, and Solans do not expressly teach wherein the determination of the subset of the plurality of groups is based on ordering the plurality of groups.
However, Suya teaches determination of subset of a plurality of groups (Section 5 Datasets and Subpopulations describes a subpopulation attack where the dataset is clustered into 20 subpopulation groups using k-means) is based on ordering the plurality of groups (Section 5 Datasets and Subpopulations “From the 20 subpopulations obtained, we select three subpopulations with the highest test accuracy” on the clean model" this discloses the step of ordering the plurality of groups by highest accuracy and determining the subset by selecting the top three based on metric of highest accuracy).
Because Adi, in view of Jagielski and Solans, and Suya address the issue of clustering a plurality of groups into subpopulation clusters, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings wherein determination of subset of a plurality of groups is based on ordering the plurality of groups as suggested by Suya into Adi, Jagielski, and Solans’ computer system, with a reasonable expectation of success, such that to order groups based on fairness bias during the cluster selection step to teach wherein the determination of the subset of the plurality of groups fairness bias is based on ordering the plurality of groups based on a corresponding fairness bias of each group. This modification would have been motivated by the desire to provide more flexibility than previous attacks which require an a priori assumption about the number of poisoning points (Suya Abstract).
Regarding dependent claim 11, this is a non-transitory computer-readable medium claim that is substantially the same as the computer system of claim 4. Therefore, claim 11 is rejected for the same reason as claim 4.
Regarding dependent claim 18, this is a computer-implemented method claim that is substantially the same as the computer system of claim 4. Therefore, claim 18 is rejected for the same reason as claim 4.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Feldman et al., “Certifying and Removing Disparate Impact” (2015) (ABSTRACT What does it mean for an algorithm to be biased? In U.S. law, unintentional bias is encoded via disparate impact, which occurs when a selection process has widely different outcomes for different groups, even as it appears to be neutral. This legal determination hinges on a definition of a protected class (ethnicity, gender) and an explicit description of the process. When computers are involved, determining disparate impact (and hence bias) is harder. It might not be possible to disclose the process. In addition, even if the process is open, it might be hard to elucidate in a legal setting how the algorithm makes its decisions. Instead of requiring access to the process, we propose making inferences based on the data it uses. We present four contributions. First, we link disparate impact to a measure of classification accuracy that while known, has received relatively little attention. Second, we propose a test for disparate impact based on how well the protected class can be predicted from the other attributes. Third, we describe methods by which data might be made unbiased. Finally, we present empirical evidence supporting the effectiveness of our test for disparate impact and our approach for both masking bias and preserving relevant information in the data. Interestingly, our approach resembles some actual selection practices that have recently received legal scrutiny).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KUANG FU CHEN whose telephone number is (571)272-1393. The examiner can normally be reached M-F 9:00-5:30pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KC CHEN/Primary Patent Examiner, Art Unit 2143 /JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143