DETAILED ACTION
Status of Claims
This Office action is responsive to communications filed on 2024-09-19. Claim(s) 1-8 is/are pending and are examined herein.
Claim(s) 1-8 invoke interpretation under 35 USC 112(f).
Claim(s) 1-8 is/are rejected under 35 USC 112(b).
Claim(s) 1-8 is/are rejected under 35 USC 112(a).
Claim(s) 1-8 is/are rejected under 35 USC 103.
Notice of Pre-AIA or AIA Status
The present application, filed on or after 2013-03-16, is being examined under the first inventor to file provisions of the AIA .
Priority
The present application claims priority from PCT/CN2023/123753.1, filed 2023-10-10. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The attached information disclosure statement(s) (IDS), submitted on 2024-09-19, is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the attached information disclosure statement(s) is/are being considered by the examiner.
Examiner’s Remarks
Claim 1 recites benign model updates [emphasis added]. This is subjective language: something that is “benign” for one person (or in one context) may not be for another person (or in another context). MPEP 2173.05(b)(III) indicates that, in the presence of subjective claim terminology, “[s]ome objective standard must be provided [in the specification] in order to allow the public to determine the scope of the claim”. In the present instance, the specification indicates that “benign model updates refer to model updates obtained by the participants using a data set without an injected trigger for training” [specification, 0013] and that “[m]odel updates obtained by normal participants using a data set without an injected trigger for training are called benign model updates” [specification, 0027]. The “benign model updates” of the claim are thus interpreted in line with the definitions given in the specification. Dependent claims 2-8 inherit this interpretation.
Claim Interpretation – 35 USC 112(f)
The following is a quotation of 35 USC 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 USC 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph, is invoked.
As explained in MPEP 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph:
the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim(s) 1-8 recite participants that perform various actions (e.g., “download a global model from a central server, and then use a data set thereof to train and update parameters of a local global model” and “upload the model update to the central server”). Moreover, the claims do not recite any hardware structure for these “participants” that is sufficient for performing the recited actions. The specification also does not indicate any specific hardware that these “participants” should have.
If applicant does not intend to have this/these limitation(s) interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph, applicant may:
amend the claim limitation(s) to avoid it/them being interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or
present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 USC 112(f) or pre-AIA 35 USC 112, sixth paragraph.
Claim Rejections - 35 USC 112(b)
The following is a quotation of 35 USC 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 USC 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1-8 is/are rejected under 35 USC 112(b) or 35 USC 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 USC 112, the applicant), regards as the invention.
The claims are generally narrative and indefinite, failing to conform with current U.S. practice. They appear to be a literal translation into English from a foreign document and are replete with grammatical and idiomatic errors.
Claim 1 is indefinite for at least the following reasons:
It recites A federated learning method against backdoor attacks [emphasis added] but this phrase is ungrammatical: it is not clear what it means for a method to be “against backdoor attacks” as presently recited. In view of the specification (cf. [specification, 0003-004, 0039], the examiner suggests “A federated learning method defending against backdoor attacks” for grammaticality.
It recites participants first download a global model from a central server, and then use a data set thereof to train and update parameters of a local global model, a parameter difference between an updated local global model and the global model originally downloaded in the round is a model update, and the participants then upload the model update to the central server [emphasis added] but this is indefinite for at least the following reasons:
The word “thereof” has ambiguous antecedent basis.
The grammar of the limitation appears to suggest that the participants collectively “download a global model from a central server, and then use a dataset thereof to train and update parameters of a local global model” (i.e., all of the participants together download a model, and then they all use the same dataset to train/update the same local global model) and that they collectively “upload the model update to the central server”. However, this conflicts with the disclosures of the specification which suggests instead that each participant has a distinct data set and trains a distinct local model. MPEP 2173.03 indicates that a claim is “indefinite when a conflict or inconsistency between the claimed subject matter and the specification disclosure renders the scope of the claim uncertain as inconsistency with the specification disclosure or prior art teachings may make an otherwise definite claim take on an unreasonable degree of uncertainty”. Consequently, this conflict with the specification renders the claim indefinite.
The relationship of the clause “a parameter difference between an updated local global model and the global model originally downloaded in the round is a model update” to the rest of these limitations is ambiguous: it is not clarified what entity computes the “model update” and/or “parameter difference” (and, if no entity computes it, how it can be that participants “upload the model update”). MPEP 2173.05(b)(II) indicates that a claim is “indefinite when a limitation of the claim is defined by reference to an object and the relationship between the limitation and the object is not sufficiently defined”. In the present instance, the claim is indefinite because the relationship of the model update and the remaining limitations of the claim are insufficiently defined.
To resolve the issues in this limitation, the examiner suggests:
“each participants a global model from a central server, and then uses a local data set each participant computes a model update, wherein the model update is a parameter difference between an updated the local global model and the global model originally downloaded in the round s the model update to the central server”
It recites the central server selects, through a cluster-based voting mode, benign model updates from a decentralized perspective [emphasis added]. However, it is not clear what it means to perform a selection “from a decentralized perspective”, nor is it clear whose perspective is required to be “decentralized”.
It recites the central server selects, through a cluster-based voting mode, benign model updates from a decentralized perspective when the benign model updates are in the majority [emphasis added]. However, this is indefinite for at least the following reasons. First, “the majority” lacks antecedent basis, and the claim does not make clear what the benign model updates are supposed to be “in the majority” of. MPEP 2173.05(b)(II) indicates that a claim is “indefinite when a limitation of the claim is defined by reference to an object and the relationship between the limitation and the object is not sufficiently defined” and, in the present instance, the entity of which the benign model updates are required to be in the majority of are left entirely undefined. Secondly, the underlined limitation appears to render the entire limitation a conditional limitation, since it appears to indicate that the selection is only performed in situations when benign model updates are in the majority. MPEP 2111.04(II) indicates that “[t]he broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only that those steps that must be performed and does not include steps that are not required to be performed because the condition(s) are not met”. This would imply that the claim does not require the selection to be performed at all. However, if the selection is not performed, all subsequent recitations of “the selected benign model updates” lack antecedent basis, rendering the whole claim indefinite. As best understood by the examiner in view of the specification, the examiner suggests replacing the underlined phrase with “… wherein the benign model updates are of the model updates” to resolve these issues.
It recites a differential training set is constructed according to the selected benign model updates and the set is utilized to train a variational auto-encoder; a differential verification set is constructed, data in the set is reconstructed by the variational auto-encoder [emphasis added]. However, these limitations are indefinite for at least the following reasons:
The limitations include a number of instances of passive voice (“is constructed”, “is utilized”, “is constructed”), rendering unclear whether these steps are actually required to be performed by the claim, and if so, what entity, if any, is required to perform these steps.
The two instances “the set” lack antecedent basis. While the former instance of this phrase presumably refers to the “differential training set”, it is not clear whether the latter instance refers to the “differential training set” or the “differential verification set” of the claim. For the purpose of compact prosecution, the claim is interpreted broadly herein as encompassing at least the latter interpretation.
To resolve the issues in this limitation, the examiner suggests:
“the central server constructs a differential training set utilizes the differential training set the central server constructs a differential verification set the variational auto-encoder reconstructs data in the differential verification set
It recites a population of the selected benign model updates is progressively expanded according to a reconstruction error [emphasis added]. However, this is indefinite for at least the following reasons. First, this is yet another limitation in the passive voice, rendering unclear whether this step is actually required to be performed by the claim, and if so, what entity, if any, is actually required to perform these steps. More substantively, it is not clear what it means for the population of selected benign model updates to be “progressively expanded” since the claim does not describe either the nature of the progression in question, nor what the population is to be expanded relative to. For example, it is not clear if it is the use of the variational auto-encoder that expands the population of selected benign model updates relative to the population that was determined by the cluster-based voting, or if the population of selected benign model updates is required to expand from round to round of federated learning, or something else entirely. Appropriate clarification of claim language is required.
It recites the central server performs a federated average algorithm on all the selected benign model updates to obtain a global model and distributes the global model to the participants for a next round of federated learning [emphasis added] but the claim already introduces both “a global model” and “federated learning” resulting in repeated nomenclature and ambiguous antecedent basis. For proper antecedent basis, the examiner suggests:
“the central server performs a federated average algorithm on all the selected benign model updates to obtain an updated global model and distributes the updated global model to the participants for a next round of the federated learning”
Dependent claims 2-8 inherit these rejections.
Claim 2 is indefinite for at least the following reasons:
It includes two recitations of model updates, despite the fact that the parent claim already introduces model updates, rendering unclear what the relationship is between these model updates and the entities introduced in the parent claim.
It recites model updates obtained by the participants [emphasis added] but the parent claim describes model updates being obtained from participants, not by participants.
It recites using a data set but the parent claim already introduces “a data set”.
It recites using data set but this is ungrammatical.
As written, the claims appears to require that model updates received from certain participants (namely, those using a data set with an injected trigger) be malignant, but this is subjective terminology since something that is “malignant” for one person (or in one context) may not be for another person (or another context). As best understood by the examiner in view of the specification [specification, 0013, 0027], the claim may have been an attempt to define “malignant” model updates as being those that are received from those participants.
In view of suggestions made under the parent claim, the examiner suggests the following to resolve these issues:
“wherein the benign model updates are the model updates obtained from whose local data set does not have an injected trigger for training, whereas malignant model updates are the model updates obtained from whose local data set has
For the purpose of compact prosecution, the claim is interpreted broadly as encompassing at least this interpretation.
Claims 3-8 are generally narrative and indefinite, and it is not clear what aspects of these claims are actually required by claim scope. The specification merely mirrors the language of the claims and provides little substantive guidance as to the intended interpretation of these claims. Substantial revision would be required for all of these claims in order to clarify claim scope. A non-exhaustive list of indefinite aspects of claims 3-8 include:
There are numerous instances of the passive voice, rendering unclear whether these steps are required to be performed, and if so, what entity, if any, is required to perform these steps.
Claim 3 recites a model update set received by the central server is denoted as 𝒲 = {Δw_1, Δw_2, ..., Δw_N} [emphasis added], but it is not clear whether this limitation is requiring that a model update set literally be denoted by the exact symbol 𝒲. It recites i is a natural number and 1 < i < N and m is a natural number and 1 < m < L but this language does not make clear whether i and m are existentially or universally quantified (i.e., if the requirement is for some natural number i strictly between 1 and N, or for all natural numbers i strictly between 1 and N; and similarly for m). It recites that the global model is assumed as a neural network with L layers but it is not clear if a step of “assuming” is required by the claim, and if so, what entity is to perform this “assuming”. It recites in a cluster thereof [emphasis added] but the intended antecedent of “thereof” is not clear. It recites in this way, each model update votes for L times, each with a weight of 1 [emphasis added] but the intended antecedent of “this way” is not clear, and it is moreover not clear whether this limitation is adding a new requirement to the claim or merely reciting something the applicant believes to be a necessary consequence of the remaining claim limitations. It recites several model updates [emphasis added] but it is not clear what numbers of model updates are covered by the word “several”, and it is moreover not clear what relationship these “model updates” bear to the “N model updates” introduced in the parent claim. It recites with high confidence but this is subjective language: confidence that is “high” for one person may not be for another person. The substantive content of claim 3 as best understood by the examiner in view of these and numerous other issues of indefiniteness appears to be a requirement that the cluster-based voting method introduced in the parent claim be a “K-means algorithm” and, for the purpose of compact prosecution, the claim is interpreted as encompassing at least this interpretation. Dependent claims 4-7 inherit these rejections.
Claim 4 recites differences between model updates in tilde{𝒲} [emphasis added] but this is repeat nomenclature: it is not clear whether this refers to the “N model updates” of parent claim 1 or the “several model updates” of parent claim 3 or some other entities entirely. It recites according to a form of Cartesian product but this is ungrammatical due to a missing article before “Cartesian product” and it is moreover unclear how this limitation fits together with the remaining limitations of the claim (since a Cartesian product of a set with itself is not the same thing as the set of all differences of distinct elements of the set). It recites thus forming a differential training set [emphasis added]. However, the intended antecedent of “thus” is unclear, and the use of the word renders unclear whether this limitation is adding a new requirement to the claim or merely reciting something the applicant believes to be a necessary consequence of the remaining claim limitations. Moreover, “a differential training set” is repeated nomenclature since parent claim 1 already introduces “a differential training set”, rendering unclear whether or not the “differential training set” of this dependent claim is bound in scope by the entity of the same name introduced in the parent claim. It includes multiple recitations of the word any that render unclear whether the recited entities are intended to be existentially or universally quantified. It recites train a variational autoencoder [emphasis added] but it is unclear whether or not the “variational autoencoder” of this dependent is bound in scope by the “variational auto-encoder” introduced in the parent claim (if so, the examiner notes that consistent punctuation is required). It recites a vector of the same dimension [emphasis added] but it is not clear what the dimension is supposed to be the same as. It recites the variational autoencoder can generate an output that is as similar as possible to the input [emphasis added]. However, the use of the word “can” renders unclear whether the generation of such an output is actually required by the claim. Moreover, “as similar as possible” is subjective language (something that is “as similar as possible” to something else for one person may not be for another person), and the specification provides no objective criterion for this terminology. For the purpose of compact prosecution, the claim is interpreted broadly similarly to claim 4 as encompassing any procedure involving computing pairwise differences. Dependent claims 5-7 inherit the rejections.
Claim 5 recites that is 𝒲 - tilde{𝒲} but the intended antecedent of “that” is ambiguous. It includes multiple recitations of the word any that render unclear whether the recited entities are intended to be existentially or universally quantified. It recites the pieces but this lacks antecedent basis. For the purpose of compact prosecution, the claim is interpreted broadly as encompassing any procedure involving computing pairwise differences. Dependent claims 6-7 inherit the rejections.
Claim 7 recites when a population of benign model updates is expanded [emphasis added]. The use of “a population of benign model updates” as repeated nomenclature, rendering unclear whether the entity introduced in the dependent is bound in scope by the entity of the same name introduced in the parent claim. The phrase “is expanded” is passive language, and it is not clear what the population is supposed to be expanded relative to. Moreover, this is a conditional limitation, rendering unclear whether the consequent step is actually required as being performed. The claim also recites a latest set tilde{𝒲} but this appears to redefine the notation tilde{𝒲} (it is previously introduced as referring to a “benign model update set”, and this dependent claim appears to redefine it as a “latest set”, rendering unclear what the relationship between the “benign model update set” and the “latest set” is intended to be). It recites if the number of sets tilde{𝒲} exceeds a preset threshold… and otherwise [emphasis added], but this too recites conditional limitations, since the conditions are not required to be satisfied. Moreover, the phrase “the number of sets tilde{𝒲}” lacks antecedent basis.
Claim 8 recites only a minor change is made to an original federated learning protocol, which on the one hand has little impact on the accuracy of the global model, and on the other hand is easy to integrate with an existing federated learning system [emphasis added] but all of these are subjective determinations. It also recites the method, through cluster-based voting and progressive selection, may steadily select a group of benign model updates for aggregation without obvious clustering of benign and malignant model updates to different locations [emphasis added]. The phrase “the method” lacks antecedent basis, the nature of the “progressive selection” is not made clear, and the use of the word “may” renders unclear whether the claim even requires this step. A determination that something is “steady” or “obvious” is subjective. The use of “benign model updates” and “benign and malignant model updates” appears to introduce repeated nomenclature with nomenclature that is already introduced in parent claims. For the purpose of compact prosecution, these limitations are not treated as having patentable weight herein.
Claim(s) 1-8 recite(s) elements which invoke interpretation under 35 USC 112(f). As noted above, the specification does not indicate any specific hardware underlying the “participants” of the claim. Moreover, even if the claims are interpreted so that the “participants” are general-purpose computers, MPEP 2181(II)(B) indicates that “the structure be more than simply a general purpose computer or microprocessor and that the specification must disclose an algorithm for performing the claimed function”, that “[a]n algorithm is defined, for example, as ‘a finite sequence of steps for solving a logical or mathematical problem or performing a task’” and that “a rejection under 35 USC 112(b) or pre-AIA 35 USC 112, second paragraph is appropriate if the specification discloses no corresponding algorithm associated with a computer or microprocessor”. In the present instance, the specification provides no algorithms for performing the indicates steps (e.g., it does not describe the algorithm used by the participants to “train and update parameters of a local global model”). The claims are consequently indefinite.
Claim Rejections - 35 USC 112(a)
The following is a quotation of the first paragraph of 35 USC 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 USC 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim(s) 1-8 is/are rejected under 35 USC 112(a) or 35 USC 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 USC 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim(s) 1-8 recite(s) limitations invoked 35 USC 112(f) and are rejected under 35 USC 112(b) for failing to disclose sufficient structure. MPEP 2181(II)(B) indicates that “[w]hen a claim containing a computer-implemented 35 USC 112(f) claim limitation is found to be indefinite under 35 USC 112(b) for failure to disclose sufficient corresponding structure (e.g., the computer and the algorithm) in the specification that performs the entire claimed function, it will also lack written description under 35 USC 112(a)”. Consequently, the claim(s) is/are rejected under 35 USC 112(a) for lack of written description.
Claim Rejections - 35 USC 103
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 USC 102(b)(2)(C) for any potential 35 USC 102(a)(2) prior art against the later invention.
Claim(s) 1-3 and 8 is/are rejected under 35 USC 103 as being unpatentable over Zhipin GU et al. (Detecting Malicious Model Updates from Federated Learning on Conditional Variational Autoencoder, published 2021-06-28; hereafter, “Gu”) in view of Tao LIU et al. (Evil vs evil: using adversarial examples to against backdoor attack in federated learning, published 2022-06-29; hereafter, “Liu”).
Claim 1
Liu discloses:
A federated learning method against backdoor attacks, ([Gu, abstract and section III.A]: Gu discloses Fedcvae, a federated learning framework that protects against model poisoning attacks [Gu, abstract], including backdoor attacks [Gu, section III.A.1 paragraph beginning “Model update poisoning”].)
wherein in each round of federated learning, ([Gu, algorithm 2]: The algorithm disclosed in Gu takes place in rounds [Gu, algorithm 2 server line 2 or client line 1].) participants first download a global model from a central server, ([Gu, algorithm 2]: Gu discloses each client k receiving the current global iterate w_t [Gu, algorithm 2 client line 2; see also, server lines 4-5].) and then use a data set thereof to train and update parameters of a local global model, ([Gu, algorithm 2]: Gu discloses each client k performing training using the local dataset P_k [Gu, algorithm 2 client lines 3-6].)
a parameter difference between an updated local global model and the global model originally downloaded in the round is a model update, ([Gu, algorithm 2]: Gu discloses computing w^k_{t+1} as a difference between the iterate w_t received from the server and a term η∇ℓ(w_t, b) [Gu, algorithm 2 client line 6]. As noted above, w_t is the “global model originally downloaded in the round” of the claim. Then η∇ℓ(w_t, b) maps to the “updated local global model” and w^k_{t+1} to the “parameter difference” and the “model update” of the claim.)
and the participants then upload the model update to the central server, ([Gu, algorithm 2]: Gu discloses each client k sending w^k_{t+1} to the central server [Gu, algorithm 2 client line 7; see also, server lines 4 and 6].)
wherein: after receiving N model updates, ([Gu, algorithm 2]: Gu discloses the server receiving w^k_{t+1} from each client k in the set S_t [Gu, algorithm 2 server lines 4 and 6].)
the central server selects, [through a cluster-based voting mode,] benign model updates from a decentralized perspective ([Gu, section IV.E and algorithm 2]: Gu discusses “two scenarios where 10% and 30% of selected clients are malicious, respectively” [Gu, section IV.C first paragraph]. The non-malicious model updates map to the “benign model updates” of the claim (cf. examiner’s remarks and the rejection of claim 2 as given below). Gu further discloses a method by which received model updates are analyzed so that certain updates may be excluded [Gu, algorithm 2 server lines 4 and 7 and 8-11]. The benign model updates which are not excluded from aggregation map to the “selected benign model updates” of the claim.)
when the benign model updates are in the majority, ([Gu, section IV.E]: As noted above, Gu discusses “two scenarios where 10% and 30% of selected clients are malicious, respectively” [Gu, section IV.C first paragraph]. This means that 90% and 70% of the clients are benign in these two situations, so the “benign model updates are in the majority” as recited by the claim.)
wherein N is a quantity of the participants in the federated learning; ([Gu, algorithm 2]: As noted above, Gu discloses the server receiving updates from each client in the set S_t [Gu, algorithm 2 server lines 4 and 6]. The number of clients in the set S_t maps to the integer N of the claim, since it is “a quantity of the participants in the federated learning” as recited by the claim.)
a differential training set is constructed according to the selected benign model updates and the set is utilized to train a variational auto-encoder; ([Gu, algorithm 2 and section III.B]: Gu discloses the use of a conditional variational auto-encoder (CVAE) to distinguish benign and malicious model updates [Gu, algorithm 2 server lines 9-11 and section III.B]. More precisely, a “CVAE is a model with the encoder-decoder architecture. The encoder module projects original variables to the low-dimensional embeddings. The decoder module then reconstructs the original variables from these embeddings” [Gu, section III.B first paragraph]. Moreover, a CVAE is unsupervised and is trained on-the-fly (cf. “we do not need to train the model in prior”) [Gu, section III.B paragraph beginning “Since”]. Such a model can be used to distinguish benign and malicious model updates because “the reconstruction error of malicious updates is much higher than that of benign ones” [Gu, section III.B paragraph beginning “Since”] so “updates with reconstruction errors higher than the threshold are considered as malicious updates and are excluded from model aggregation” [Gu, section III.B paragraph beginning “In each”]. In other words, the conditional variational auto-encoder (CVAE) of Gu maps to the “variational auto-encoder” of the claim. All of the data passing through the CVAE maps to the “differential training set” of the claim, since all of this data can be considered training data for this unsupervised model (i.e., it is “utilized to train [the] variational auto-encoder” as recited by the claim). Moreover, this data includes the “selected benign model updates” as mapped above, so the dataset falls under the broadest reasonable interpretation of being “constructed according to the selected benign model updates” as recited by the claim.)
a differential verification set is constructed, ([Gu, section III.B]: The data passing through the CVAE maps to the “differential verification set” of the claim.)
data in the set is reconstructed by the variational auto-encoder, ([Gu, section III.B]: As noted above, Gu discloses that the decoder of the CVAE “reconstructs the original variables from these embeddings” [Gu, section III.B first paragraph]. The data being reconstructed maps to the “data in the [differential verification] set” of the claim.)
and a population of the selected benign model updates is progressively expanded according to a reconstruction error; ([Gu, algorithm 2 and section III.B]: As noted above, Gu discloses computing reconstruction errors [Gu, algorithm 2 server line 9 and section III.B], any of which map to the “reconstruction error” of the claim. Moreover, as noted above, the non-excluded models map to the “population of the selected benign model updates” of the claim. This falls under the broadest reasonable interpretation of this limitation as best understood by the examiner in view of the 112(b) rejections.)
and the central server performs a federated average algorithm on all the selected benign model updates to obtain a global model without a backdoor ([Gu, algorithm 2 and section III]: Gu discloses updating the global iterate using aggregation methods after excluding updates [Gu, algorithm 2 server lines 11-12]. More precisely, Gu discloses that they “use the existing aggregation methods like FedAvg after excluding the malicious updates” [Gu, section III.B last paragraph]. As noted above, the non-excluded updates map to the “selected benign model updates” of the claim, so the aggregation disclosed in Gu maps to the step of “perform[ing] a federated average algorithm on all the selected benign model updates” as recited by the claim. The updated global iterate maps to the “[updated] global model” of the claim.)
and distributes the global model to the participants for a next round of federated learning. ([Gu, algorithm 2]: Gu discloses beginning each round by broadcasting the current global iterate to the clients [Gu, algorithm 2 server lines 4-5].
Gu might not distinctly disclose:
[selects,] through a cluster-based voting mode, [benign model updates]
Liu is in the field of machine learning. It discloses a federated learning method that protects against backdoor attacks [Liu, title and abstract]. Moreover, Gu in view of Liu discloses:
[selects,] through a cluster-based voting mode, [benign model updates] ([Liu, section 4]: In the federated learning method of Liu, “the server uses K-means clustering algorithm to divide the updated models into ‘benign’, ‘malicious’, ‘suspected benign’ and ‘suspected malicious’” [Liu, section 4 first paragraph]. K-means clustering falls under the broadest reasonable interpretation of a “cluster-based voting method” as recited by the claim; the examiner notes that the specification describes the use of K-means clustering in precisely this context (e.g., [specification, 0014]). In the combination with Gu, both K-means clustering as in Liu and variational auto-encoders as in Gu are used to select the model updates to be used for aggregation.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the federated learning method of Gu with aspects of the method disclosed in Liu because the system disclosed in Liu “can reduce the attack success rate from 99% to 1%” [Liu, abstract], so the combination would be more robust overall.
Claim 2
Gu in view of Liu discloses the elements of the parent claim(s). It also discloses:
[The federated learning method according to claim 1, wherein] the benign model updates refer to model updates obtained by the participants using a data set without an injected trigger for training, whereas model updates obtained by the participants using data set based on the injected trigger for training are malignant model updates. ([Gu, section IV.E; Liu, section 3.1]: Gu discusses backdoor attacks in which the “training dataset [of a malicious client] includes a mix of correctly labeled inputs and backdoored inputs to induce misclassification” [Gu, section IV.E]. The backdoored inputs in the training data map to the “injected trigger” of the claim. The examiner notes that Liu also discloses that “a malicious client will purposely perturb the training data (e.g., with trigger) to mislead the model” [Liu, section 3.1].)
The same motivation to combine applies.
Claim 3
Gu in view of Liu discloses the elements of the parent claim(s). It also discloses:
[The federated learning method according to claim 1, wherein] a specific implementation of mining the benign model updates by the central server is as follows: first, a model update set received by the central server is denoted as 𝒲 = {Δw_1, Δw_2, ..., Δw_N}, wherein Δw_i is a model update uploaded by an ith participant, and i is a natural number and 1 < i < N; ([Gu, algorithm 2]: The model updates w^k_{t+1} for k in S_t [Gu, algorithm 2 server lines 4 and 6] map to the Δw_i of the claim.)
the global model is assumed as a neural network with L layers, ([Gu, section II; Liu, figure 2]: The models in Gu are neural networks. For example, Gu discusses SGC in [Gu, section II], which is a method for training neural networks. The models in Liu are also neural networks. For example, [Liu, figure 2] depicts the global and local models as being neural networks.)
and for an mth layer parameter Δw_{i,m} in a model update Δw_i, a zero vector and K-1 model updates farthest from Δw_{i,m} are selected as initial points of a K-means algorithm, and Δw_{i,m} votes for all model updates in a cluster thereof after dividing into clusters, wherein m is a natural number and 1 < m < L; in this way, each model update votes for L times, each with a weight of 1; and finally, several model updates with a highest vote form a benign model update set with high confidence, denoted as tilde{𝒲}. ([Liu, section 4]: As noted under the 112(b) rejections, the claim as recited includes numerous issues which render unclear what the scope of the claim is. As best understood by the examiner, the claim appears to be requiring that the “cluster-based voting method” of the parent claim be a K-means algorithm. However, as noted under the parent claim, Liu does disclose the use of K-means clustering to identify benign model updates [Liu, section 4 first paragraph].)
Claim 8
Gu in view of Liu discloses the elements of the parent claim(s). It also discloses:
[The federated learning method according to claim 1,] wherein the federated learning method does not rely on adjustment of differential privacy, weight clipping and a learning rate, ([Liu, entire document]: The method disclosed in Liu does not make use of differential privacy or clipping at all [Liu, entire document], so it certainly does not rely on adjustment of” these aspects. Liu also indicates that it makes use of a “fixed learning rate” [Liu, section III.A first paragraph; emphasis added] so it also does not “rely on adjustment of… a learning rate” as recited by the claim.)
and only a minor change is made to an original federated learning protocol, which on the one hand has little impact on the accuracy of the global model, and on the other hand is easy to integrate with an existing federated learning system; and moreover, the method, through cluster-based voting and progressive selection, may steadily select a group of benign model updates for aggregation without obvious clustering of benign and malignant model updates to different locations. ([Liu, entire document]: As noted under the 112(b) rejections, these limitations include numerous subjective aspects and it is not clear what, if anything, their patentable weight is. The disclosures if Liu appear to satisfy the requirements of these indefinite limitations are best understood by the examiner.)
The same motivation to combine applies.
Claim(s) 4-5 and 7 is/are rejected under 35 USC 103 as being unpatentable over Gu in view of Liu, further in view of Irina HIGGINS et al. (β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework, published 2017; hereafter, “Higgins”).
Claim 4
Gu in view of Liu discloses the elements of the parent claims. It might not distinctly disclose:
[The federated learning method according to claim 3, wherein] after obtaining the set tilde{𝒲}, the central server first calculates differences between model updates in tilde{𝒲} according to a form of Cartesian product, thus forming a differential training set; any difference data in the differential training set is Δw_a - Δw_b, wherein Δw_a and Δw_b are any two different model updates in the set tilde{𝒲}; and then the differential training set is utilized to train a variational autoencoder, of which an input is any difference data in the differential training set and an output is a vector of the same dimension, and the variational autoencoder can generate an output that is as similar as possible to the input.
Higgins is in the field of machine learning. It discusses variational autoencoders [Higgins, abstract]. Moreover, it discloses:
after obtaining the set tilde{𝒲}, the central server first calculates differences between model updates in tilde{𝒲} according to a form of Cartesian product, thus forming a differential training set; any difference data in the differential training set is Δw_a - Δw_b, wherein Δw_a and Δw_b are any two different model updates in the set tilde{𝒲}; and then the differential training set is utilized to train a variational autoencoder, of which an input is any difference data in the differential training set and an output is a vector of the same dimension, and the variational autoencoder can generate an output that is as similar as possible to the input. ([Higgins, figure 5]: As noted under the 112(b) rejections, this claim both inherits and contains numerous issues of indefiniteness which cause substantial uncertainty regarding the scope of the claim. As best understood by the examiner, the claim appears to be requiring some process of generating pairwise differences between data points. Higgins discloses such a procedure of computing differences between pairs of data [Higgins, figure 5]. In other words, z_{1,l} and z_{2,l} correspond to Δw_a and Δw_b of the claim, respectively, with the z^l_{diff} for varying l being the differences Δw_a - Δw_b of the claim as best understood by the examiner in view of the 112(b) rejections.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the federated learning framework of Gu in view of Liu with aspects of the system disclosed in Higgins because the latter is “stable to train, makes few assumptions about the data and relies on tuning a single hyperparameter” [Higgins, abstract], thereby resulting in a more effective system.
Claim 5
Gu in view of Liu and Higgins discloses the elements of the parent claim(s). It also discloses:
[The federated learning method according to claim 4, wherein] a complementary set of the set tilde{𝒲}, is taken, that is 𝒲 - tilde{𝒲}; any difference data in the differential verification set is Δw_c - Δw_d, wherein Δw_c is any model update in the complementary set, Δw_d is any model update in the set tilde{𝒲}; the variational autoencoder is utilized to reconstruct each piece of difference data in the differential verification set; and several pieces of difference data with a least average reconstruction error are selected, and Δw_c in the pieces of difference data is added to the set tilde{𝒲}. ([Higgins, figure 5]: As noted under the 112(b) rejections, this claim both inherits and contains numerous issues of indefiniteness which cause substantial uncertainty regarding the scope of the claim. As best understood by the examiner, the claim appears to be requiring some process of generating pairwise differences between data points. Higgins discloses such a procedure of computing differences between pairs of data [Higgins, figure 5]. In other words, z_{1,l} and z_{2,l} correspond to Δw_c and Δw_d of the claim, respectively, with z^l_{diff} being the differences Δw_c - Δw_d of the claim, as best understood by the examiner in view of the 112(b) rejections.)
Claim 7
Gu in view of Liu and Higgins discloses the elements of the parent claim(s). It also discloses:
[The federated learning method according to claim 5, wherein] when a population of benign model updates is expanded, a latest set tilde{𝒲} is utilized to update the differential training set and fine-tune the variational autoencoder for judgment; ([Liu, section 4; Gu, algorithm 2 and section III.B]: As noted under the 112(b) rejections, this claim inherits and contains numerous issues of indefiniteness which cause substantial uncertainty regarding the scope of the claim. As noted under the parent claims, the set tilde{𝒲} maps to the benign model updates determined by the K-means clustering of Liu. Moreover, the “differential training set” as mapped under the parent claim is used to train/fine-tune the “variational autoencoder” as mapped above. The inferences performed by the variational autoencoder fall under the broadest reasonable interpretation of “judgment” as recited by the claim.)
if the number of sets tilde{𝒲} exceeds a preset threshold, the federated averaging algorithm is performed on the sets; and otherwise, the differential verification set is updated with the latest set tilde{𝒲}, and the benign model updates are expanded again through reconstruction until the number of model updates contained in sets tilde{𝒲} exceeds the preset threshold. ([Gu, algorithm 2 and section III]: As noted under the 112(b) rejections, this claim inherits and contains numerous issues of indefiniteness which cause substantial uncertainty regarding the scope of the claim. In particular, these limitations are conditional: the claim positively recites neither something exceeding a preset threshold, nor not, so neither of the actions that are contingent on these hypotheses is in the broadest reasonable interpretation of the claim. Nonetheless, as noted under the parent claim, Gu [Gu, algorithm 2 and section III] does disclose the step of “performing the federated averaging algorithm” as recited by this claim.)
The same motivation to combine applies.
Claim(s) 6 is/are rejected under 35 USC 103 as being unpatentable over Gu in view of Liu and Higgins, further in view of Simon HAWKINS et al. (Outlier Detection Using Replicator Neural Networks, published 2002-01-01; hereafter, “Hawkins”).
Claim 6
Gu in view of Liu and Higgins discloses the elements of the parent claim(s). It might not distinctly disclose a specific measure of reconstruction error as a mean square error. In other words, it might not distinctly disclose:
[The federated learning method according to claim 5, wherein] the reconstruction error is measured by a mean square error between input data of the variational autoencoder and reconstruction data.
Hawkins is in the field of machine learning. It discusses a method of anomaly detection using “replicator neural networks (RNNs)” [Hawkins, abstract]. The examiner notes that an RNN in the sense discussed in Hawkins is the same as what is now called an “autoencoder” (e.g., “In the RNN model the input variables are also the output variables so that the RNN forms an implicit, compressed model of the data during training” [Hawkins, section 1]). Moreover, Gu in view of Liu, Higgins, and Hawkins discloses:
[The federated learning method according to claim 5, wherein] the reconstruction error is measured by a mean square error between input data of the variational autoencoder and reconstruction data. ([Hawkins, section 3.1]: Hawkins discloses the use of “mean square error” (denoted e_l) to measure reconstruction error [Hawkins, section 3.1 equation 7 and the surrounding text]. In the equation, x_{i,j} corresponds to the “input data” of the claim and o^l_{i,j} to the “reconstruction data” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the federated learning framework of Gu in view of Liu and Higgins with the use of reconstruction for anomaly detection as disclosed in Hawkins because it “let[s] the data speak for itself without relying on too many assumptions” and is “able to identify outliers… with high accuracy” [Hawkins, section 5], so the combination would be more effective overall.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wei-han LEE at al. (US20240070286A1, effectively filed 2022-08-31; hereafter, “Lee”) discloses a federated learning system that identifies malicious updates using an anomaly detector and excludes them from aggregation [Lee, figures 3-4].
Suyi LI et al. (Learning to Detect Malicious Clients for Robust Federated Learning, published 2020-02-01; hereafter, “Li”) discuses a federated learning method that identifies malicious updates using a variational autoencoder and excludes them from aggregation [Li, abstract and section 3.3].
Junyu SHI et al. (Challenges and Approaches for Mitigating Byzantine Attacks in Federated Learning, published 2022; hereafter, “Shi”) discusses federated learning [Shi, figure 1], Byzantine attacks on federated learning systems (including backdoor attacks) [Shi, section II.B], and numerous methods for guarding against such attacks [Shi, table I].
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Shishir AGRAWAL whose telephone number is +1 703-756-1183. The examiner can normally be reached Monday through Thursday, 08:30-14:30 Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey SHMATOV can be reached on +1 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is +1 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at +1 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call +1 800-786-9199 (IN USA OR CANADA) or +1 571-272-1000.
/S.A./Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights