Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1 –7 are pending.
Any references to applicant’s specification are made by way of applicant’s U.S. pre-grant printed patent publication.
This action is in response to the communication filed on 6/2/26.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the features of “…subsets of models layers…” and “…portions of one or more model layers…” must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 – 7 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claim 1, the applicant’s specification fails to disclose the features of “…wherein the one or more individual segments are selected from the group consisting of one or more subsets of models layers of the first client's model or one or more portions of one or more model layers of the first client's model …” and “…wherein the one or more individual segments are selected from the group consisting of one or more subsets of models layers of the second client's model or one or more portions of one or more model layers of the second client's model …”.
Specifically, the examiner notes that the applicant’s original disclosure teaches that the term “segment” is equivalent in meaning to that of a “portion of model layer” (e.g. Specification, par. 46: “…The central server 5 then requests a … portion of a model layer(s), i.e., client model segments…”; “…These client model segments are then subject to homomorphic encryption using a public key … to encrypt each … portion of model layer (segment) …”).
Applicant does not appear to disclose that “segments” consist of subsets of layers of a client model (i.e. “…selected from the group consisting of one or more subsets of models layers …”).
Depending claims are rejected by virtue of dependency.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 – 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, the recitation “…wherein the one or more individual segments are selected from the group consisting of one or more subsets of models layers of the first client's model …” and “…wherein the one or more individual segments are selected from the group consisting of one or more subsets of models layers of the second client's model …” renders the scope of the claims indefinite. Specifically, the examiner notes that the applicant’s specification does not appear to disclose that “segments” consist of subsets of layers of a client model (i.e. “…selected from the group consisting of one or more subsets of models layers …”). Rather, applicant’s original disclosure teaches that the term “segment” is equivalent in meaning to that of a “portion of model layer” (e.g. Specification, par. 46: “…The central server 5 then requests a … portion of a model layer(s), i.e., client model segments…”.
Thus, the lack of clear correspondence between the claim language and the applicant’s specification renders the meaning of the claimed terms (e.g. “segments”) unclear.
Depending claims are rejected by virtue of dependency.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, and 7 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang et al. (Zhang), “BatchCrypt: efficient homomorphic encryption for cross-silo federated learning”.
Regarding claim 1, as best determined in view of the above noted deficiencies of clarity, Zhang discloses:
A process for securing individual client data during federated learning of a global model, the process comprising: during a first global model training run (e.g. Zhang, Abstract):
i. receiving, by a first client, a request from a central server, for one or more individual segments of a first client's model trained on first client data, wherein the one or more individual segments are selected from the group consisting of one or more subsets of models layers of the first client's model or one or more portions of one or more model layers of the first client's model and further wherein the selected one or more segments represent less than the first client's model (e.g. Zhang, pg. 500, Algorithm 1: steps 4-7, 11; sect. 2.3, par. 1, 2; sect. 4.3, par. 1 – aggregator server initializes a model (i.e. “first” client’s model, as it is shared with a first client) and requests a plurality of plurality of first, second, third, etc. clients to train upon it, such that the first client can send model gradients, i.e. portions of the model’s layers, back to the server as a result);
ii. generating, by a key distributor, a first public-private key pair for the first global model training run and providing a first public key to the first client (e.g. Zhang, sect. 2.3, par. 2 – a randomly selected client acts as a key distributer by generating a public/private key pair and distributing the key to each client);
iii. homomorphically encrypting, by the first client, each of the one or more individual segments of the first client's model using the first public key (e.g. Zhang, sect. 2.3, par. 1, 2 – each client homomorphically encrypts their own portions of the model with the public key);
iv. providing, by the first client, responsive to the request, the encrypted one or more individual segments of the first client's model and the first public key to the server (e.g. Zhang, sect. 2.3, par. 2 – each client transmits the resulting homomorphic encryption of the model back to the aggregator server);
v. receiving, by a second client, the request from a central server, for one or more individual segments of a second client's model trained on second client data, wherein the one or more individual segments are selected from the group consisting of one or more subsets of models layers of the second client's model or one or more portions of one or more model layers of the second client's model and further wherein the selected one or more segments represent less than the second client's model (e.g. Zhang, pg. 500, Algorithm 1: steps 4-7, 11; sect. 2.3, par. 1, 2; sect. 4.3, par. 1 – aggregator server initializes a model (i.e. “second” client’s model, as it is shared with a second client) and requests a plurality of plurality of first, second, third, etc. clients to train upon it, such that the second client can send model gradients, i.e. portions of the model’s layers, back to the server as a result);
vi. providing, by the key distributor, the first public key to the second client (e.g. Zhang, sect. 2.3, par. 2 – a randomly selected client acts as a “leader” and key distributer by generating a public/private key pair and distributing the key to each client);
vii. homomorphically encrypting, by the second client, each of the one or more individual segments of the second client's model using the first public key (e.g. Zhang, sect. 2.3, par. 1, 2 – each client homomorphically encrypts their own portions of the model with the public key);
viii. providing, by the second client, responsive to the request, the encrypted one or more individual segments of the second client's model and the first public key to the server (e.g. Zhang, sect. 2.3, par. 2 – each client transmits the resulting homomorphic encryption of the model back to the aggregator server);
ix. aggregating, by the server, the encrypted one or more individual segments of the first and second clients' models to generate an updated first global model (e.g. Zhang, sect. 2.3, par. 2 – the aggregator server receives the updated, homomorphically encrypted model portions and adds them all up);
x. notifying, by the server, the key distributor to provide the private key from the first public-private key pair to requesting clients (e.g. Zhang, Introduction, par. 2; sect. 2.1, par. 1; sect. 2.3, par. 2 – the aggregator server notifies the leader client to generate and synchronize, i.e. distribute, the private key to all participating clients);
xi. pushing, by the server, the updated first global model to the first and second clients; training the updated first global model on the first and second data at the first and second clients (e.g. Zhang, sect. 2.3, par. 2 – the aggregator server distributes the aggregated model back to the clients);
and repeating steps i. to xi. for at least one additional global model training run until a final global model meeting at least one predetermined criteria is generated by the server (e.g. Zhang, Introduction, par. 2; sect. 2.2, “Secure Aggregation” – the process is repeated for any number of iterations).
Regarding claim 2, Zhang discloses:
wherein, prior to first training on first and second clients' data, initial parameters W for an untrained model are determined by the server and provided to the first and second clients, c.sub.i, wherein i=1 for the first client and i=2 for the first client; and further wherein each client c.sub.i initializes its model with W and trains over data D.sub.i to produce its own new parameters, w.sub.i. (e.g. Zhang, Introduction, par. 2; sect. 2.3, par. 1, 2 – each client, i.e. c1 and c2, the server causes model weights, i.e. W, to be distributed to each client).
Regarding claim 7, Zhang discloses:
wherein the client data is selected from the group consisting of personal health information, personal financial information, personal identifying information, and asset location information (e.g. Zhang, Introduction, par. 1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3 – 6 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (Zhang), “BatchCrypt: efficient homomorphic encryption for cross-silo federated learning” in view of Ding et al. (Ding), US 2022/0300618 A1.
Regarding claim 3, Zhang does not disclose, but Ding does disclose the server performing a model averaging function for the aggregated models (e.g. Ding par. 9). It would have been obvious to one of ordinary skill in the art to apply the teachings of Ding because one of ordinary skill in the art would have been motivated by Zhang’s teachings to consider the features of model averaging (e.g. Zhang, sect. 7, “Model Averaging”). Thus, the combination enables:
wherein the aggregating, by the server, the encrypted one or more individual segments of the first and second clients' models is performed in accordance with the following procedure:
PNG
media_image1.png
59
107
media_image1.png
Greyscale
wherein each c.sub.i sends its trained w.sub.i, along with a size of its training data set t.sub.i=|D.sub.i| to the server in advance thereof (e.g. Ding, par. 9, 29, 60).
Regarding claim 4, Zhang does not disclose, but Ding does disclose the models as parameter matrices (e.g. Ding par. 28, 30, 70). It would have been obvious to one of ordinary skill in the art to apply the teachings of Ding because one of ordinary skill in the art would have been motivated by Zhang’s teachings that the use of ciphertext matrices should be considered (e.g. Zhang, sect. 7, “Applicability in Vertical FL”). Thus, the combination enables:
wherein the one or more individual segments of the first and second clients' models are parameter matrices M (e.g. Ding par. 28, 30, 70).
Regarding claim 5, the combination enables:
wherein for each of the first and second clients, the server calculates a number of parameter matrices each client should send back to the server to generate a global model parameter matrix, and further wherein, when the server has received all requested w.sub.i parameter matrices and t.sub.i training example counts, the server creates the global model using the following formula to calculate each aggregated parameter matrix W.sub.j:
PNG
media_image2.png
50
163
media_image2.png
Greyscale
(e.g. Zhang, sect. 2.3, par. 1, 2; Ding, par. 28 – 30).
Regarding claim 6, Zhang does not disclose, but Ding does disclose the models as deep neural networks with parameter matrices (e.g. Ding par. 3, 7, 28, 30, 70). It would have been obvious to one of ordinary skill in the art to apply the teachings of Ding because one of ordinary skill in the art would have been motivated by Zhang’s teachings that deep learning models and ciphertext matrices should be considered (e.g. Zhang, sect. 3; sect. 7, “Applicability in Vertical FL”). Thus, the combination enables:
wherein the first and second client models are deep neural network (DNN) with M>0 parameter matrices (e.g. Zhang, sect. 3; Ding par. 3, 7, 28, 30, 70).
Response to Arguments
Applicant's arguments filed 6/2/26 have been fully considered but they are not persuasive.
Applicant argues or alleges essentially that:
…
… Respectfully, and contrary to the Examiner's suggestion in the rejection, Zhang's aggregator DOES NOT request one or more individual segments of a first (or any) client's model. … There certainly is no description in Zhang of individual segments being specifically requested … as claimed. …
…
(Remarks, pg. 5, 6)
Examiner respectfully responds:
The examiner respectfully disagrees. Specifically, Zhang’s aggregator clearly prompts each client to train upon the client’s shared model and provide results in return (e.g.
(e.g. Zhang, pg. 500, Algorithm 1: steps 4-7, 11; sect. 2.3, par. 1, 2). Thus, Zhang clearly teaches a “request” from an aggregator.
Applicant argues or alleges essentially that:
…
… Similarly, contrary to the Examiner's rejection, Zhang does not disclose a client sending portions of the model back to the server. … There certainly is no description in Zhang of individual segments being specifically …shared as claimed. …
…
(Remarks, pg. )
Examiner respectfully responds:
The examiner respectfully disagrees. Specifically, Zhang clearly teaches that each client sends their individual set of gradients (i.e. segments, i.e. portions of one or more model layers) of the client’s shared model back to the aggregator, such that the aggregator can aggregate all the individual client sets for the shared client model (e.g. Zhang, pg. 500, Algorithm 1: steps 11; sect. 2.3, par. 1, 2; sect. 4.3, par. 1).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY L WILLIAMS whose telephone number is (571)272-7965. The examiner can normally be reached on 7:30 am - 4:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Farid Homayounmehr can be reached on 571-272-3739. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFERY L WILLIAMS/Primary Examiner, Art Unit 2495
Prosecution Insights