DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This office action is in response to the application filled on 10/20/2023. Claims 1-20 are pending and have been examined.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/28/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1: The claim recites a method which falls into the statutory category of process.
Step 2A Prong 1: The claim recites multiple abstract ideas: transmitting the first shared data to a server for the server to determine second shared data based on the first shared data, this limitation amounts to a mental process given a human being can transform data to create a second shared data; updating the shared portion of the model based on the second shared data to obtain an updated shared portion, updating a portion of a model using data is a mental process that can reasonably be done by a human being using the aid of a generic computer to alter the parameters of a model after looking at some data; and generating, based on the updated shared portion, an updated model for performing a next one of the plurality of iterative updates in response to determining that the next one of the plurality of iterative updates is not a last one of the plurality of iterative updates, generating an updated model based on data after determining if more updates are necessary is considered a mental process given a human being can alter the parameters of model after looking at data and deciding whether or not additional updates are needed all reasonably with the aid of a generic computer.
Step 2A Prong 2: The claim does not integrate the abstract idea into a practical application since the additional elements of:
performing a plurality of iterative updates on a model held by a data party of a plurality of data parties participating in training the model, wherein the model comprises a shared portion and a dedicated portion, and performing one of the plurality of iterative updates comprises, is merely linking the abstract ideas to a technological environment/field of use: federated learning.
performing one or more times of iterative training on the model based on a training sample held by the data party to obtain model data, wherein the model data comprise first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model, is insignificant extra-solution activity: Well-understood, Routine, Conventional.
transmitting the first shared data to a server for the server to determine second shared data based on the first shared data, is insignificant extra-solution activity: data gathering.
transmitting the first shared data to a server for the server to determine second shared data based on the first shared data, is mere instructions to apply the abstract idea by a generic computer component, a server.
receiving the second shared data from the server, is insignificant extra-solution activity: data gathering.
Step 2B: Claim 1 does not include additional elements that are sufficient to amount to significantly more than a judicial exception. As discussed above, the additional elements of transmitting and receiving data are considered insignificant extra-solution activity because they are well-understood, routine, conventional activity as evidence by MPEP §2106.05(d)(II)(I). Additionally, the server used to determine the second shared data is considered mere instructions to apply the abstract idea. Furthermore, the model being held by a data party of a plurality of data parties participating in training the model, wherein the model comprises a shared portion and a dedicated portion amounts to merely linking the abstract ideas to the field of use/environment of federated learning; meanwhile, performing one or more times of iterative training on the model based on a training sample held by the data party to obtain model data is considered well-understood, routine, conventional activity because it is a common process used in conventional federated learning frameworks such as FedAvg where local training is done on different terminals then the gradients (model data) are aggregated (Par 0022, federated learning performs machine learning on a terminal device by combining computing power and data on different terminals, so as to ensure that individual data do not leave a domain. In addition, gradients are aggregated to combine terminal devices to perform joint training, thereby improving generalization of an overall model. However, it is difficult for a conventional federated learning framework (for example, FedAvg) ……). Claim 1 is not patent eligible.
Regarding claim 2, the rejection of claim 1 is incorporated, further the claim recites wherein the model data comprise a model parameter or gradient data obtained after one or more iterative updates. This limitation amounts to more specifics of the insignificant extra-solution activity of performing iterative training to obtain model data. The limitation remains well-understood, routine, conventional (as shown in the rejection for claim 1). As such, the claim does not have any additional elements that amount to an integration of the judicial exception into a practical application nor to significantly more. Claim 2 is not patent eligible.
Regarding claim 3, the rejection of claim 2 is incorporated, further the claim recites wherein the model data comprise the gradient data (this limitation amounts to more specifics of the data obtained through iterative training which remains well-understood, routine, conventional as in claim 2), and wherein generating the updated model comprises: updating the dedicated portion of the model based on the local data in the model data to obtain an updated dedicated portion; and generating, based on the updated shared portion and the updated dedicated portion, the updated model (this limitation amounts to more specifics of the abstract idea of generating an updated model including a mental process given a human being can reasonably update the parameters of portions of models to generate a new model using the aid of a generic computer). As such, the claim does not have any additional elements that amount to an integration of the judicial exception into a practical application nor to significantly more. Claim 3 is not patent eligible.
Regarding claim 4, the rejection of claim 2 is incorporated, further the claim recites wherein updating the shared portion of the model comprises: using the second shared data as a model parameter in the shared portion of the model. This limitation amounts to more specifics of the abstract idea of updating the shared portion of the model using the received data as a parameter. As such, the claim does not have any additional elements that amount to an integration of the judicial exception into a practical application nor to significantly more. Claim 4 is not patent eligible.
Regarding claim 5, the rejection of claim 4 is incorporated, further the claim recites wherein updating the shared portion of the model comprises: updating the model parameter in the shared portion of the model based on a learning rate and the second shared data. This limitation amounts to more specifics of the abstract idea of updating the shared portion of the model using the received data and an arbitrary learning rate. As such, the claim does not have any additional elements that amount to an integration of the judicial exception into a practical application nor to significantly more. Claim 5 is not patent eligible.
Regarding claim 6, the rejection of claim 1 is incorporated, further the claim recites wherein the second shared data are a weighted sum value or a weighted average value of the first shared data of the plurality of data parties. The limitation amounts to more specifics of abstract idea of determining a second shared data involving a mathematical concept: weighted average value. As such, the claim does not have any additional elements that amount to an integration of the judicial exception into a practical application nor to significantly more. Claim 6 is not patent eligible.
Regarding claim 7, the rejection of claim 1 is incorporated, further the claim recites wherein the model held by each of the plurality of data parties has a same model structure. This limitation amounts to more specifics of the technological environment in which the abstract ideas are applied. As such, the claim does not have any additional elements that amount to an integration of the judicial exception into a practical application nor to significantly more. Claim 7 is not patent eligible.
Regarding claims 8-14, given the claims are merely variations of claims 1-7 in which a non-transitory computer readable medium, which falls into the statutory category of manufacture, is claimed instead of a process, the rejections above are incorporated. Therefore, claims 8-14 are not patent eligible.
Regarding claims 15-20, given the claims are merely variations of claims 1-7 in which a system comprising multiple components, which falls into the statutory category of machine, is claimed instead of a process, the rejections above are incorporated. Therefore, claims 15-20 are not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kang (CN110399742B).
Regarding claim 1, Kang teaches performing a plurality of iterative updates on a model held by a data party of a plurality of data parties participating in training the model (Abs, the encryption migration model is generated by the K encryption sharing models uploaded by K terminals of K participants in the i-1 training period according to K terminals of K participants. The first terminal updates the encryption sharing model in the first local neural network model of the first terminal according to the sent encryption migration model in the ith training period, ith training period implies iterative process, the first terminal (a data party) updates the first local neural network model (which is the model here) and k terminals and k participants means there is a plurality of data parties), wherein the model comprises a shared portion and a dedicated portion, and performing one of the plurality of iterative updates comprises (Pg.4, A possible implementation, wherein the feature extraction layer of the encryption shared model is close to the input layer of the first local neural network model; the characteristic extraction layer of the first private model is close to the output layer of the first local neural network model, the first local neural network contains a shared model (shared portion) and a private model (dedicated portion)): performing one or more times of iterative training on the model based on a training sample held by the data party to obtain model data, wherein the model data comprise first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model (Pg.4, the K encryption sharing models trained in the first training period are trained by the K terminals according to
the initial encryption migration model and the respective training data of each terminal, Pg.14, the first private model can be independently trained by the first data, the first local neural network can be trained according to the respective training data of each terminal generating a shared model (shared portion of model data) and a private model (local data)); transmitting the first shared data to a server for the server to determine second shared data based on the first shared data (Abs, the encryption migration model is generated by the K encryption sharing models uploaded by K terminals of K participants, encryption sharing model (first data) is used to determine the encryption migration model (second shared data)); receiving the second shared data from the server; updating the shared portion of the model based on the second shared data to obtain an updated shared portion (Abs, the first terminal obtains the encryption migration model sent by the parameter server in the ith training period…….the first terminal updates the encryption sharing model in the first local neural network model of the first terminal according to the sent encryption migration model in the ith training period, updating the encryption sharing model (shared portion) based on the encryption migration model (second shared data) that was obtained from the server); and generating, based on the updated shared portion, an updated model for performing a next one of the plurality of iterative updates in response to determining that the next one of the plurality of iterative updates is not a last one of the plurality of iterative updates (Pg.4, the parameter server determines whether to end the training according to K encrypted loss values uploaded by the K terminals in the i-1 training period, determines if any more iterations are needed, each iteration updates the encrypted sharing model, as shown above, which is a part of the first local neural network, meaning the first local neural network (the model) is updated until training ends).
Regarding claim 2, Kang teaches wherein the model data comprise a model parameter or gradient data obtained after one or more iterative updates (Pg. 4, A possible implementation, the encryption shared model comprises an N-layer feature extraction layer, the encryption shared model (which is the model data being sent) is a model with layers which implies it includes model parameters such as weights).
Regarding claim 3, Kang teaches wherein the model data comprise the gradient data (Pg.15, step one, participating party k using gradient descent algorithm through local data Sk and local depth neural network model calculating the loss [[Lk, t]] and model gradient Then according to the gradient updating the local depth neural network model to obtain step two, participating party k the loss [[Lk, t]] and the updated encryption sharing model and sending to the parameter server, the model data may include both the encryption sharing model and the loss, which are both derived through gradient data, as such, the model data, especially the encryption sharing mode, includes gradient data), and wherein generating the updated model comprises: updating the dedicated portion of the model based on the local data in the model data to obtain an updated dedicated portion; and generating, based on the updated shared portion and the updated dedicated portion, the updated model (Pg.3, the first terminal can according to the prediction type of the first local neural network model, in the first local neural network model, independently training the first private model, the first private model can be independently trained by the first data, the private model (dedicated portion) is updated by the first data independently (implying the data is local and each party has its own data), given the private model and encrypted sharing model are parts of the same first local neural network, updating both (encrypted sharing model is updated as stated above) generates a new local neural network).
Regarding claim 4, Kang teaches wherein updating the shared portion of the model comprises: using the second shared data as a model parameter in the shared portion of the model (Pg.14, the first terminal can according to the characteristic of the first local neural network model, setting the weight of the encryption migration model in the first local neural network model,
so that the parameter in the encryption migration model can be better fused to the first local neural network model……. The first private model of the participating party k local encryption deep neural network model remains unchanged, the first local neural network (more importantly only the shared encryption model) is updated by setting a weight to the encryption migration model (second shared data) which implies it is used as a model parameter).
Regarding claim 5, Kang teaches wherein updating the shared portion of the model comprises: updating the model parameter in the shared portion of the model based on a learning rate and the second shared data (Pg.14, the first terminal can according to the characteristic of the first local neural network model, setting the weight of the encryption migration model in the first local neural network model, so that the parameter in the encryption migration model can be better fused to the first local neural network model……. The first private model of the participating party k local encryption deep neural network model remains unchanged, Pg. 15, participating party k using gradient descent algorithm…then according to the gradient updating the local depth neural network model, the first local neural network (more importantly only the shared encryption model) is updated by setting a weight to the encryption migration model (second shared data) which implies it is used as a model parameter (which means it updates the model’s parameters) and through gradient descent which implies the use of a learning rate).
Regarding claim 6, Kang teaches wherein the second shared data are a weighted sum value or a weighted average value of the first shared data of the plurality of data parties (Pg.15, the parameter server through weighted average K encrypted sharing model uploaded by the K terminal, determining the encryption migration model of the ith training period, the second shared data (the encryption migration model) is determined through the weighted average of the encrypted sharing models (first shared data of the data parties)).
Regarding claim 7, Kang teaches wherein the model held by each of the plurality of data parties has a same model structure (Pg. 14, For example, can be shown in FIG. 3, encryption sharing model is neural network model front N layer, the first private model is a model (Mk-N) layer…. The encryption sharing models of all participants have the same structure. the first private model of each participant can be set by each participant according to the specific task of itself, the model structure of all participants is the same, an identical encryption sharing model and a second private model that can be set to a specific task).
Regarding claims 8-14, the inventive concept is essentially the same as claims 1-7 except for a non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations. Kang teaches a non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations (Pg.19, Based on the above embodiments, in the embodiment of the present invention, there is provided a computer-readable storage medium, which is stored with a computer program, the computer program when executed by a processor to realize the training and prediction of the local neural network model in any method embodiments).
Regarding claims 15-20, the inventive concept is essentially the same as claims 1-7 except for a computer-implemented system, comprising: one or more computers; and one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising. Kang teaches a computer-implemented system, comprising: one or more computers; and one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising (Pg.19, Those skilled in the art will appreciate that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Thus, the present invention may employ a complete hardware embodiment, a complete software embodiment, or in the form of an embodiment in combination with software and hardware aspects. Furthermore, the present invention can be embodied in the form of a
computer program product embodied in one or more computer usable storage media (including, but not limited to, disk storage, CD-ROM, optical memory, etc.) in which computer usable program code is contained.)
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-3, 6-10, 13-17 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 6-10, 15-20 of U.S. Patent No. 12547766B2. Although the claims at issue are not identical, they are not patentably distinct from each other because the instant application recites broader claims than the referenced patent.
U.S Application 18491515
U.S. Patent No. 12547766B2
Claim 1: A computer-implemented method for training a privacy protection model, comprising: performing a plurality of iterative updates on a model held by a data party of a plurality of data parties participating in training the model, wherein the model comprises a shared portion and a dedicated portion, and performing one of the plurality of iterative updates comprises:
performing one or more times of iterative training on the model based on a training sample held by the data party to obtain model data, wherein the model data comprise first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model;
transmitting the first shared data to a server for the server to determine second shared data based on the first shared data; receiving the second shared data from the server;
updating the shared portion of the model based on the second shared data to obtain an updated shared portion; and generating, based on the updated shared portion, an updated model for performing a next one of the plurality of iterative updates in response to determining that the next one of the plurality of iterative updates is not a last one of the plurality of iterative updates.
Claim 1: A computer-implemented method for privacy preserving model training, comprising:
performing a plurality of iterative update rounds on a model held by a data party of a plurality of data parties participating in training, wherein an iterative update round comprises:
performing, based on a training sample held by the data party of a plurality of data parties, iterative training on the model to train both a shared portion of the model and a dedicated portion of the model to obtain model data, wherein the model data comprises first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model, and wherein the iterative training comprises adding a perturbation to the first shared data corresponding to the shared portion of the model to perform privacy preservation on at least the first shared data in the model data;
transmitting the first shared data to a server, which determines, based on first shared data of the plurality of data parties, second shared data;
obtaining the second shared data returned by the server;
training the shared portion of the model based on the second shared data to obtain an updated model; and performing a next iterative update round based on the updated model or using the updated model as a final model.
Claim 2: wherein the model data comprise a model parameter or gradient data obtained after one or more iterative updates.
Claim 8: wherein the model data comprises:
a model parameter or gradient data obtained after one or more times of iterative training.
Claim 3: wherein the model data comprise the gradient data, and wherein generating the updated model comprises:
updating the dedicated portion of the model based on the local data in the model data to obtain an updated dedicated portion;
and generating, based on the updated shared portion and the updated dedicated portion, the updated model.
Claim 9: The computer-implemented method of claim 8, wherein, when the model data comprises the gradient data obtained after one or more times of iterative training:
updating the dedicated portion of the model based on the local data in the model data.
In claim 1 the shared portion of the model is updated to obtain an updated model and since the dedicated portion of the model is also updated then it’s implied that the update model has both the updated shared portion and the updated dedicated portion
Claim 6: wherein the second shared data are a weighted sum value or a weighted average value of the first shared data of the plurality of data parties
Claim 6: wherein the second shared data is a weighted sum value or a weighted average value of the first shared data of the plurality of data parties.
Claim 7: wherein the model held by each of the plurality of data parties has a same model structure.
Claim 7: wherein models respectively held by the plurality of data parties have a same model structure.
Claim 8: A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:
performing a plurality of iterative updates on a model held by a data party of a plurality of data parties participating in training the model, wherein the model comprises a shared portion and a dedicated portion, and performing one of the plurality of iterative updates comprises:
performing one or more times of iterative training on the model based on a training sample held by the data party to obtain model data, wherein the model data comprise first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model;
transmitting the first shared data to a server for the server to determine second shared data based on the first shared data; receiving the second shared data from the server;
updating the shared portion of the model based on the second shared data to obtain an updated shared portion; and generating, based on the updated shared portion, an updated model for performing a next one of the plurality of iterative updates in response to determining that the next one of the plurality of iterative updates is not a last one of the plurality of iterative updates.
Claim 10: A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform one or more operations for privacy preserving model training, comprising:
performing a plurality of iterative update rounds on a model held by a data party of a plurality of data parties participating in training, wherein an iterative update round comprises:
performing, based on a training sample held by the data party of a plurality of data parties, iterative training on the model to train both a shared portion of the model and a dedicated portion of the model to obtain model data, wherein the model data comprises first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model, and wherein the iterative training comprises adding a perturbation to the first shared data corresponding to the shared portion of the model to perform privacy preservation on at least the first shared data in the model data;
transmitting the first shared data to a server, which determines, based on first shared data of the plurality of data parties, second shared data;
obtaining the second shared data returned by the server;
training the shared portion of the model based on the second shared data to obtain an updated model; and
performing a next iterative update round based on the updated model or using the updated model as a final model.
Claim 9: wherein the model data comprise a model parameter or gradient data obtained after one or more iterative updates.
Claim 17: wherein the model data comprises:
a model parameter or gradient data obtained after one or more times of iterative training.
Claim 10: wherein the model data comprise the gradient data, and wherein generating the updated model comprises:
updating the dedicated portion of the model based on the local data in the model data to obtain an updated dedicated portion;
and generating, based on the updated shared portion and the updated dedicated portion, the updated model.
Claim 18: wherein, when the model data comprises the gradient data obtained after one or more times of iterative training:
updating the dedicated portion of the model based on the local data in the model data.
In claim 10 the shared portion of the model is updated to obtain an updated model and since the dedicated portion of the model is also updated then it’s implied that the update model has both the updated shared portion and the updated dedicated portion
Claim 13: wherein the second shared data are a weighted sum value or a weighted average value of the first shared data of the plurality of data parties.
Claim 15: wherein the second shared data is a weighted sum value or a weighted average value of the first shared data of the plurality of data parties.
Claim 14: wherein the model held by each of the plurality of data parties has a same model structure
Claim 16: wherein models respectively held by the plurality of data parties have a same model structure.
Claim 15: A computer-implemented system, comprising: one or more computers; and one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising:
performing a plurality of iterative updates on a model held by a data party of a plurality of data parties participating in training the model, wherein the model comprises a shared portion and a dedicated portion, and performing one of the plurality of iterative updates comprises:
performing one or more times of iterative training on the model based on a training sample held by the data party to obtain model data, wherein the model data comprise first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model;
transmitting the first shared data to a server for the server to determine second shared data based on the first shared data; receiving the second shared data from the server;
updating the shared portion of the model based on the second shared data to obtain an updated shared portion; and generating, based on the updated shared portion, an updated model for performing a next one of the plurality of iterative updates in response to determining that the next one of the plurality of iterative updates is not a last one of the plurality of iterative updates.
Claim 19: A computer-implemented system, comprising: one or more computers; and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations for privacy preserving model training, comprising:
performing a plurality of iterative update rounds on a model held by a data party of a plurality of data parties participating in training, wherein an iterative update round comprises:
performing, based on a training sample held by the data party of a plurality of data parties, iterative training on the model to train both a shared portion of the model and a dedicated portion of the model to obtain model data, wherein the model data comprises first shared data corresponding to the shared portion of the model and local data corresponding to the dedicated portion of the model, and wherein the iterative training comprises adding a perturbation to the first shared data corresponding to the shared portion of the model to perform privacy preservation on at least the first shared data in the model data;
transmitting the first shared data to a server, which determines, based on first shared data of the plurality of data parties, second shared data;
obtaining the second shared data returned by the server;
training the shared portion of the model based on the second shared data to obtain an updated model; and
performing a next iterative update round based on the updated model or using the updated model as a final model.
Claim 16: wherein the model data comprise a model parameter or gradient data obtained after one or more iterative updates.
Claim 20: determining the model data based on the dedicated gradient data and the shared gradient data obtained added with the perturbation; or
Claim 17: wherein the model data comprise the gradient data, and wherein generating the updated model comprises: updating the dedicated portion of the model based on the local data in the model data to obtain an updated dedicated portion; and generating, based on the updated shared portion and the updated dedicated portion, the updated model.
Claim 20: updating the dedicated portion of the model based on the dedicated gradient data, updating the shared portion of the model based on the shared gradient data obtained added with the perturbation, and performing next iterative training based on an updated model.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NATNAEL A ASEGDEW whose telephone number is (571)270-0407. The examiner can normally be reached 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATNAEL A ASEGDEW/Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights