DETAILED ACTION
This application is in response to claims filed 28 June 2024 for application 18759671 filed 28 June 2024. Currently claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-9 and 11-20 are rejected under 35 U.S.C. 102(A)(1) as being anticipated by Bellala et al. (US 20180218171 A1).
Regarding claims 1, 11 and 20, Bellala discloses: A method comprising:
receiving, by at least one receiving device, first data and an indicator of second data that was transmitted in at least one message from at least one transmitting device, the second data being omitted from the at least one message (“In the anonymization phase 170, the mediator M 140 sends the set of prepared data to the Nth party (e.g., party P3 130). Here, the Nth party indicates the party to which the last shared public key belongs to. Then, the Nth party can perform decryption, then shuffle on the prepared data set and send randomly shuffled data to the (N−1)th party (e.g., party P2 120). The (N−1)th party can then further decrypt and shuffle the data segments, and this process continues until the data is decrypted and shuffled by the 1st party (e.g., party P1 110). Finally, the mediator M 140 receives anonymized data from the 1st party (e.g., P1 110) with only one layer of encryption using the mediator M 140's public key.” [0021], “Under a secure sum protocol, each party P1 110, P2 120 and P3 130 divides its own data (e.g., a value) into multiple data segments, which are represented by the shaded blocks in FIGS. 1A-1B. The segmented data may be encrypted using at least one public key from L 150 before being shared with other parties and/or non-trusted mediator M 140. Under the protocol described herein, parties generally do not transmit data to another party (P1 110, P2 120, or P3 130) or non-trusted mediator M 140 without any encryption. Data from each party may generally go through three distinct phases under the secure protocol that involves at least data initialization 160, anonymization 170, and sum computation 180.” [0019], note: encrypted data replaces raw data which is interpreted as omitted data);
in response to receiving the at least one message and based on the indicator of the second data, generating a representation of an original data from the at least one transmitting device, the original data comprising the first data and the second data (“In the anonymization phase 170, the mediator M 140 sends the set of prepared data to the Nth party (e.g., party P3 130). Here, the Nth party indicates the party to which the last shared public key belongs to. Then, the Nth party can perform decryption, then shuffle on the prepared data set and send randomly shuffled data to the (N−1)th party (e.g., party P2 120). The (N−1)th party can then further decrypt and shuffle the data segments, and this process continues until the data is decrypted and shuffled by the 1st party (e.g., party P1 110). Finally, the mediator M 140 receives anonymized data from the 1st party (e.g., P1 110) with only one layer of encryption using the mediator M 140's public key.” [0021]); and
processing, by the at least one receiving device, the representation of the original data (“As used herein, “shuffling on the prepared data set” generally involves randomly rearranging the order of data segments within the prepared data set. As used herein, “anonymizing data” generally refers to removing identifying attributes from the data, for example, by reordering and/or encrypting the data that were owned by multiple participating parties. As used herein, a “party” generally refers to a data entity as represented by at least one computing device hosting a private data set. As used herein, a “mediator” generally refers to a non-trusted entity as represented by at least one computing device with which each participating party is willing to share its data set in an encrypted form. Note that because the “mediator” here is a non-trusted entity, parties will not share their private data set with the mediator in plain text form.” [0022]), wherein the processing comprises:
performing one or more operations using the first data as an input (“In the anonymization phase 170, the mediator M 140 sends the set of prepared data to the Nth party (e.g., party P3 130). Here, the Nth party indicates the party to which the last shared public key belongs to. Then, the Nth party can perform decryption, then shuffle on the prepared data set and send randomly shuffled data to the (N−1)th party (e.g., party P2 120). The (N−1)th party can then further decrypt and shuffle the data segments, and this process continues until the data is decrypted and shuffled by the 1st party (e.g., party P1 110). Finally, the mediator M 140 receives anonymized data from the 1st party (e.g., P1 110) with only one layer of encryption using the mediator M 140's public key.” [0021]);
refraining from performing the one or more operations using the second data as an input [0021];
performing at least one aggregate operation based at least in part on a result of performing the one or more operations on the first data and on a quantity of the second data that was omitted from the at least one message (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]); and
storing a result of the at least one aggregate operation (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]).
Regarding claims 2 and 12, Bellala discloses: The method of claim 1, wherein:
the original data comprises a plurality of training gradients for a distributed machine learning system (“Note that, in each use case, the cost function and the update operation in the gradient descent technique involve a summation over the training data.” [0030], “This dataset is distributed across multiple parties, e.g., the m training samples are distributed across N parties as {m.sub.1, m.sub.2, . . . m.sub.N}, where m.sub.i denotes the number of samples with party i” [0042]);
the at least one transmitting device comprises at least one worker of the distributed machine learning system (“First, each party P1 110, P2 120, P3 130 and the non-trusted mediator M 140 generates a pair of keys, including a public key Y and a private key R. Each party may share its public keys with the non-trusted mediator M 140 and other parties, but may store its private key such that the private key is only known to the party creating the key. As illustrated in FIG. 1A, P1 110 generates a public key Y.sub.1 and a private key R.sub.1; P2 120 generates a public key Y.sub.2 and a private key R.sub.2; and, P3 130 generates a public key Y.sub.3 and a private key R.sub.3. Furthermore, P1 110, P2 120, and P3 130 may share with non-trusted mediator M 140 their respective public key Y.sub.1 115, Y.sub.2 125, and Y.sub.3 135.” [0017]);
the at least one receiving device comprises an aggregator that aggregates training gradients received from workers of the distributed machine learning system (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]);
performing the one or more operations using the first data as an input comprises processing, via at least one processor of the aggregator, the training gradients included in the first data (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]);
the method further comprises calculating an offset value to account for the training gradients included in the second data that were omitted from the at least one message (“As used herein, “shuffling on the prepared data set” generally involves randomly rearranging the order of data segments within the prepared data set. As used herein, “anonymizing data” generally refers to removing identifying attributes from the data, for example, by reordering and/or encrypting the data that were owned by multiple participating parties. As used herein, a “party” generally refers to a data entity as represented by at least one computing device hosting a private data set. As used herein, a “mediator” generally refers to a non-trusted entity as represented by at least one computing device with which each participating party is willing to share its data set in an encrypted form. Note that because the “mediator” here is a non-trusted entity, parties will not share their private data set with the mediator in plain text form.” [0022], “The collusion-resistant data anonymization may be achieved by randomly permuting the data (or segments of the data) submitted by the multiple parties. This anonymization provides that the mediator along with the colluding semi-honest parties cannot breach an honest party's data anonymity as long as the adversary has control over at most (N−2) parties.” [0015], note: the permutation is interpreted as an offset);
performing the at least one aggregate operation on the result of performing the one or more operations on the first data and on the quantity of the second data comprises performing the at least one aggregate operation based on the result of processing the training gradients included in the first data and the offset value that accounts for the training gradients included in the second data that were omitted from the at least one message (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]); and
the method further comprises transmitting, to the at least one worker, a gradient update based on the result of the at least one aggregate operation (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]).
Regarding claims 3 and 13, Bellala discloses: . The method of claim 2, wherein performing one or more operations using the first data as an input comprises:
evenly distributing the plurality of training gradients of the first data to a plurality of compute elements of the at least one receiving device message (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]); and
performing the one or more operations using the first data by performing the one or more operations across the plurality of compute elements message (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]).
Regarding claims 4 and 14, Bellala discloses: The method of claim 1, further comprising tracking, based on the indicator of second data, a quantity of second data omitted from the at least one message (“In the anonymization phase 170, the mediator M 140 sends the set of prepared data to the Nth party (e.g., party P3 130). Here, the Nth party indicates the party to which the last shared public key belongs to. Then, the Nth party can perform decryption, then shuffle on the prepared data set and send randomly shuffled data to the (N−1)th party (e.g., party P2 120). The (N−1)th party can then further decrypt and shuffle the data segments, and this process continues until the data is decrypted and shuffled by the 1st party (e.g., party P1 110). Finally, the mediator M 140 receives anonymized data from the 1st party (e.g., P1 110) with only one layer of encryption using the mediator M 140's public key.” [0021]).
Regarding claims 5 and 15, Bellala discloses: The method of claim 4, wherein tracking the quantity of second data omitted from the at least one message comprises incrementing, based on the indicator of the second data, a register that tracks the quantity of second data omitted from the at least one message (“In the anonymization phase 170, the mediator M 140 sends the set of prepared data to the Nth party (e.g., party P3 130). Here, the Nth party indicates the party to which the last shared public key belongs to. Then, the Nth party can perform decryption, then shuffle on the prepared data set and send randomly shuffled data to the (N−1)th party (e.g., party P2 120). The (N−1)th party can then further decrypt and shuffle the data segments, and this process continues until the data is decrypted and shuffled by the 1st party (e.g., party P1 110). Finally, the mediator M 140 receives anonymized data from the 1st party (e.g., P1 110) with only one layer of encryption using the mediator M 140's public key.” [0021]).
Regarding claims 6 and 16, Bellala discloses: The method of claim 4, wherein tracking the quantity of second data omitted from the at least one message comprises shifting, based on the indicator of the second data, a storage location in memory of an output of the one or more operations (“In the anonymization phase 170, the mediator M 140 sends the set of prepared data to the Nth party (e.g., party P3 130). Here, the Nth party indicates the party to which the last shared public key belongs to. Then, the Nth party can perform decryption, then shuffle on the prepared data set and send randomly shuffled data to the (N−1)th party (e.g., party P2 120). The (N−1)th party can then further decrypt and shuffle the data segments, and this process continues until the data is decrypted and shuffled by the 1st party (e.g., party P1 110). Finally, the mediator M 140 receives anonymized data from the 1st party (e.g., P1 110) with only one layer of encryption using the mediator M 140's public key.” [0021], note: after each encryption and worker change the storage location changes which holds the local data and encrypted data).
Regarding claims 7 and 17, Bellala discloses:
The method of claim 1, further comprising transmitting, from the at least one receiving device to the at least one transmitting device, prior to receiving the at least one message, a segment size parameter that indicates a number of portions of data that can be simultaneously processed by the at least one receiving device (“Note also that although the gradient descent technique is described in details in this disclosure, the gradient descent technique can be replaced with any other variants, such as, the mini-batch gradient descent, the stochastic gradient descent, etc. As an example, in case of the mini-batch gradient descent, the parties may decide a-priori on the batch size b. Then, each party can use a random sub-sample of size b from their data (rather than the entire data) to compute the local gradient and the cost function value. Similarly, the mediator can normalize the global gradient and global cost function value using N*b rather than m.” [0050]).
Regarding claims 8 and 18, Bellala discloses: The method of claim 1, wherein performing the one or more operations that use the first data as input comprises performing a mathematical operation indicated by a header of the at least one message (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023], see also [0050]).
Regarding claims 9 and 19, Bellala discloses: The method of claim 1, further comprising transmitting, by the at least one receiving device, a result of processing the at least one message to the at least one transmitting device (“In sum computation phase 180, the mediator M 140 can decrypt the data segments using its own secret key and computes the sum of the values from the N parties. Then, the mediator M 140 can share the sum with the N parties (e.g., P1 110, P2 120, and P3 130).” [0023]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Bellala in view of Sharma et al. (US 20210158167 A1)
Regarding claim 10, Bellala does not explicitly disclose, however, Sharma teaches: The method of claim 1, wherein the indicator of the second data comprises a volume of data that is less than a volume of data of the second data (“In various embodiments, the compression and encryption module 32 is configured to, such as before delivering the plurality of local parameter gradients, compress the plurality of local parameter gradients at the plurality of local computers, and, such as after delivering the plurality of local parameter gradients, decompress the compressed plurality of local parameter gradients at the central computer. In some examples, the compression and encryption module 32 is configured to, such as before delivering the plurality of local parameter gradients, compress the plurality of local parameter gradients at the plurality of local computers, and, such as after delivering the plurality of local parameter gradients, decompress the compressed plurality of local parameter gradients at the plurality of local computers.” [0039]).
Bellala and Sharma are in the same field of endeavor of distributed learning and are analogous. Bellala discloses a system having privacy preserving transmission between workers and the aggregator of data and gradients. Sharma discloses a system wherein gradients in transmissions are encrypted and compressed. It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the known indicator of omitted data as disclosed by Bellala with the known smaller volume indicator as taught by Sharma to yield predictable results.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC NILSSON whose telephone number is (571)272-5246. The examiner can normally be reached M-F: 7-3.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at (571)-272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERIC NILSSON/ Primary Examiner, Art Unit 2151
