Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the application filed 01/25/2024. Claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted 01/25/2024 has been considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 1: The claim recites “A method performed at a client node of a federated learning system, the method comprising:”; therefore, it is directed to the statutory category of a process.
Step 2A Prong 1: The claim recites, inter alia:
generating a first loss gradient tensor by training a machine-learning (ML) model using a private data input: These limitations recite mathematical relationships similar to organizing information and manipulating information through mathematical correlations per MPEP 2106.04(a)(2)(I)(A)(iv).
generating a second loss gradient tensor by training the ML model using a randomized data input: These limitations recite mathematical relationships similar to organizing information and manipulating information through mathematical correlations per MPEP 2106.04(a)(2)(I)(A)(iv).
using the first and second loss gradient tensors in an iterative process to update the second loss gradient tensor, the iterative process being repeated until the randomized data input approximates the private data input: These limitations recite mathematical calculations similar to using an algorithm for determining the optimal number of visits by a business representative to a client per MPEP 2106.04(a)(2)(I)(C)(v).
identifying in the updated second loss gradient tensor a plurality of index positions of gradient values that are greater than a p-th percentile: These limitations recite a mentally performable process with the aid of pen and paper of using observation and judgement to identify index positions of gradient values that are greater than a p-th percentile.
and pruning the gradient values that are greater than a p-th percentile from the first loss gradient tensor to thereby generate a third loss gradient tensor that does not include the gradient values that are greater than a p-th percentile: These limitations recite mathematical relationships similar to organizing information and manipulating information through mathematical correlations per MPEP 2106.04(a)(2)(I)(A)(iv).
Thus, the claim recites a judicial exception.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A method performed at a client node of a federated learning system, the method comprising: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. a method, to a particular technological environment or field of use, e.g. performed at a client node of a federated learning system. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 2
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
providing the third loss gradient tensor from the client node to a central server of the federated learning system: These additional elements amount to insignificant extra-solution activity in the form of mere data gathering per MPEP § 2106.05(g).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include insignificant extra-solution activity of data gathering recited by “providing the third loss gradient tensor from the client node to a central server of the federated learning system” which are well-understood routine and conventional activities similar to receiving or transmitting data over a network per MPEP 2106.05(d)(II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 3
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the p-th percentile is a 90-th percentile: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. the p-th percentile, to a particular technological environment or field of use, e.g. is a 90-th percentile. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 4
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the gradient values that are greater than the p-th percentile are gradient values having a high privacy-breaching capacity when a model inversion attack is successfully conducted: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. the gradient values that are greater than the p-th percentile, to a particular technological environment or field of use, e.g. are gradient values having a high privacy-breaching capacity when a model inversion attack is successfully conducted. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 5
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the p-th percentile is selected based on a configuration, data, and training parameters of the federation learning system and then experimentally validated: These additional elements amount to insignificant extra-solution activity in the form of selecting a particular data source or type of data to be manipulated per MPEP § 2106.05(g).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include insignificant extra-solution activity of data gathering recited by “wherein the p-th percentile is selected based on a configuration, data, and training parameters of the federation learning system and then experimentally validated” which are well-understood routine and conventional activities similar to presenting offers and gathering statistics per MPEP 2106.05(d)(II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 6
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of claim 1 as well as, inter alia:
wherein using the first and second loss gradient tensors in the iterative process to update the second loss gradient tensor comprises calculating an inversion loss: These limitations recite mathematical calculations similar to an act of calculating using mathematical methods to determine a variable or number per MPEP 2106.04(a)(2)(I)(C).
Thus, the claim recites a judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a
practical application and is not considered to be significantly more. As such, the claim is patent
ineligible.
Claim 7
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of claim 6 as well as, inter alia:
PNG
media_image1.png
214
746
media_image1.png
Greyscale
These limitations recite the use of mathematical formulas or equations similar to a formula for computing an alarm limit per MPEP 2106.04(a)(2)(I)(C).
Thus, the claim recites a judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a
practical application and is not considered to be significantly more. As such, the claim is patent
ineligible.
Claim 8
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of claim 1 as well as, inter alia:
wherein pruning the gradient values that are greater than a p-th percentile from the first loss gradient tensor comprises zeroing out the index positions of gradient values that are greater than a p-th percentile: These limitations recite mathematical relationships similar to organizing information and manipulating information through mathematical correlations per MPEP 2106.04(a)(2)(I)(A)(iv).
Thus, the claim recites a judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a
practical application and is not considered to be significantly more. As such, the claim is patent
ineligible.
Claim 9
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the method results in a Peak Signal-to-Noise Ratio (PSNR) and a Structural Similarity Index (SSIM) that are less than a PSNR and SSIM resulting from a random pruning operation and a gradient compression operation: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. the method, to a particular technological environment or field of use, e.g. results in a Peak Signal-to-Noise Ratio (PSNR) and a Structural Similarity Index (SSIM) that are less than a PSNR and SSIM resulting from a random pruning operation and a gradient compression operation. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 10
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the method results in a lower loss and a higher accuracy than a loss or accuracy resulting from a random pruning operation and a gradient compression operation: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. the method, to a particular technological environment or field of use, e.g. results in a lower loss and a higher accuracy than a loss or accuracy resulting from a random pruning operation and a gradient compression operation. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claims 11-20
Step 1: This claim recites " A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:"; therefore, it is directed to the statutory category of an article of manufacture.
Step 2A Prong 1: Claims 11-20 recite the same judicial exception as Claims 1-10, respectively.
Step 2A Prong 2: The judicial exception recited in these claims are not integrated into a practical application. The only difference between Claims 11-20 and Claims 1-10, is that Claims 11-20 are directed to " A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). With that exception, the analysis at this step for Claims 11-20 mirrors that of Claims 1-10, respectively.
Step 2B: The additional elements from Step 2A Prong 2 do not contain significantly more than the judicial exception for these claims. The only difference between Claims 11-20 and Claims 1-10, is that Claims 11-20 are directed to " A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising, cannot amount to significantly more than the judicial exception. See MPEP 2106.05(f). With that exception, the analysis at this step for Claims 11-20 mirrors that of Claims 1-10, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-6, 8-11, 12-16, & 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (“Deep Leakage from Gradients”, 33rd Conference on Neural Information Processing Systems, arXiv) (2019), hereafter Zhu, in view of Zhang et al. (“Preserving data privacy in federated learning through large gradient pruning”, Computers & Security 125, ScienceDirect) (2023), hereafter Zhang.
Zhu was cited in IDS filed 01/25/2024.
Regarding independent claim 1, Zhu teaches a method performed at a client node of a federated learning system ([Abstract] discusses exchanging gradients is a widely used method in multi-node machine learning systems; [Introduction] discusses each client node within a multi-node machine learning system has its own training data and only communicates gradient during training), comprising:
generating a first loss gradient tensor by training a machine-learning (ML) model using a private data input ([Sec. 3] discusses computing a first loss gradient tensor where x is represented by private training data);
generating a second loss gradient tensor by training the ML model using a randomized data input ([Sec. 3.1] discusses randomly initializing dummy input x and dummy label input y, then using them in a second loss gradient tensor;
using the first and second loss gradient tensors in an iterative process to update the second loss gradient tensor, the iterative process being repeated until the randomized data input approximates the private data input ([Alg. 1, Steps 3-7] discusses a loop wherein the dummy data is updated each iteration by minimizing distance between the dummy gradient and the real gradient and this is repeated until the data matches and converges).
Zhu does not explicitly teach identifying in the updated second loss gradient tensor a plurality of index positions of gradient values that are greater than a p-th percentile; and pruning the gradient values that are greater than a p-th percentile from the first loss gradient tensor to thereby generate a third loss gradient tensor that does not include the gradient values that are greater than a p-th percentile.
However, in the same field of endeavor, Zhang teaches a method for identifying a plurality of index positions of the second loss with gradient values that are greater than a p-th percentile ([Sec. 3.2 & Alg. 1] discusses identifying gradient positions that are greater than a defined threshold through a comparison step, and thus, the method identifies a plurality of index positions of gradient values that are greater than a p-th percentile); and pruning the gradient values greater than a p-th percentile to generate a third loss gradient tensor ([Sec. 3.2 & Alg. 1] discusses pruning gradient values that are greater than a defined threshold and generating the resulting third loss gradient tensor that does not include the gradient values that were greater than a p-th percentile).
Because Zhu teaches generating a first loss gradient tensor using a private data input, generating a second loss gradient tensor using randomized input, and using both loss gradient tensors to update the second loss gradient tensor; and Zhang teaches identifying index positions of the second loss with gradient values greater than a p-th percentile, and pruning those values to generate a third loss gradient tensor, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate identifying and pruning index positions of the second loss gradient tensor with gradient values that are greater than a p-th percentile to generate a third loss gradient tensor as taught by Zhang into Zhu’s method, with a reasonable expectation of success, to teach generating a first loss gradient tensor by training a machine-learning (ML) model using a private data input; generating a second loss gradient tensor by training the ML model using a randomized data input; using the first and second loss gradient tensors in an iterative process to update the second loss gradient tensor, the iterative process being repeated until the randomized data input approximates the private data input; identifying in the updated second loss gradient tensor a plurality of index positions of gradient values that are greater than a p-th percentile; and pruning the gradient values that are greater than a p-th percentile from the first loss gradient tensor to thereby generate a third loss gradient tensor that does not include the gradient values that are greater than a p-th percentile. This combination would have been motivated by the desire to avoid gradient-based inversion attacks by pruning because processed gradients are difficult to be exploited by gradient-based inversion attacks, and the sensitive information in original images is protected (Zhang [Sec. 3.2]).
Regarding dependent claim 3, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including wherein the p-th percentile is a 90-th percentile (Zhang [Sec. 5.1] discusses using a threshold of 90%).
Regarding dependent claim 4, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including wherein the gradient values that are greater than the p-th percentile are gradient values having a high privacy-breaching capacity when a model inversion attack is successfully conducted (Zhang [Sec. 1] discusses large magnitude gradients typically carry more information associated with training data and thus, carry a higher privacy-breaching capacity during an inversion attack).
Regarding dependent claim 5, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including wherein the p-th percentile is selected based on a configuration, data, and training parameters of the federation learning system and then experimentally validated (Zhang [Sec. 5.1-5.2 & Fig. 5] discusses the system is able to set the p-th percentile threshold value based on the default metrics, training data, or goals of the system; and the different thresholds are experimentally validated).
Regarding dependent claim 6, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including wherein using the first and second loss gradient tensors in the iterative process to update the second loss gradient tensor comprises calculating an inversion loss (Zhu [Alg. 1, Steps 3-7] discusses an iterative loop wherein the dummy data is updated each iteration; Zhu [Sec. 3.1 & Eq. 4] discusses calculating an inversion loss for each gradient in a given iteration).
Regarding dependent claim 8, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including wherein pruning the gradient values that are greater than a p-th percentile from the first loss gradient tensor comprises zeroing out the index positions of gradient values that are greater than a p-th percentile (Zhang [Sec. 3.4] discusses pruning the large magnitude gradients to 0 at each position that has been found to be above a p-th threshold).
Regarding dependent claim 9, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including wherein the method results in a Peak Signal-to-Noise Ratio (PSNR) and a Structural Similarity Index (SSIM) that are less than a PSNR and SSIM resulting from a random pruning operation and a gradient compression operation (Zhang [Table 2] discusses that the method results in both a PSNR and SSIM that is less than the PSNR and SSIM of random pruning and gradient compression operations).
Regarding dependent claim 10, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including wherein the method results in a lower loss and a higher accuracy than a loss or accuracy resulting from a random pruning operation and a gradient compression operation (Zhang [Sec. 5] discusses the process of random pruning using MID can reduce the quality of reconstructed images while still not effectively preventing leakage; Zhu [Sec. 5.2] discusses that in gradient compression, gradients with small magnitudes are pruned to zero; however, that leaves the risk of high magnitude gradients being leaked and thus, the method of inversion gradient pruning results in a lower loss and a higher accuracy than both random pruning and gradient compression).
Regarding claims 11, 13-16, & 18-20, claims 11, 13-16, & 18-20 are non-transitory computer-readable storage medium claims that are substantially the same as the method of Claims 1, 3-6, & 8-10, respectively. Therefore, claims 11, 13-16, & 18-20 are rejected for the same reasons as Claims 1, 3-6, & 8-10, respectively.
Claims 2 & 12 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu, in view of Zhang, as applied in claims 1 & 11, and further in view of McMahan et al. (“Communication-Efficient Learning of Deep Networks from Decentralized Data”, 20th International Conference on Artificial Intelligence and Statistics, arXiv) (2017), hereafter McMahan.
Regarding dependent claim 2, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 1, including generating a third loss gradient tensor (Zhang [Sec. 3.2 & Alg. 1] discusses pruning gradient values that are greater than a defined threshold and generating the resulting third loss gradient tensor that does not include the gradient values that were greater than a p-th percentile).
The combination of Zhu and Zhang does not explicitly teach providing the third loss gradient tensor from the client node to a central server of the federated learning system.
However, in a similar field of endeavor, McMahan teaches the architecture of a client node providing a gradient tensor to a central server ([Sec. 2 & Alg. 1] discusses the client node providing the results to the central server of the federated learning system).
Because the combination of Zhu and Zhang teaches generating a third loss gradient tensor, and McMahan teaches a client node providing results to a central server of the federated learning system, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the client node providing the results to a central server as taught by McMahan into the combination of Zhu and Zhang’s method, with a reasonable expectation of success, to teach providing the third loss gradient tensor from the client node to a central server of the federated learning system. This combination would have been motivated by the desire such that each client has a local training dataset which is never uploaded to the server. Instead, each client computes an update to the current global model maintained by the server, and only this update is communicated (McMahan [Introduction]).
Regarding dependent claim 12, claim 12 is a non-transitory computer-readable storage medium claim that is substantially the same as the method of Claim 2. Therefore, claim 12 is rejected for the same reasons as Claim 2.
Claims 7 & 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu, in view of Zhang, as applied in claims 6 & 16, and further in view of Geiping et al. (“Inverting Gradients - How easy is it to break privacy in federated learning?”, Dep. of Electrical Engineering and Computer Science, University of Siegen, arXiv) (2020), hereafter Geiping.
Geiping was cited in IDS filed 01/25/2024.
Regarding dependent claim 7, the combination of Zhu and Zhang teaches the claimed invention as claimed in claim 6, including calculating an inversion loss using the first and second loss gradient tensors (Zhu [Alg. 1, Steps 3-7] discusses an inversion loss loop wherein the dummy data is updated each iteration by minimizing distance between the dummy gradient and the real gradient and this is repeated until the data matches and converges).
The combination of Zhu and Zhang does not explicitly teach the inversion loss is found by the following equation:
PNG
media_image1.png
214
746
media_image1.png
Greyscale
However, in a similar field of endeavor, Geiping teaches the use of the claimed inversion loss equation ([Sec. 4 & Eq. 4] discusses the use of an equivalent equation to calculate the inversion loss).
Because the combination of Zhu and Zhang teaches iteratively updating the second loss gradient tensor using inversion loss, and Geiping teaches inversion loss being found using the claimed equation, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the inversion loss equation as taught by Geiping into the combination of Zhu and Zhang’s method, with a reasonable expectation of success, to teach the inversion loss is found by the following equation:
PNG
media_image1.png
214
746
media_image1.png
Greyscale
This combination would have been motivated by the desire to minimize eq. (4) only based on the sign of its gradient, which we optimize with Adam [17] with step size decay so the actual update step still being unsigned based on accumulated momentum, so that an image can still be accurately recovered (Geiping [Sec. 4]).
Regarding dependent claim 17, claim 17 is a non-transitory computer-readable storage medium claim that is substantially the same as the method of Claim 7. Therefore, claim 17 is rejected for the same reasons as Claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bhalgat et al. (US 2022/0261648 A1, published 08/18/2022) ([0005] a method to train a machine learning model using gradient pruning, comprising: computing, using a first batch of training data, a first gradient tensor comprising a gradient for each parameter of a parameter tensor for a machine learning model; identifying a first subset of gradients in the first gradient tensor based on a first gradient criteria; and updating a first subset of parameters in the parameter tensor based on the first subset of gradients; [0036] the pruning component 120 utilizes a percentile-based threshold for the gradient values to generate the indices 125. To do so, the pruning component 120 may first sort the gradients in the gradient tensor 115A based on their absolute value or magnitude (e.g., the distance from zero)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RILEY S ACOSTA whose telephone number is (571)272-8714. The examiner can normally be reached Monday-Thursday 6am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer N Welch can be reached at (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RILEY S ACOSTA/Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights