Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the following communication: Non-Provisional Application filed Mar. 6, 2024.
Claims 1-20 are pending in the case. Claims 1, 13 and 17 are independent claims.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
As to claim 1:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Perkins US 2022/0253647 Para [38] – “the machine learning model and/or one or more sets of inference data may be reviewed manually to evaluate the machine learning model and/or the one or more sets of inference data and/or to determine whether the one or more sets of inference data are compatible with the machine learning model.”
Yes, the limitation “receiving a first local model and a first random quantization instruction; training the first local model to obtain a model update amount” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “performing, based on the first random quantization instruction, random quantization on the model update amount to obtain a quantization model update amount; encoding the quantization model update amount to obtain encoded data; sending the encoded data” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “receiving, in response to sending the encoded data, a second local model and a second random quantization instruction to perform, until a global model converges, iterative updating.” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “training the first local model … ” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No
As to claim 13:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “the first global model comprises a plurality of local models, and wherein the plurality of local models are in one-to-one correspondence with the plurality of terminals” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “a random quantization instruction; receive, from the plurality of terminals, encoded data; decode the encoded data to obtain quantization model update amounts of the plurality of terminals; aggregate the quantization model update amounts to obtain a second global model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “the second global model to instruct the plurality of terminals to perform iterative updating until the second global model converges” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “a memory configured to store instructions; and one or more processors coupled to the memory and configured to execute the instructions to: deliver, to a plurality of terminals … deliver … receive” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No
As to claim 1:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a Machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “receiving a first local model and a first random quantization instruction; training the first local model to obtain a model update amount” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “performing, based on the first random quantization instruction, random quantization on the model update amount to obtain a quantization model update amount; encoding the quantization model update amount to obtain encoded data; sending the encoded data” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “receiving, in response to sending the encoded data, a second local model and a second random quantization instruction to perform, until a global model converges, iterative updating.” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “a memory configured to store instructions; and one or more processors coupled to the memory and configured to execute the instructions to: receive, from a server” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No
Claims 2-12 are dependent on claim 1 and includes all the limitations of claim 1. Therefore, claims 2-12 recite the same abstract idea. Claims 14-16 are dependent on claim 13 and includes all the limitations of claim 13. Therefore, claims 14-16 recite the same abstract idea. Claims 18-20 are dependent on claim 17 and includes all the limitations of claim 17. Therefore, claims 18-20 recite the same abstract idea. The claims recite additional limitations directed to mathematic calculation, but do not otherwise add any meaningful limits beyond the abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zong et al. (hereinafter Zong) “Communication Reducing Quantization for Federated Learning with Local Differential Privacy Mechanism” 2021 in view of Du et al. (hereinafter Du) U.S. Patent No. 11,017,322.
With respect to independent claim 1, Zong teaches a method, comprising:
receiving a first local model and a first random quantization instruction; training the first local model to obtain a model update amount (see e.g., Section II.B – “Model aggregation and broadcasting: The server decodes the received … and recovers the discrete value into the estimated value … Then the server performs model aggregation according to (2) and broadcasts the new global model”); performing, based on the first random quantization instruction, random quantization on the model update amount to obtain a quantization model update amount (see e.g., Section II.B – Step 3 and 4); encoding the quantization model update amount to obtain encoded data (see e.g., Section II.B – Step 3 and 4); sending the encoded data (see e.g., Section II.B – “Then the server performs model aggregation according to(2)and broadcasts the new global model”); and receiving, in response to sending the encoded data, a second local model and a second random quantization instruction to perform, until a global model converges, iterative updating (see e.g., Section II.B Step 2-5 are repeated until the training procedure converges or the loss is lower than a preset threshold).
Zong does not expressly show the quantization is random quantization. However, Du discloses similar feature (see e.g., Col. 8 line 35 – Col. 10 line 35 - “directly applying the above quantization would result in deterministic quantization error/noise. In some embodiments, a dither noise vector v.sub.k,tϵR.sup.d may be added to the quantization to randomize the quantization effects. The dither noise v.sub.k,t may be independent of the g.sub.k,t and follows a uniform distribution. In some embodiments, the dither noise vector v.sub.k,t may include a plurality of scalar value to be respectively added to each coordinate in the g.sub.k,t.”). Both Zong and Du are directed to federated learning process. Accordingly, it would have been obvious to the skilled artisan before the effective filing date of the claimed invention having Zong and Du in front of them to modify the system of Zong to include the above feature. The motivation to combine Zong and Du comes from Du. Du discloses the motivation to reduce deterministic quantization error by randomizing the quantization effect (see e.g. col. 10). This motivation for combination also applies to the remaining claims which depend on this combination.
With respect to dependent claim 2, the modified Zong teaches the first random quantization instruction comprises random step quantization instruction (see e.g., Algorithm 1, 1.15-22 and Du col. 8-10).
With respect to dependent claim 3, the modified Zong teaches performing the random quantization further comprises performing, by using a random step quantization method, the random quantization (see e.g., Algorithm 1, 1.15-22 Du col. 8 and 10).
With respect to dependent claim 4, the modified Zong teaches the random step quantization method comprises random quantization steps that are randomly and evenly distributed (see e.g., Algorithm 1, 1.15-22 and page 78 – “The dither vectors … are randomized from a uniform distribution over …”).
With respect to dependent claim 5, the modified Zong teaches performing the random quantization comprises further performing, by using a random quantizer, the random quantization (see e.g., Algorithm 1, 1.15-22 and Du col. 8 and 10).
With respect to dependent claim 6, the modified Zong teaches the random quantizer comprises an upward quantizer and a downward quantizer (Zong-Du does not expressly show the “upward quantizer” and “downward quantizer.” However, this recited feature is merely a design/implementation choice, therefore it would have been obvious for a person of ordinary skill in the art to try).
With respect to dependent claim 7, the modified Zong teaches the first random quantization instruction comprises a random quantizer instruction, and wherein performing the random quantization comprises performing, based on the random quantizer instruction, the random quantization (see e.g., Du col. 8 and 10 – “a dither noise vector v.sub.k,tϵR.sup.d may be added to the quantization to randomize the quantization effects”).
With respect to dependent claim 8, the modified Zong teaches performing the random quantization further comprises further performing, by using a random step quantization method, the random quantization (see e.g., Algorithm 1, 1.15-22 and Du col. 8 and 10 – “a dither noise vector v.sub.k,tϵR.sup.d may be added to the quantization to randomize the quantization effects”).
With respect to dependent claim 9, the modified Zong teaches the random step quantization method comprises random quantization steps that are randomly and evenly distributed (see e.g., Algorithm 1, 1.15-22 and page 78 – “The dither vectors … are randomized from a uniform distribution over …”).
With respect to dependent claim 10, the modified Zong teaches performing the random quantization further comprises further performing, by using a random quantizer, the random quantization (see e.g., Algorithm 1, 1.15-22 Du col. 8 and 10).
With respect to dependent claim 11, the modified Zong teaches the random quantizer comprises an upward quantizer and a downward quantizer (Zong-Du does not expressly show the “upward quantizer” and “downward quantizer.” However, this recited feature is merely a design/implementation choice, therefore it would have been obvious for a person of ordinary skill in the art to try).
With respect to dependent claim 12, the modified Zong teaches using a plurality of types of third random quantization instructions in the iterative updating (see e.g., Section II.B Step 2-5).
With respect to independent claim 13, the modified Zong teaches an apparatus, comprising: a memory configured to store instructions; and one or more processors coupled to the memory and configured to execute the instructions to:
deliver, to a plurality of terminals, a first global model, wherein the first global model comprises a plurality of local models, and wherein the plurality of local models are in one-to-one correspondence with the plurality of terminals(see e.g., Section II.B – Step 1); deliver, to the plurality of terminals, a random quantization instruction (see discussion above with respect to claim 1); receive, from the plurality of terminals, encoded data (see e.g., Section II.B – Step 4); decode the encoded data to obtain quantization model update amounts of the plurality of terminals (see e.g., Section II.B – Step 5 Algorithm 1); aggregate the quantization model update amounts to obtain a second global model (see e.g., Page 76 and Algorithm 2); and deliver, to the plurality of terminals, the second global model to instruct the plurality of terminals to perform iterative updating until the second global model converges (see e.g., Section II.B – Steps 2-5).
With respect to dependent claim 14, the modified Zong teaches the random quantization instruction comprises a random step quantization instruction or a random quantizer instruction, and wherein the random step quantization instruction and the random quantizer instruction are for obtaining the quantization model update amounts by using a random step quantization method or a random quantizer (see e.g., Algorithm 1, 1.15-22 Du col. 8 and 10).
With respect to dependent claim 15, the modified Zong teaches the random step quantization method comprises random quantization steps that are randomly and evenly distributed (see e.g., Algorithm 1, 1.15-22 and page 78 – “The dither vectors … are randomized from a uniform distribution over …”).
With respect to dependent claim 16, the modified Zong teaches the random quantizer comprises an upward quantizer and a downward quantizer (Zong-Du does not expressly show the “upward quantizer” and “downward quantizer.” However, this recited feature is merely a design/implementation choice, therefore it would have been obvious for a person of ordinary skill in the art to try).
Claim 17 is rejected for the similar reasons discussed above with respect to claim 1.
With respect to dependent claim 18, the modified Zong teaches the first random quantization instruction comprises a random step quantization instruction or a random quantizer instruction, and wherein the one or more processors are further configured to execute the instructions to perform, based on the random step quantization instruction or the random quantizer instruction and by using a random step quantization method or a random quantizer, the random quantization (see e.g., Algorithm 1, 1.15-22 Du col. 8 and 10).
With respect to dependent claim 19, the modified Zong teaches the random step quantization method comprises random quantization steps that are randomly and evenly distributed (see e.g., Algorithm 1, 1.15-22 and page 78 – “The dither vectors … are randomized from a uniform distribution over …”).
With respect to dependent claim 20, the modified Zong teaches the random quantizer comprises an upward quantizer and a downward quantizer, and wherein a first quantity of upward quantizers is the same as a second quantity of downward quantizers (Zong-Du does not expressly show the “upward quantizer” and “downward quantizer.” However, this recited feature is merely a design/implementation choice, therefore it would have been obvious for a person of ordinary skill in the art to try).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co. v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert. denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir. 2005); Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEIYONG WENG whose telephone number is (571)270-1660. The examiner can normally be reached on Mon.-Fri. 8 am to 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Matthew Ell, can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://portal.uspto.gov/external/portal. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/PEI YONG WENG/Primary Examiner, Art Unit 2141
Prosecution Insights