CTNF 18/496,177 CTNF 98442 DETAILED ACTION This action is responsive to the claims filed on 10/12/2023. Claims 1-20 are pending for examination. Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Information Disclosure Statement The information disclosure statement (IDS) submitted on 01/06/2025 and 10/23/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Priority 02-27 Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Chinese Application No. 202110475677.0, filed on April 29, 2021. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-7, 9-17, and 19-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Statutory Categories Claims 1-7 are directed to a method. Claims 9-10 are directed to a method. Claims 12-17 are directed to an apparatus. Claims 19-20 are directed to an apparatus. Independent Claims – Claims 1 and 11 Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon? Yes. Independent claims 1 and 11 recites limitations that are abstract ideas in the form of mental processes: Claim 1 recites : training the first neural network model based on the training sample to update N parameters in the M parameters, until data processing precision of the first neural network model meets a preset condition, to obtain a second neural network model, wherein N is a positive integer less than M, and the N parameters are determined based on a capability of affecting the data processing precision by each of the M parameters. ( this limitation merely recites mathematical concepts in the form of mathematical algorithms, calculations, or formulas, see paragraphs [00126-00127] and [00184-00185] of the specification for the related mathematical disclosure ) Claim 1 also recites the following additional elements for the purposes of Step 2A Prong Two analysis: A model training method, comprising: obtaining a training sample and a first neural network model, comprising M parameters; and (obtaining networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) The additional limitations fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea. This claim recites the following additional elements for the purposes of Step 2B analysis: A model training method, comprising: obtaining a training sample and a first neural network model, comprising M parameters; and (obtaining networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) The claim also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible. Claim 11 recites limitations substantially similar to claim 1, as such a similar analysis applies. Claim recites the additional limitations for consideration: a processor, and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations comprising ( Under step 2A prong II and step 2B: this limitation merely recites computer components as a tool to perform an existing process and is therefore considered mere instructions to apply an exception under MPEP 2106.05(f) ) Dependents of Claim 1 and 11 The remaining dependent claims corresponding to independent claims 1 and 11 do not recite additional elements, whether considered individually or in combination, that are sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. The analysis of which is shown below: The claims below recite additional limitations which fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice. The claims also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claims are unpatentable. Claim 2 recites the additional limitation of : The method according to claim 1, wherein in the second neural network model, parameters other than the N parameters in the M parameters are not updated. ( not updating a certain portion of parameters is a mental process of evaluation which can reasonably be performed in human mind or with aid of pen and paper ) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Claim 3 recites the additional limitation of : The method according to claim 1, wherein the N parameters are N parameters that most affect the data processing precision of the first neural network model in the M parameters; or the N parameters are N parameters whose capabilities of affecting the data processing precision of the first neural network model are greater than a threshold in the M parameters. ( choosing N parameters that most affect the model’s precision at a high level of generality as to the precision or parameters, is being considered a mental process of evaluation which can reasonably be performed in human mind or with aid of pen and paper ) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Claim 4 recites the additional limitation of : The method according to claim 1, wherein a proportion of N to M is less than 10%. ( choosing N parameters that are less than 10% of total parameters is being considered a mental process of evaluation which can reasonably be performed in human mind or with aid of pen and paper ) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Claim 5 recites the additional limitation of : The method according to claim 1, wherein before the training the first neural network model based on the training sample to update N parameters in the M parameters, the method further comprises: receiving a model update indication sent by a terminal device, wherein the model update indication indicates to update the N parameters in the first neural network model, or the model update indication indicates to update a target proportion of parameters in the first neural network model. (obtaining networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Claim 6 recites the additional limitation of : The method according to claim 1, wherein the second neural network model comprises N updated parameters, and after the obtaining the second neural network model, the method further comprises: obtaining model update information, comprising a numerical variation of each of the M parameters in the second neural network model relative to a value before update; (obtaining networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) compressing the model update information to obtain the compressed model update information; ( compressing model information without any indication of a compression method is being considered a mental process of evaluation which can reasonably be performed in human mind or with aid of pen and paper ) and sending the compressed model update information to the terminal device. (obtaining networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Claim 7 recites the additional limitation of : The method according to claim 1, wherein after the obtaining the second neural network model, the method further comprises: sending model update information to the terminal device, wherein the model update information comprises N updated parameters, and the model update information does not comprise the parameters other than the N parameters in the M parameters. (sending networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Dependents of Claim 8 Dependent claims of claim 8 recites limitations that are abstract ideas in the form of mental processes and/or do not recite additional elements, whether considered individually or in combination, that are sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. The analysis of which is shown below: The claims below recite additional limitations which fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice. The claims also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claims are unpatentable. Claim 9 : Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon? Yes. Claim 9 recites limitations that are abstract ideas in the form of mental processes: decompressing the compressed model update information to obtain the model update information, wherein the model update information comprises a plurality of parameters obtained by updating the target quantity or target proportion of parameters; ( decompressing model information without any indication of a decompression method is being considered a mental process of evaluation which can reasonably be performed in human mind or with aid of pen and paper ) and a difference between a quantity of parameters comprised in the model update information and the target quantity falls within a preset range, or a proportion of a quantity of parameters comprised in the model update information to a quantity of parameters comprised in the first neural network model and the target proportion fall within a preset range. ( this limitation merely recites mathematical concepts in the form of mathematical algorithms, calculations, or formulas, see paragraphs [00184-00186] of the specification for the related mathematical disclosure ) Claim 9 also recites the following additional elements for the purposes of Step 2A Prong Two analysis: A parameter configuration method during model updating, comprising:displaying a configuration interface, comprising a first control that indicates a user to enter a quantity or proportion of parameters that need to be updated in a first neural network model; (For the purposes of step 2A prong II: displaying a interface to collect data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) obtaining a target quantity or a target proportion entered by the user by using the first control; and (For the purposes of step 2A prong II: obtaining data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) sending a model update indication to a server, wherein the model update indication comprises the target quantity or the target proportion, (For the purposes of step 2A prong II: sending networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g),) and the target quantity or the target proportion indicates to update the target quantity or target proportion of parameters in the first neural network model during training of the first neural network model. ( this limitation is merely applying target quantity or target proportion to update the parameters of the first neural network and is considered as mere instruction to apply an exception, see MPEP 2106.05(h) ) The method according to claim 8, wherein after the sending the model update indication to the server, the method further comprises: receiving compressed model update information sent by the server, and (receiving networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) The additional limitations fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea. This claim recites the following additional elements for the purposes of Step 2B analysis: A parameter configuration method during model updating, comprising:displaying a configuration interface, comprising a first control that indicates a user to enter a quantity or proportion of parameters that need to be updated in a first neural network model; (For the purposes of step 2B: displaying a interface to collect data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) obtaining a target quantity or a target proportion entered by the user by using the first control; and (obtaining data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) sending a model update indication to a server, wherein the model update indication comprises the target quantity or the target proportion, (sending networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) and the target quantity or the target proportion indicates to update the target quantity or target proportion of parameters in the first neural network model during training of the first neural network model. ( this limitation is merely applying target quantity or target proportion to update the parameters of the first neural network and is considered as mere instruction to apply an exception, see MPEP 2106.05(h) ) The method according to claim 8, wherein after the sending the model update indication to the server, the method further comprises: receiving compressed model update information sent by the server, and (receiving networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) The claim also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible. Claim 10 recites the additional limitation of : Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon? Yes. Claim 10 recites limitations that are abstract ideas in the form of mental processes: and decompressing the compressed model update information to obtain the model update information, wherein the model update information comprises numerical variations of a plurality of parameters, obtained by updating the plurality of parameters in the first neural network model. ( decompressing model information without any further indication of a decompression method is being considered a mental process of evaluation which can reasonably be performed in human mind or with aid of pen and paper ) Claim 10 also recites the following additional elements for the purposes of Step 2A Prong Two analysis: A parameter configuration method during model updating, comprising:displaying a configuration interface, comprising a first control that indicates a user to enter a quantity or proportion of parameters that need to be updated in a first neural network model; (For the purposes of step 2A prong II: displaying a interface to collect data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) obtaining a target quantity or a target proportion entered by the user by using the first control; and (For the purposes of step 2A prong II: obtaining data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) sending a model update indication to a server, wherein the model update indication comprises the target quantity or the target proportion, (For the purposes of step 2A prong II: sending networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) and the target quantity or the target proportion indicates to update the target quantity or target proportion of parameters in the first neural network model during training of the first neural network model. ( this limitation is merely applying target quantity or target proportion to update the parameters of the first neural network and is considered as mere instruction to apply an exception, see MPEP 2106.05(h) ) The method according to claim 8, wherein after the sending the model update indication to the server, the method further comprises: receiving compressed model update information sent by the server, (sending networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g)) The additional limitations fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice. Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea. This claim recites the following additional elements for the purposes of Step 2B analysis: A parameter configuration method during model updating, comprising:displaying a configuration interface, comprising a first control that indicates a user to enter a quantity or proportion of parameters that need to be updated in a first neural network model; (displaying a interface to collect data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) obtaining a target quantity or a target proportion entered by the user by using the first control; and (obtaining data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) sending a model update indication to a server, wherein the model update indication comprises the target quantity or the target proportion, (sending networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) and the target quantity or the target proportion indicates to update the target quantity or target proportion of parameters in the first neural network model during training of the first neural network model. ( this limitation is merely applying target quantity or target proportion to update the parameters of the first neural network and is considered as mere instruction to apply an exception, see MPEP 2106.05(h) ) The method according to claim 8, wherein after the sending the model update indication to the server, the method further comprises: receiving compressed model update information sent by the server, (sending networked monitored data is merely data gathering and is considered insignificant extra-solution activity under MPEP 2106.05(g), for the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.) The claim also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible. Claims 12-17 and 19-20 recite limitations substantially similar to claims 2-7 and 9-10 respectively, as such a similar analysis under 101 applies. Claims 19 and 20 recites the additional limitations for consideration: a processor, and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations comprising ( this limitation merely recites computer components as a tool to perform an existing process and is therefore considered mere instructions to apply an exception under MPEP 2106.05(f) ) Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15 AIA Claim s 1-4 and 11-14 are rejected under 35 U.S.C. 102( a)(1 ) as being unpatentable by Sun et al., ( Sun, X., Ren, X., Ma, S., & Wang, H. (2017, July). meprop: Sparsified back propagation for accelerated deep learning with reduced overfitting. In International Conference on Machine Learning (pp. 3299-3308). PMLR. ), hereafter referred to as Sun . Claim 1: Sun teaches the following limitations: A model training method, comprising: ( Sun, page 1, col. 2, paragraph 4, “ We propose a sparsified back propagation technique for neural network learning, in which only a small subset of the full gradient is computed to update the model parameters. ”, Sun is directed to training neural networks using sparsified backpropagation. The disclosed procedure is a model training method because it performs forward propagation, backpropagation, and parameter modification on neural-network models using training data. ) obtaining a training sample and a first neural network model, comprising M parameters; and ( Sun, page 2, section 2.1, “ Forward propagation of neural network models, including feedforward neural networks, RNN, LSTM, consists of linear transformations and non-linear transformations ”; Sun, page 2, section 2.1, “ where W ∈ Rn×m ”; Sun, page 3, section 2.2, “ We have coded two neural network models, including an LSTM model … and a feedforward NN model (MLP) ”; Sun, page 5, section 4, paragraph 2, “ We use the standard benchmark dataset in prior work (Collins, 2002), which is derived from the Penn Treebank corpus, and use sections 0-18 of the Wall Street Journal (WSJ) for training (38,219 examples), and sections 22-24 for testing (5,462 examples). ”; Sun, page 5, section 4.1, “We set the dimension of the hidden layers to 500 for all the tasks.” Sun teaches obtaining training examples from benchmark datasets and applying those examples to neural network models, including LSTM and MLP models. Sun’s neural network includes parameter matrices such as W ∈ Rn×m, and the individual entries of those weight matrices are model parameters. Under the broadest reasonable interpretation, the claimed “M parameters” are the total trainable parameters of the first neural network model, including the entries of the weight matrices used by Sun’s LSTM/MLP models. Thus, Sun teaches obtaining a training sample and a first neural network model comprising M parameters. ) training the first neural network model based on the training sample to update N parameters in the M parameters, ( Sun, page 2, section 2, “ We propose a simple yet effective technique for neural network learning. The forward propagation is computed as usual. During back propagation, only a small subset of the full gradient is computed to update the model parameters. The gradient vectors are “quantized” so that only the top-k components in terms of magnitude are kept. We first pre sent the proposed method and then describe the implementation details. ”, Sun performs ordinary neural-network training on training data, but during backpropagation it keeps only a selected subset of gradient components and updates only the corresponding subset of parameters. Because the full model has a larger parameter set and only the selected subset is modified during the update step, the disclosure teaches training the first model by updating N parameters among M total parameters. ) until data processing precision of the first neural network model meets a preset condition, to obtain a second neural network model, ( Sun, page 6, col. 2, paragraph 1, “ In the table, meProp means applying meProp to the corresponding baseline model, h =500meansthat the hidden layer dimension is 500, and k = 20 means that meProp uses top-20 elements (among 500 in total) for back propagation. Note that, for fair com parisons, all experiments are first conducted on the development data and the test data is not observable. Then, the optimal number of iterations is decided based on the optimal score on development data, and the model of this iteration is used upon the test data to obtain the test scores ”, Sun evaluates trained models using task scores/accuracy on development and test data and uses the best development-data performance to select the trained model. In the context of neural-network classification, such score/accuracy is the data-processing precision of the model. Training to the selected development-data performance condition yields the trained updated model. ) wherein N is a positive integer less than M, ( Sun, page 6, col. 2, paragraph 2, “ As we can see, applying meProp can substantially speed up the back propagation. It provides a linear reduction in the computational cost. Surprisingly, results demonstrate that we can update only 1–4% of the weights at each back propagation pass. ”, Updating only a small percentage of the model weights necessarily means the number of updated parameters is positive and smaller than the total number of parameters. Thus, the selected updated parameter count is less than the full parameter count of the first neural-network model. ) and the N parameters are determined based on a capability of affecting the data processing precision by each of the M parameters. ( Sun, page 1, col. 2, paragraph 2, “ We propose a top-k search method to find the most important parameters. Interestingly, experimental results demonstrate that we can update only 1–4% of the weights at each back propagation pass. This does not result in a larger number of training iterations. The proposed method is general-purpose and it is independent of specific models and specific optimizers (e.g., Adam and AdaGrad). ”, Sun’s MeProp method selects the top- k gradient components by magnitude and updates the corresponding model parameters. Gradient magnitude reflects how much a parameter contributes to changing the training objective, and MeProp reports that this selective update can improve accuracy while avoiding modification of weakly relevant parameters. Accordingly, the selected parameters are determined according to their training significance, i.e., their ability to affect model performance/precision. ) Claim 2: Sun teaches the limitations of claim 1, Sun further teaches: The method according to claim 1, wherein in the second neural network model, parameters other than the N parameters in the M parameters are not updated. ( Sun, page 1, col. 2, paragraph 2, “ We propose a top-k search method to find the most important parameters. Interestingly, experimental results demonstrate that we can update only 1–4% of the weights at each back propagation pass. This does not result in a larger number of training iterations. The proposed method is general-purpose and it is independent of specific models and specific optimizers (e.g., Adam and AdaGrad). ”, Because Sun’s MeProp method masks the unselected gradient components and modifies only the parameters corresponding to the selected top- k components, the remaining model parameters are left unchanged during that update. The claim does not require permanent pruning; it requires that parameters other than the selected N parameters are not updated in the claimed training operation, which is taught by MeProp’s sparse backpropagation update. ) Claim 3: Sun teaches the limitations of claim 1, Sun further teaches: The method according to claim 1, wherein the N parameters are N parameters that most affect the data processing precision of the first neural network model in the M parameters; or the N parameters are N parameters whose capabilities of affecting the data processing precision of the first neural network model are greater than a threshold in the M parameters. ( Sun, page 2, col. 2, paragraph 1, “ The proposed meProp uses approximate gradients by keeping only top-k elements based on the magnitude va lues. That is, only the top-k elements with the lar gest absolute values are kept. For example, suppose a vector v = 1,2,3,−4 , then top2(v) = 0,0,3,−4 . We denote the indices of vector σ(y)’s top-k values as {t1, t2, ..., tk}(1 ≤ k ≤ n) ”, MeProp keeps the top- k components of the gradient rather than random components. The selected components are the ones with the largest magnitudes and are described as carrying the most important training information. That teaches the first alternative of claim 3: selecting the N parameters that most affect model performance/precision. ) Claim 4: Sun teaches the limitations of claim 1, Sun further teaches: The method according to claim 1, wherein a proportion of N to M is less than 10%. ( Sun, page 6, section 4.2, “Surprisingly, results demonstrate that we can update only 1–4% of the weights at each back propagation pass”; Sun, page 6, section 4.3, “when k=5, it means that the backprop ratio is 5/500=1%.” Sun expressly teaches updating only a 1–4% portion of the weights during each back propagation pass. Because 1–4% is less than 10%, Sun teaches that the proportion of updated parameters N relative to total parameters M is less than 10%. ) Claims 11-14 recite limitations substantially similar to claims 1-4, as such a similar analysis applies. Claim 11 recites the following additional limitation for consideration which Sun further teaches: a processor, and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations comprising ( Sun, page 6, section 4.1, further teaches that “ experiments on CPU are conducted on a computer with the INTEL(R) Xeon(R) 3.0GHz CPU, ” and that GPU experiments are conducted on an NVIDIA GeForce GTX 1080. Sun’s coded C#/PyTorch neural-network training framework executing on a computer with a CPU/GPU teaches processor-executed software instructions. A computer executing the coded training framework necessarily includes memory storing the coded instructions and model data used by the processor during training. Thus, Sun teaches an apparatus comprising a processor and memory coupled to the processor to store instructions, which when executed, cause the processor to perform the claimed model-training operations. ) Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-103 AIA The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claim s 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Koch et al., ( US10360517B2 ), hereafter referred to as Koch and in further view of Alistarh ( US11636343B2 ), hereafter referred to as Alistarh . Claim 5: Koch, in the same field of machine learning, further teaches the following which Sun fails to teach: The method according to claim 1, wherein before the training the first neural network model based on the training sample to update N parameters in the M parameters, the method further comprises: receiving a model update indication sent by a terminal device, ( Koch, col. 13, lines 42-43, “ a user may execute model tuning application 222, which causes presentation of a first user interface window … User device 200 accepts commands from a user and relays necessary instructions to selection manager device 104 ”; Koch, col. 28, line 24, “ a tuning evaluation to select hyperparameters is requested of selection manager device 104 using the tuning evaluation parameters ”; Koch, col. 29, line 19, “ The computing devices of worker system 106 receive instructions from selection manager device 104 ,” and col. 36, line 50, “ the model type is trained using the hyperparameter configuration. ” Koch teaches a user device that receives user commands and sends training/tuning instructions to a separate selection manager device before the model is trained. The relayed instructions include tuning evaluation parameters and hyperparameter configurations that control how model training is performed. Such training-control instructions constitute a model update indication because they indicate the model-training/update operation to be performed by the receiving training system. ) It would have been obvious to one of ordinary skill in the art to modify the sparse neural-network training method of Sun to incorporate the distributed user-device configuration teachings of Koch, such that a terminal or user device sends an instruction or indication before training begins, because Sun already teaches reducing neural-network training work by updating only a selected subset of model parameters during backpropagation, while Koch teaches a known distributed machine-learning arrangement in which a user device receives user commands through a user interface and relays necessary instructions to another computing device responsible for model selection or tuning. A motivation for this modification would have been to allow Sun’s sparse-training process to be remotely configured and initiated by a user from a terminal device, thereby improving usability and allowing the user to control sparse-training parameters without manually altering the training system itself, with the predictable result that the training device receives a model-training instruction before performing the sparse update process. Alistar, in the same field of neural network training, teaches the following which Sun and Koch fail to teach: wherein the model update indication indicates to update the N parameters in the first neural network model, or the model update indication indicates to update a target proportion of parameters in the first neural network model. ( Alistarh, col. 9, lines 31-43, “ parameters or user input, such as a desired target sparsity ratio for each layer… standard training of the weights may occur via backpropagation ”; Alistarh, col. 10, table 1, “ let SL … be the target sparsity threshold for each layer,” “Compute the current target sparsity threshold Sp,L,” and “Thresholdp = #Parameters * Sp,L − #PrunedL. ” Alistarh teaches that user input can specify a desired target sparsity ratio for a neural network layer, and that the system computes the number of weights to be affected from the number of parameters and the target sparsity threshold. In view of Sun’s sparse parameter updating, the user-specified sparsity ratio would indicate a target proportion of parameters to be updated, removed, reintroduced, or otherwise affected during neural-network training. Thus, the model update indication comprises information indicating a target proportion of parameters in the first neural network model. ) It would have been obvious to one of ordinary skill in the art to further modify the combination of Sun and Koch to incorporate the target sparsity or parameter-proportion teachings of Alistarh, because Sun teaches sparse parameter updating during neural-network training, Koch teaches receiving user-entered configuration instructions from a user device, and Alistarh teaches controlling neural-network sparsification using a target sparsity threshold expressed as a percentage or number of neural-network elements. A motivation for this modification would have been to allow the user-provided indication in the Sun/Koch system to specify the amount or proportion of neural-network parameters to be updated, retained, or affected during training, thereby giving the user direct control over the degree of sparsity and allowing the system to balance computational efficiency, model size, and accuracy in a predictable manner. Claim 15 recites limitations substantially similar to claims 5, as such a similar analysis applies . 07-21-aia AIA Claim s 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Koch in view of Alistarh . Claim 8: Koch teaches the following: A parameter configuration method during model updating, comprising:displaying a configuration interface, comprising a first control that indicates a user to enter a quantity or proportion of parameters that need to be updated in a first neural network model; ( Koch, col. 13, line 43, “ a user may execute model tuning application 222, which causes presentation of a first user interface window, ” including “ menus and selectors such as drop down menus, buttons, text boxes, hyperlinks, etc. ”; Koch, col. 14, line 57, “ Example tables include a “Tuner Information” output table that summarizes values of options specified by the user to control execution of model tuning application 222 ; a “Tuner Results” output table that includes a default configuration and up to ten of the best hyperparameter configurations (based on an extreme (minimum or maximum) objective function value) identified, where each configuration listed includes the hyperparameter values and objective function value for comparison; a “Tuner Evaluation History” output table that includes all of the hyperparameter configurations evaluated, where each configuration listed includes the hyperparameter values and objective function value for comparison; a “Best Configuration” output table that includes values of the hyperparameters and the objective function value for the best configuration identified; a “Tuner Summary” output table that includes statistics about execution of the tuning process; a “Tuner Task Timing” output table that includes timing information about the different tasks performed during the tuning process; and a trained model output that includes information to execute the model generated using the input dataset with the best hyperparameter configuration ”, Koch teaches displaying a model-tuning interface with user-selectable controls, including text boxes, menus, and selectors, for receiving model-training configuration values. Therefore, the teachings provide a configuration interface having a control through which the user enters a target quantity or target proportion of neural-network parameters for the model updating/training process. ) obtaining a target quantity or a target proportion entered by the user by using the first control; and ( Koch, col. 21, line 29, “ In an operation 522 , an eleventh indicator of an objective function may be received. For example, the eleventh indicator indicates a name of an objective function. The objective function specifies a measure of model error (performance) to be used to identify a best configuration of the hyperparameters among those evaluated. The eleventh indicator may be received by training application 122 after selection from a user interface window or after entry by a user into a user interface window. ”, Koch teaches receiving user-selected or user-entered configuration values through the displayed interface. This teaches obtaining the target quantity or proportion from the user through the interface control before initiating the training/tuning process. ) sending a model update indication to a server, ( Koch, col. 13, line 52, “ a connection is established with selection manager device 104… User device 200 accepts commands from a user and relays necessary instructions to selection manager device 104 ”; Koch, col. 28, line 24, “ a tuning evaluation to select hyperparameters is requested of selection manager device 104 using the tuning evaluation parameters ”; Koch, col. 29, line 19, “ The computing devices of worker system 106 receive instructions from selection manager device 104, ” and Koch, col. 36, line 50, “ the model type is trained using the hyperparameter configuration. ” Koch teaches that the user device sends user-entered model-tuning instructions to a separate selection manager device, and that those instructions are used to cause model training according to a hyperparameter configuration. The transmitted instruction therefore indicates the model update/training operation to be performed and constitutes a model update indication sent to a server. ) Alistar, in the same field of neural network training, teaches the following which Sun and Koch fail to teach: wherein the model update indication comprises the target quantity or the target proportion, and the target quantity or the target proportion indicates to update the target quantity or target proportion of parameters in the first neural network model during training of the first neural network model. ( Alistarh, col. 5, lines 34-37, “ gradually remove weights until a set sparsity threshold … is reached ”; Alistarh, col. 9, lines 31-33, “ user input, such as a desired target sparsity ratio for each layer, ” and “ standard training of the weights may occur via backpropagation ”; Alistarh, col. 10, tabl 1, “ SL … [is] the target sparsity threshold for each layer, ” “ Sp,L ,” and “ Thresholdp = #Parameters * Sp,L − #PrunedL .” Alistarh teaches that a user-input target sparsity ratio or threshold is used during a series of training/pruning steps and is converted into a number of weights to be affected based on the number of parameters in a layer. When combined with Sun’s teaching of updating only a selected subset of neural-network parameters during training, the target sparsity value entered through Koch’s interface indicates the target quantity or target proportion of parameters to be updated or otherwise affected during training of the first neural network model. ) It would have been obvious to one of ordinary skill in the art to further modify the combination of Koch to incorporate the target sparsity or parameter-proportion teachings of Alistarh, because Koch teaches receiving user-entered configuration instructions from a user device, and Alistarh teaches controlling neural-network sparsification using a target sparsity threshold expressed as a percentage or number of neural-network elements. A motivation for this modification would have been to allow the user-provided indication in the Koch system to specify the amount or proportion of neural-network parameters to be updated, retained, or affected during training, thereby giving the user direct control over the degree of sparsity and allowing the system to balance computational efficiency, model size, and accuracy in a predictable manner. Claim 18 recite limitations substantially similar to claims 8, as such a similar analysis applies. Claim 18 recites the following additional limitation for consideration which Koch further teaches: a processor, and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations comprising ( Koch, col. 2, line 38, “ In another example embodiment, a computing device is provided. The computing device includes, but is not limited to, a processor and a non-transitory computer-readable medium operably coupled to the processor. The computer-readable medium has instructions stored thereon that, when executed by the processor, cause the computing device to automatically select hyperparameter values based on objective criteria for training a predictive model. ” ) 07-21-aia AIA Claim s 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Caldas et al., ( Caldas, S., Konečny, J., McMahan, H. B., & Talwalkar, A. (2018). Expanding the reach of federated learning by reducing client resource requirements. arXiv preprint arXiv:1812.07210 . ), hereafter referred to as Caldas . Claim 7: Sun teaches the limitations of claim 1, Caldas, in the same field of machine learning, further teaches the following sun fails to teach: The method according to claim 1, wherein after the obtaining the second neural network model, the method further comprises:sending model update information to the terminal device, ( Caldas, page 3, section 3, “ In this section, we present our proposed strategies for reducing Federated Learning’s (FL) server-to-client communication costs, namely lossy compression techniques ”, Caldas discloses sending model information from the server to the client as part of federated training. That teaches the claimed transmission of model update information to the terminal device after the model has been produced or updated. ) wherein the model update information comprises N updated parameters, and the model update information does not comprise the parameters other than the N parameters in the M parameters. ( Caldas, page 4, section 3.2, paragraph 3, “ To extend this idea to FL and realize communication and computation savings, we instead zero out a fixed number of activations at each fully-connected layer, so all possible sub-models have the same reduced architecture; see Figure 2. The server can map the necessary values into this reduced architecture, meaning only the necessary coefficients are transmitted to the client, re-packed as smaller dense matrices. The client (which may be fully unaware of the original model’s architecture) trains its sub-model and sends its update, which the server then maps back to the global model2. For convolutional layers, zeroing out activations would not realize any space savings, so we instead drop out a fixed percentage of filters. ”, Caldas teaches sending only the coefficients needed for a reduced sub-model, rather than transmitting all parameters of the full global model. Transmitting only the necessary coefficients teaches model update information that includes the selected updated parameters while omitting the remaining parameters. ) It would have been obvious to one of ordinary skill in the art to modify Sun to incorporate the reduced sub-model communication teachings of Caldas , because Sun teaches that only a small selected subset of model parameters is updated during neural-network training, while Caldas teaches transmitting smaller subsets of a global model to client devices to reduce communication and computation in federated learning. A motivation for this modification would have been to avoid sending unnecessary full-model information when only a selected subset of parameters has been updated or is needed by the client, thereby reducing bandwidth and client-side processing requirements, with the predictable result that the transmitted model update information includes the selected updated parameters while omitting parameters outside that sparse subset. Claims 17 recite limitations substantially similar to claims 7, as such a similar analysis applies . 07-21-aia AIA Claim s 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Koch in view of Alistarh and in further view of Caldas . Claim 9: Koch and Alistarh teaches the limitations of claim 8, Caldas, in the same field of machine learning, further teaches the following which Koch and Alistarh fails to teach: The method according to claim 8, wherein after the sending the model update indication to the server, the method further comprises: receiving compressed model update information sent by the server, and ( Caldas, page 3, section 3, “ In this section, we present our proposed strategies for reducing Federated Learning’s (FL) server-to-client communication costs, namely lossy compression techniques ”, Caldas teaches that, in federated learning, the server sends compressed model information to the client. This satisfies the claimed receiving step from the perspective of the terminal/client device after configuration and coordination with the server. ) decompressing the compressed model update information to obtain the model update information, ( Caldas, page 2, figure 1, “ Combination of our proposed strategies during FL training. We reduce the size of the model to be communicated by (1) constructing a sub-model via Federated Dropout, and by (2) lossily compressing the resulting object. This compressed model is then sent to the client, who (3) decompresses and trains it using local data, and (4) compresses the final update. This update is sent back to the server, where it is (5) decompressed and finally, (6) aggregated into the global model ”, Caldas expressly teaches the client receiving compressed model information and decompressing it before local training. The decompressed information is then available as the model information used by the terminal device. ) wherein the model update information comprises a plurality of parameters obtained by updating the target quantity or target proportion of parameters; ( Caldas, page 4, section 3.2, “ To further reduce communication costs, we propose an algorithm in which each client, instead of locally training an update to the whole global model, trains an update to a smaller sub-model. These sub-models are subsets of the global model and, as such, the computed local updates have a natural interpretation as updates to the larger global model. ”, Caldas’s federated dropout sends and trains only a reduced subset of the global model. The dropout/subsampling rate determines the proportion of the model participating in the client-side training round, so the resulting transmitted model information comprises a plurality of parameters corresponding to that reduced target proportion. ) and a difference between a quantity of parameters comprised in the model update information and the target quantity falls within a preset range, or a proportion of a quantity of parameters comprised in the model update information to a quantity of parameters comprised in the first neural network model and the target proportion fall within a preset range. ( Caldas, page 6, section 4.3, paragraph 2, “ Figure 4 shows how the convergence of our three models behaves under different federated dropout rates. We repeat each experiment 10 times and report the mean among these repetitions. The main takeaway from these experiments is that, for every model, it is possible to find a federated dropout rate less than 1.0 that matches or, in some cases, even improves on the final accuracy of the model. ”, A fixed dropout rate establishes the predetermined proportion of the global model to be included in the sub-model. The claimed preset range is satisfied or rendered obvious by applying an allowed tolerance around that fixed rate, since practical compression/dropout schemes necessarily operate according to predetermined percentages or rates. ) It would have been obvious to one of ordinary skill in the art to modify Koch and Alistarh to incorporate the reduced sub-model communication teachings of Caldas, Caldas teaches transmitting smaller subsets of a global model to client devices to reduce communication and computation in federated learning. A motivation for this modification would have been to avoid sending unnecessary full-model information when only a selected subset of parameters has been updated or is needed by the client, thereby reducing bandwidth and client-side processing requirements, with the predictable result that the transmitted model update information includes the selected updated parameters while omitting parameters outside that sparse subset. Claims 19 recite limitations substantially similar to claims 9, as such a similar analysis applies . 07-21-aia AIA Claim s 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Sun in view of Caldas and in further view of Bacon et al., ( US11763197B2 ), hereafter referred to as Bacon . Claim 6: Sun teaches the limitations of claim 1, Caldas, in the same field of machine learning, further teaches the following sun fails to teach: and sending the compressed model update information to the terminal device. ( Caldas, page 3, section 3, “ In this section, we present our proposed strategies for reducing Federated Learning’s (FL) server-to-client communication costs, namely lossy compression techniques ”, Caldas teaches that, in federated learning, the server sends compressed model information to the client. This satisfies the claimed receiving step from the perspective of the terminal/client device after configuration and coordination with the server. ) The rationale for combining Sun with Caldas is similar to that as applied for claim 7 above. Bacon in the same field of neural network training, teaches the following which Sun and Caldas fails to teach: The method according to claim 1, wherein the second neural network model comprises N updated parameters, ( Bacon, col. 12, line 59, “ In some implementations, training the machine-learned model at 404 can include training the machine-learned model based at least in part on the local dataset such that updated values are determined only for a pre-selected portion of the set of parameters. In such implementations, the update matrix can be descriptive of only the updated values for the pre-selected portion of the set of parameters. ”, Bacon teaches training a machine-learned model so that updated values are determined only for a selected subset of the model parameters. In the context of the base sparse-training rejection, the resulting trained model includes the selected updated subset rather than requiring every parameter of the model to be updated. ) and after the obtaining the second neural network model, the method further comprises: obtaining model update information, comprising a numerical variation of each of the M parameters in the second neural network model relative to a value before update; ( Bacon, col. 12, line 30, “ At 404, the client computing device trains the machine learned model based at least in part on a local dataset to obtain an update matrix that is descriptive of updated values for the set of parameters of the machine-learned model. The update matrix is restricted to have a pre-defined structure. The local dataset is stored locally by the client computing device. In some implementations, the update matrix is descriptive of updated values for the set of parameters and/or differences between the updated values and the global values. ”, Bacon teaches producing an update matrix after local model training. Because the update matrix describes the changed model-parameter values resulting from training, it is model update information obtained after the trained model/update has been produced. Furthemore, Bacon expressly teaches that the update information may be expressed as differences between updated parameter values and prior global parameter values. A difference between an updated value and the earlier model value is a numerical variation relative to the value before update. ) compressing the model update information to obtain the compressed model update information; ( Bacon, col. 13, line 33, “ At 506, the client computing device encodes the update matrix to obtain an encoded update. ”, Bacon teaches encoding the update matrix after the update matrix has been generated. The patent further describes subsampling, quantization, and compressed/sketched update forms, so the encoded update is the compressed form of the model update information. ) It would have been obvious to one of ordinary skill in the art to further modify the combination of Sun and Caldas to incorporate Bacon’s teaching of model update information expressed as updated values or differences from prior/global values, because the existing combination teaches user-configured sparse updating and compressed server-to-client transfer, while Bacon teaches representing trained model changes using update matrices that include numerical parameter values or numerical differences resulting from training. A motivation for this modification would have been to provide a conventional and useful representation for the decompressed model update information, namely numerical variations of a plurality of parameters produced by updating the neural-network model, with the predictable result that the compressed information received and decompressed by the terminal device contains parameter-change values suitable for reconstructing or applying the model update. Claim 16 recites limitations substantially similar to claim 6, as such a similar analysis applies . 07-21-aia AIA Claim s 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Koch in view of Alistarh and Caldas and in further view of Bacon . Claim 10: Koch and Alistarh teaches the limitations of claim 8, Caldas further teaches the following which Koch and Alistarh fails to teach: The method according to claim 8, wherein after the sending the model update indication to the server, the method further comprises:receiving compressed model update information sent by the server, anddecompressing the compressed model update information to obtain the model update information, ( Caldas, page 2, figure 1, “ Combination of our proposed strategies during FL training. We reduce the size of the model to be communicated by (1) constructing a sub-model via Federated Dropout, and by (2) lossily compressing the resulting object. This compressed model is then sent to the client, who (3) decompresses and trains it using local data, and (4) compresses the final update. This update is sent back to the server, where it is (5) decompressed and finally, (6) aggregated into the global model ”, Caldas expressly teaches the client receiving compressed model information and decompressing it before local training. ) The rationale for combining Koch and Alistarh with Caldas is similar to that as applied for claim 9 above. Bacon in the same field of neural network training, teaches the following which Koch, Alistarh, and Caldas fails to teach: wherein the model update information comprises numerical variations of a plurality of parameters, obtained by updating the plurality of parameters in the first neural network model. ( Bacon, col. 13, line 25, “ At 504 , the client computing device trains the machine-learned model based at least in part on a local dataset to obtain an update matrix that is descriptive of updated values for the set of parameters of the machine-learned model. The local dataset is stored locally by the client computing device. In some implementations, the update matrix is descriptive of updated values for the set of parameters and/or differences between the updated values and the global values. ”; Bacon, col. 2, line 5, “ The operations include obtaining global values for a set of parameters of a machine-learned model. The operations include training the machine-learned model based at least in part on a local dataset to obtain an update matrix that is descriptive of updated values for the set of parameters of the machine-learned model. The local dataset is stored locally by the client computing device. The operations include encoding the update matrix to obtain an encoded update. The operations include communicating the encoded update to a server computing device.” , Bacon teaches that model update information may be represented as an update matrix containing updated parameter values or differences between updated parameter values and prior/global parameter values. A difference between an updated parameter value and a prior/global value is a numerical variation of that parameter. Because the update matrix is obtained by training the machine-learned model, the numerical variations are obtained by updating a plurality of parameters in the neural-network model. ) It would have been obvious to one of ordinary skill in the art to further modify the combination of Koch, Alistarh, and Caldas to incorporate Bacon’s teaching of model update information expressed as updated values or differences from prior/global values, because the existing combination teaches user-configured sparse updating and compressed server-to-client transfer, while Bacon teaches representing trained model changes using update matrices that include numerical parameter values or numerical differences resulting from training. A motivation for this modification would have been to provide a conventional and useful representation for the decompressed model update information, namely numerical variations of a plurality of parameters produced by updating the neural-network model, with the predictable result that the compressed information received and decompressed by the terminal device contains parameter-change values suitable for reconstructing or applying the model update. Claim 20 recites limitations substantially similar to claim 10, as such a similar analysis applies . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure : Aji, A. F., & Heafield, K. (2017, September). Sparse communication for distributed gradient descent. In Proceedings of the 2017 conference on empirical methods in natural language processing (pp. 440-445). Evci, U., Gale, T., Menick, J., Castro, P. S., & Elsen, E. (2020, November). Rigging the lottery: Making all tickets winners. In International conference on machine learning (pp. 2943-2952). PMLR. Jayakumar, S., Pascanu, R., Rae, J., Osindero, S., & Elsen, E. (2020). Top-kast: Top-k always sparse training. Advances in Neural Information Processing Systems , 33 , 20744-20754. Any inquiry concerning this communication or earlier communications from the examiner should be directed to HYUNGJUN B YI whose telephone number is (703)756-4799. The examiner can normally be reached M-F 9-5. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /H.B.Y./Examiner, Art Unit 2146 /USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146 Application/Control Number: 18/496,177 Page 2 Art Unit: 2146 Application/Control Number: 18/496,177 Page 3 Art Unit: 2146 Application/Control Number: 18/496,177 Page 4 Art Unit: 2146 Application/Control Number: 18/496,177 Page 5 Art Unit: 2146 Application/Control Number: 18/496,177 Page 6 Art Unit: 2146 Application/Control Number: 18/496,177 Page 7 Art Unit: 2146 Application/Control Number: 18/496,177 Page 8 Art Unit: 2146 Application/Control Number: 18/496,177 Page 9 Art Unit: 2146 Application/Control Number: 18/496,177 Page 10 Art Unit: 2146 Application/Control Number: 18/496,177 Page 11 Art Unit: 2146 Application/Control Number: 18/496,177 Page 12 Art Unit: 2146 Application/Control Number: 18/496,177 Page 13 Art Unit: 2146 Application/Control Number: 18/496,177 Page 14 Art Unit: 2146 Application/Control Number: 18/496,177 Page 15 Art Unit: 2146 Application/Control Number: 18/496,177 Page 16 Art Unit: 2146 Application/Control Number: 18/496,177 Page 17 Art Unit: 2146 Application/Control Number: 18/496,177 Page 18 Art Unit: 2146 Application/Control Number: 18/496,177 Page 19 Art Unit: 2146 Application/Control Number: 18/496,177 Page 20 Art Unit: 2146 Application/Control Number: 18/496,177 Page 21 Art Unit: 2146 Application/Control Number: 18/496,177 Page 22 Art Unit: 2146 Application/Control Number: 18/496,177 Page 23 Art Unit: 2146 Application/Control Number: 18/496,177 Page 24 Art Unit: 2146 Application/Control Number: 18/496,177 Page 25 Art Unit: 2146 Application/Control Number: 18/496,177 Page 26 Art Unit: 2146 Application/Control Number: 18/496,177 Page 27 Art Unit: 2146 Application/Control Number: 18/496,177 Page 28 Art Unit: 2146 Application/Control Number: 18/496,177 Page 29 Art Unit: 2146 Application/Control Number: 18/496,177 Page 30 Art Unit: 2146 Application/Control Number: 18/496,177 Page 31 Art Unit: 2146 Application/Control Number: 18/496,177 Page 32 Art Unit: 2146 Application/Control Number: 18/496,177 Page 33 Art Unit: 2146 Application/Control Number: 18/496,177 Page 34 Art Unit: 2146 Application/Control Number: 18/496,177 Page 35 Art Unit: 2146 Application/Control Number: 18/496,177 Page 36 Art Unit: 2146 Application/Control Number: 18/496,177 Page 37 Art Unit: 2146 Application/Control Number: 18/496,177 Page 38 Art Unit: 2146 Application/Control Number: 18/496,177 Page 39 Art Unit: 2146 Application/Control Number: 18/496,177 Page 40 Art Unit: 2146 Application/Control Number: 18/496,177 Page 41 Art Unit: 2146 Application/Control Number: 18/496,177 Page 42 Art Unit: 2146
Prosecution Insights