DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The filing date of the present invention is 12/16/2022.
This action is in response to amendment and/or remarks filed on 01/26/2026. In the current amendments, claims 11-2 and 6 have been amended. Claims 1-7 are currently pending and have been examined.
In response to amendments and/or remarks filed on 01/26/2026, the 35 U.S.C 112(b) rejections made in the previous Office Action has been withdrawn.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-7 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/16/2022 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over
Li et al. (US 2021/0182661 A1) in view of Lampinen et al. (“AN ANALYTIC THEORY OF GENERALIZATION DYNAMICS AND TRANSFER LEARNING IN DEEP LINEAR NETWORKS”) and further in view of Ghadjati et al. (“Communication Channel Equalization Based on Levenberg Marquardt Trained Artificial Neural Networks”).
Regarding claim 1 (Currently Amended)
Li teaches ……said method comprising: learning a third neural network configured to determine a transfer function from the parameters of a second neural network to the parameters of said first neural network, the second neural network being less complex than said first neural network and also being used to implement said processing …function, (para [0090] “Referring specifically to FIG. 8A, the embodiment 800 shows how devices may be arranged according to a tree arrangement in accordance with an embodiment of the invention. The tree arrangement provides a hierarchy that allows for configuration data (e.g. first and second configuration data) to be efficiently exchanged. The tree arrangement of FIG. 8A has a master device 802 that acts as a central parameter server to provide first configuration data and receive second configuration data. The master device 802 is communicatively coupled to two sub-master devices 804 and 806. A sub-master device may include a form of master device that is configured to communicate with a defined subset of slave devices. The first sub-master 804 is communicatively coupled to three slave devices 808, 810 and 812. This subset of slave devices receives first configuration data from the first sub-master 804 and sends second configuration data to the first sub- master 804. The first sub-master 804 passes the second configuration data to the master device 802 and forwards first configuration data from the master device 802. The second sub-master 806 performs a similar role with a subset of two slave devices 814 and 816. The sub-masters” Examiner interprets Master 802 as third model and sub master 804 as second model and the slave models 808, 810 and 812 as first model and each of the model has neural network)
the learnings of the first and second neural networks having been performed by the same input and output signals, (para [0090] “The first sub-master 804 is communicatively coupled to three slave devices 808, 810 and 812. This subset of slave devices receives first configuration data from the first sub-master 804 and sends second configuration data to the first sub - master 804. The first sub-master 804 passes the second configuration data to the master device 802 and forwards first configuration data from the master device 802. The second sub - master 806 performs a similar role with a subset of two slave devices 814 and 816. The sub-masters 804, 806 may have a version of the neural network model”)
…
after detection of an evolution of said processing function communication channel, adapting the parameters of said second neural network by means of input signals associated with a learning sequence emitted on said communication channel; (para [0056] “Once instantiated, the slave device 120 is config ured to train the second version of the neural network model 170 using data from the first data source 130. The training includes retrieving a training set from the first data source 130 and using this to train a set of trainable parameters for the second version of the neural network model 170. In accordance with an embodiment and aspect of the invention, the training may include updating values of weights and / or biases using gradient information that is computed by evaluating the differential of a loss function”)
Li does not teach said transfer function making it possible to deduce determine parameters of said first neural network from parameters of said neural second network;
…
and adapting the parameters of said first neural network by using the adapted parameters of the second network and said transfer function.
Lampinen teaches said transfer function making it possible to deduce determine parameters of said first neural network from parameters of said neural second network; …and adapting the parameters of said first neural network by using the adapted parameters of the second network and said transfer function. (pg. 5 “a perturbation of the low rank teacher W by a high dimensional noise matrix Z. The relation between the singular modes of a low rank matrix and its noise perturbed version has been studied extensively in Benaych-Georges & Nadakuditi (2012), in the high dimensional limit we are working in, namely N1, N3 → ∞ with the aspect ratio. In this limit, the top N2 singular values and vectors of Σ31 converge to sˆ(sα), where the transfer function from a teacher singular value s to a training data singular value sˆ is given by the function”)
Li and Lampinen are analogous art because they are both directed to Machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for training and enhancement of neural network models disclosed by Li to include analytic theory of the nonlinear dynamics of generalization in deep linear networks of Lampinen in order to provide “analytic solutions to the training and testing error of deep networks as a function of training time, number of examples, network size and initialization, and the task structure and SNR” as disclosed by Lampinen (abstract “We develop an analytic theory of the nonlinear dynamics of generalization in deep linear networks, both within and across tasks. In particular, our theory provides analytic solutions to the training and testing error of deep networks as a function of training time, number of examples, network size and initialization, and the task structure and SNR. Our theory reveals that deep networks progressively learn the most important task structure first, so that generalization error at the early stopping time primarily depends on task structure and is independent of network size.”).
Li in view of Lampinen does not teach a method for adapting the parameters of a first neural network used in a communication network to implement, a signal processing function by an equipment, an equalization function to process a received input signal emitted on a communication channel in order to obtain estimate an output signal.
Ghadjati teaches a method for adapting the parameters of a first neural network used in a communication network to implement, a signal processing function by an equipment, an equalization function to process a received input signal emitted on a communication channel in order to obtain estimate an output signal (pg. 2 left col “equalization, then, is the set of techniques used to compensate the effects of a transmission channel (ISI). Transversal equalizers are the simplest to implement. Indeed, it is simply using a digital filter with finite impulse response corresponds to the input-output relationship:” also see pg. 6 left col “In this paper, Levenberg-Marquardt algorithm is used to update the multilayer perceptron based decision feedback equalizer (MLP-DFE) to enhance the performance of the equalization function in a digital communication channel.”)
Li and Lampinen and Ghadjati are analogous art because they are all directed to Machine Learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for training and enhancement of neural network models disclosed by Li in view of Lampinen to include communication channel equalization based on neural network of Ghadjati in order to reduce “the learning MSE (Mean Square Error) and eliminates efficiently the effects of ISI comparatively to the MLP-BP, RBF and conventional equalizers” as disclosed by Ghadjati (abstract “To compensate distortions caused by these factors and to find the original information being transmitted, equalization process is performed at the receiver. Previous authors have shown that nonlinear feed-forward equalizers based on either MLP (Multi Layer Perceptron) or RBF (Radial Basis Function) can outperform linear equalizers. In this paper, we suggest an adaptive neural network equalizer using Levenberg-Marquardt training algorithm, (MLP-LM), which considerably reduces the learning MSE (Mean Square Error) and eliminates efficiently the effects of ISI comparatively to the MLP-BP, RBF and conventional equalizers.”).
Regarding claim 2 (Currently Amended)
Li in view of Lampinen with Ghadjati teaches claim 1.
Li further teaches wherein adapting the parameters of said first neural network is performed at a lower frequency than adapting the parameters of said second neural network. (Para [0086] “As in other aspects and embodiments of the invention, the parameter data 680 is received by the slave device 620 and is used to instantiate a second version of the neural network model 670. The instantiated second version of the neural network model 670 is trained on data from a first data source 630 that is inaccessible by the master device 610. Training in this manner generates gradient data 690, which is communicated back to the master device 610 over at least one network 650. The master device 610 uses the received gradient data 690 to update its version of the neural network model, i.e. parameters for the first version of the neural network model, and in turn is able to send out updated parameter data 680 to the slave device 620. The exchange of parameter data 680 and gradient data 690 may be at a higher frequency to the exchange of data in the teacher-student case” Examiner notes that first version of the neural network model doesn’t use higher frequency exchange of data during training process which means it uses lower frequency exchange of data during training process. Higher frequency exchange is only used for the updating steps of the first version)
Regarding claim 3 (Currently Amended)
Li in view of Lampinen with Ghadjati teaches claim 1.
Li further teaches the method further including complementarily adapting the adapted parameters of said first neural network according to the input signals associated with the learning sequence. (Li abstract “A slave device receives a version of a neural network model from a master. The slave accesses a local and/or private data source and uses the data to perform optimization of the neural network model. This can be done such as by computing gradients or performing knowledge distillation to locally train an enhanced second version of the model. The slave sends the gradients or enhanced neural network model to a master. The master may use the gradient or the second version of the model to improve a master model.”)
Regarding claim 4
Li in view of Lampinen with Ghadjati teaches claim 1.
Li further teaches non-transitory computer readable medium having stored thereon instructions which, when said method is executed by a computer processor, cause the processor to implement the method of claim 1. (Para [0050] “Referring to FIG. 1, a distributed training system 100 is shown in accordance with an embodiment of the invention. The system 100 includes a master device 110 and a slave device 120. The master device 110 and the slave device 120 include computing devices, i.e. devices with at least one processor and a memory wherein the at least one processor is configured to execute computer program code loaded into the memory to perform one or more functions. In accordance with one embodiment and aspect of the invention, the master device 110 includes a server device and the slave device 120 includes a client device”)
Regarding claim 5
Li in view of Lampinen with Ghadjati teaches claim 1.
Li further teaches comprising a processor and a memory, the memory having stored thereon instructions which, when executed by the processor, cause the processor to implement the method of claim 1. (Para [0050] “Referring to FIG. 1, a distributed training system 100 is shown in accordance with an embodiment of the invention. The system 100 includes a master device 110 and a slave device 120. The master device 110 and the slave device 120 include computing devices, i.e. devices with at least one processor and a memory wherein the at least one processor is configured to execute computer program code loaded into the memory to perform one or more functions. In accordance with one embodiment and aspect of the invention, the master device 110 includes a server device and the slave device 120 includes a client device”)
Regarding claims 6 and 7
Claims 6-7 recites analogous limitations to claims 1-2 and therefore is rejected on the same ground as claims 1-2.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN C MANG whose telephone number is (571)270-7598. The examiner can normally be reached Mon - Fri 8:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at 5712707519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN C MANG/Primary Examiner, Art Unit 2126
Prosecution Insights