DETAILED ACTION
This Action is responsive to Claims filed 03/11/2026.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1, 7, and 13 have been amended. Claim 19 has been canceled. Claims 1-2, 5-8, 11-14, and 17-18 are currently pending.
Response to Arguments
Applicant’s arguments, see Pages 7-9, filed 03/11/2026, regarding the 35 U.S.C. 103 Rejection(s) of Claims 1-2, 5-8, 11-14, and 17-19 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Regarding the newly amended “…fewer rows…” limitation: The Examiner submits the Combination of the previously sited references continues to read on the generically-recited “…fewer rows…” limitation. Yu, in particular teaches relevant manipulations to a sparse parameter matrix. See the updated mapping below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-2, 5-8, 11-14, and 17-18 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The newly amended “determining a parameter value change characteristic of interest to be percentage change in value;” of the independent claims is unclear. The only reference to “interest” is Paragraph [0046] of the instant Specification. The grammar of this claim limitation is convoluted and generates confusion as to whether the percentage change in value is something similar to a “target” parameter value to utilize (being “of interest” or focus), or if it is characteristic of something akin to an interest rate in an economics-sense. For the purpose of this Action, the Examiner takes the former interpretation, but rewording the limitation would improve clarity.
The dependent claims do not rectify this ambiguity, and are therefore similarly rejected.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-2, 5-8, 11-14, and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Publication 2021/0224655 to Jeon et al. (hereinafter "Jeon") in view of US Patent Publication 2022/0147818 to Zhang et. al. (hereinafter 'Zhang'), US20180373975 to Yu et al. (hereinafter "Yu"), Xiao et al. (AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters, 2019) (hereinafter “Xiao”), and Sanh et al. (Movement Pruning: Adaptive Sparsity by Fine-Tuning, 2020), hereinafter Sanh.
As per claim 1, Jeon teaches: "a computer-implemented method for compressing a neural network parameter matrix, the method comprising:
Monitoring, by the compression engine (mapping to a typical processor or processor module, which Jeon teaches), changes in parameter values (weight values) of a neural network parameter matrix while training the neural network model (Jeon, at Figure 3, Step S310, counts the number of updates of weights for each epoch of neural network training);
Identifying, by the compression engine, a set of key parameters (e.g. those parameters deemed important based on the number of updates during training) of the neural network parameter matrix based on a relative extent of individual changes in value meeting a threshold percentage change of the individual parameter values as recorded by the flags ([0078] of Jeon describes pruning weights which are determined to be unimportant based on the number of updates during training);
creating a compressed neural network matrix for a second neural network model by:
pruning the neural network parameter matrix by removing a neuron having a highest count of parameters with unchanged parameter values relative to other neurons… ([0052] Jeon describes that a system wherein all of weights are initialized to a random value before training. Only if the weight is updated during training is the count updated and the value changed from the random value to the output value. Accordingly following a training by Jeon, those weights that remain random can be identified because they have a count of zero. Jeon teaches, at [0065] that those weights with the smallest number of updates are removed.) …the unchanged parameter values being changed below a threshold level of change; ([0078] of Jeon reads on the use of a threshold) Accordingly, Jeon teaches pruning a weight having the highest count of randomly generated parameter values (e.g., with an update count of zero).
Accordingly, Jeon, who discloses that all parameters are randomly generated,
teaches 'removing a neuron having a highest count relative to other' , when removing
weights at Fig. 3, S330. The examiner maintains that removing one or more weights, which may or may not include all connections to a neuron, reasonably reads on removing a neuron, especially given that neuron pruning would have been a well-known method of model compression at the time of the Applicant’s filing.
However, Jeon fails to explicitly describe “reshaping, by a compression engine, a neural network parameter matrix from an NxM-type matrix to an NxMx2-type matrix to include flags for each parameter of the NxM-type matrix;” and “adjusting, by a monitor module of the compression engine, the flags corresponding to individual parameter values during the training, the adjusting of the flags being performed by recording, by the monitor module, individual changes in value of the individual parameter values;” in the context of the data the mapped “flags” track (Jeon counts a number of updates, rather than a change in value). However, Xiao, in a similar field of model compression, teaches forming an auxiliary parameter matrix (mapping broadly to flag, given the term flag can be any structure recording a value corresponding to another, in this example, an auxiliary parameter mapping to a real parameter; and mapping broadly to reshaping a data matrix with an extra dimension for the auxiliary parameters), and iteratively pruning model parameters based on the auxiliary parameters’ change as the magnitude of the change moves toward 0 (threshold level of change) (Xiao Pages 4-5, Section 3.3).
Xiao teaches “To build a better generalized and easy-to-use pruning method, we propose AutoPrune, which prunes the network through optimizing a set of trainable auxiliary parameters instead of original weights. The instability and noise during training on auxiliary parameters will not directly affect weight values, which makes pruning process more robust to noise and less sensitive to hyperparameters. Moreover, we design gradient update rules for auxiliary parameters to keep them consistent with pruning tasks. Our method can automatically eliminate network redundancy with recoverability, relieving the complicated prior knowledge required to design thresholding functions, and reducing the time for trial and error” (Abstract). It would have been obvious to one of ordinary skill in the art at the time of the Applicant’s filing to track and compress a model by recording data such as taught by Xiao to robustly and efficiently compress a model.
The combination of Jeon and Xiao further fails to disclose "fine tuning only the unchanged parameter values of the compressed neural network matrix to generate a final compressed matrix for a final compressed neural network model" and ”…thereby creating the compressed neural parameter matrix having fewer rows than the neural network parameter matrix…” as claimed.
Yu discloses including, in a second neural network parameter matrix (Yu, Fig. 2, matrix 220), [0021] In some circumstances, importance parameters may be determined for respective weight parameters and/or groups of weight parameters. Weight parameters and/or groups of weight parameters having importance parameters below a specified threshold value may be removed, in some circumstances ... [0023] For example, array A may include neural network weight parameters from a relatively set of neural network weight parameters, such as relatively sparse matrix W (e.g., matrix 220).
It is further noted that Yu, like Jeon, discloses removing a neuron at [0028]: "Also,
by determining redundancy on a node-by-node basis rather than on a weight parameter-by- weight parameter basis and/or by removing redundant nodes rather than removing redundant weight parameters, for example, computational and/or storage overhead associated with sparse matrix operations may be reduced, in an embodiment." It would have been an obvious application of known methods for a person skilled in the art at the time of the Applicant’s filing to reasonably prune weights and/or nodes when implementing the methods of Jeon. The Examiner submits Yu’s storage of the weight parameters in a CSR matrix format would necessarily result in a matrix with fewer rows as nodes are pruned ([0019]-[0023] particularly highlight the structure Yu utilizes and how node removal can affect said structure).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Jeon with those of Yu to perform the steps of Jeon (identifying, reshaping) on the second parameter matrix of Yu, particularly as reshaping the network affects the size and sparsity of a CSR matrix, because it is a merely the use of a known technique to improve similar devices. One would be motivated to make this combination because, as described at [0028] of Yu, it enables a mask layer to be 'trained ... as part of a process to identify and/or remove redundant nodes).
The combination of Jeon, Xiao, and Yu, however, fails to describe "fine tuning only unchanged parameter values of the compressed neural network matrix to generate a final compressed matrix for a final compressed neural network model."
Zhang, in the same field of endeavor as Jeon, describes fine-tuning only randomly generated parameter values to generate a final compressed model ([0114] describes a compression process including the step of retraining (e.g., fine-tuning) a base model by initializing new parameters randomly, then fine-tuning these parameters while all other weights of the model are frozen). If the parameter values are initially random, and the values are tracked, then an unchanged parameter and a randomly generated parameter become indistinct at the time of pruning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Zhang with that of Jeon, Xiao, and Yu to teach fine-tuning only randomly generated weights of the compressed neural network model as recited in claim 1. One would be motivated to incorporate the teachings of Zhang into the compression system of Jeon because, as described at [0075] of Zhang, it would allow for fast adaptation of a machine learning model when new features are added to augment the originally observed data. In addition, the combination would merely comprise a combination of prior art methods in known ways to yield predictable results.
The combination of Jeon, Yu, Xiao, and Zhang fails to read on the exact monitoring of the percentage change in a weight as claimed in “determining a parameter value change characteristic of interest to be percentage change in value;” and “…meeting a threshold percentage change…” However, Sanh, in a similar field of endeavor, teaches “In this work, we argue that to effectively reduce the size of models for transfer learning, one should instead use movement pruning, i.e., pruning approaches that consider the changes in weights during fine-tuning. Movement pruning differs from magnitude pruning in that both weights with low and high values can be pruned if they shrink during training.” (Introduction). See above how the combination of Jeon, Xiao, Yu, and Zhang would read on the use of thresholds for a pruning metric, in which a combination of Sanh would necessarily use said change in the weight parameter values.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Sanh with that of Jeon, Xiao, Yu, and Zhang. One would be motivated to use the pruning method of Sanh because it is a merely the use of a known technique to improve similar devices. One would be motivated to make this combination because, as described in the Introduction of Sanh, wherein Sanh highlights the benefits of their method in particular environments over other known pruning methods.
As per claim 5, the combination of Jeon, Xiao, Zhang and Yu teaches the method of claim 1, wherein the (set of key parameters) are identified according to a percentage change in value among the parameter value changes within the neural network parameter matrix. (Jeon [0054] describes that the weights with the small number of updates are removed, weights with the larger number are retained), Xiao Section 3.3 describes values with high magnitudes of change are preserved versus ones with low magnitudes. It would have been obvious to one of ordinary skill in the art that parameters with high percentage changes would be kept.
Jeon fails to disclose using the 'set of key' parameters to make identification and the “percentage change in value” of a parameter value. Yu, however, discloses operating on a matrix (220) that includes only important parameters (Yu, [0023] "For example, a relatively sparse mxn matrix W, such as 220, may be stored as signals and/or states representative of three one-dimensional arrays, such as arrays A, JA, and/or IA of embodiment 200. For example, array A may include neural network weight parameters, such as weight parameters determined to have an importance parameter meeting and/or exceeding a specified threshold parameter".
[0023] of Yu teaches “For example, a relatively sparse mxn matrix W, such as 220, may be stored as signals and/or states representative of three one-dimensional arrays, such as arrays A, JA, and/or IA of embodiment 200. For example, array A may include neural network weight parameters, such as weight parameters determined to have an importance parameter meeting and/or exceeding a specified threshold parameter.” It would be obvious in a combination of Jeon, Zhang, and Yu that stored weights or weight parameters being compared to prior iteration’s values in such as in Jeon, and an importance parameter as taught by Yu would reasonably include a metric of how much change a weight has undergone, in addition to or in replacement of a direct counter of the number of times the value has changed.
See above how Sanh reads on monitoring the specific change in the weight parameters’ values change as a pruning metric.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Jeon and Xiao with those of Yu to perform the steps of Jeon (identifying, reshaping) on the second parameter matrix of Yu, which includes only key parameters, because it is a merely the use of a known technique to improve similar devices. One would be motivated to make this combination because, as described at ,i [0028] of Yu, it enables a mask layer to be 'trained ... as part of a process to identify and/or remove redundant nodes).
As per claim 6, the combination of Jeon, Xiao, Zhang, and Yu teaches the method of claim 1, further comprising: adding the flag for each parameter in the neural network parameter matrix, (Jeon: Fig. 2 Number-of-weight-updates matrix). (Jeon: [0049] describes maintaining a number of weight update matrix that stores a count of number of times the weight is updated during training) (Xiao Section 3.3 describes the auxiliary parameter matrix, which are also interpretable as flags tracking the value changes of each parameter);
As per claims 7, 11, and 12: Claims 7, 11, and 12 recite similar limitations to claims 1, 5, and 6, therefore, the above analysis and Rejection applies equally to both sets of claims.
As per claims 13, 17, and 18: Claims 13, 17, and 18 recite similar limitations to claims 1, 5, and 6, therefore, the above analysis and Rejection applies equally to both sets of claims.
Claims 2, 8 and 14 are rejected under 35 U.S.C. as unpatentable over Jeon, Xiao, Yu, Zhang, and Sanh as described above, and further in view of He, Changai et al. "Using Convolutional Neural Network with BERT for Intent Determination." 2019 International Conference on Asian Language Processing (IALP). IEEE, 2019. 65-70. Web (hereinafter "He").
As per claim 2, Jeon discloses the method of claim 1 using a Deep Neural Network, such as Convolutional Neural Network, but fails to explicitly disclose wherein" the neural network model is a pre-trained bidirectional encoder representations from transformers (BERT) model...”
He, in the same field of endeavor, discloses using Convolutional Neural Networks with BERT for intent determination. (Page 65, col. 1, 1st 11- Page 68, col. 11st 11) using Convolutional Neural Networks with BERT for intent determination. It would be obvious to one of skill in the art before the effective filing date of the claimed invention to combine the teachings of Jeon, Xiao, Zhang, and Yu with that of He to provide a method wherein a neural network model is a pre-trained BERT model. One would be motivated to do so because, as described by He at Page 66, column 2, 112 using CNN on top of BERT to leverage the effectiveness of BERT for particular tasks, within a CNN solution. Such a modification of Jeon is no more than using a known technique to improve similar methods, products and systems in the same way to achieve predictable results.
As per claims 8 and 14, similar limitations are recited to claim 2, therefore the Rejection, combination, and motivation to combine apply equally to claims 8 and 14.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRIFFIN T BEAN whose telephone number is (703)756-1473. The examiner can normally be reached M - F 7:30 - 4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRIFFIN TANNER BEAN/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights