Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to submission of application on 12/11/2023.
Claims 1-21 are presented for examination.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5, 8-10, 12, 15-17, and 19 are rejected under 35 U.S.C. § 103 as being unpatentable over Bae, et al (Layer-wise hint-based training for knowledge transfer in a teacher-student framework, herein Bae), and Kumar, et al(US 2023/0042750 A1, System and Method for Training a Self-Supervised Ego Vehicle, herein Kumar).
Regarding claim 1,
Bae teaches A method for fine-tuning a neural network (Bae, Figure 2 (A), and, page 242, line 1 “We devise a layer-wise hint training method to improve the existing hint-based knowledge distillation (KD) training approach, which is employed for knowledge transfer in a teacher-student framework using a residual network (ResNet).” And page 248, column 2, paragraph 1, line 2 “Therefore, we introduce supervised layer-wise pretraining of the ResNet (SLResNet), which also comprises two stages: layer-wise training and fine-tuning.”
PNG
media_image1.png
628
542
media_image1.png
Greyscale
PNG
media_image2.png
78
492
media_image2.png
Greyscale
In other words, fine tuning is fine tuning, and layer-wise method is method for fine-tuning a neural network.) , comprising:
executing a plurality of fine-tuning layers of a neural network, each fine-tuning layer corresponding to a respective reference layer of a reference neural network (Bao, page 242, paragraph 1, line 3 “To achieve this objective, the proposed method first iteratively trains the student ResNet and incrementally employs hint-based information extracted from the retrained teacher ResNet containing several hint and guided layers.” In other words, student network is fine-tuning neural network, teacher network is reference neural network, iteratively trains the student ResNet is executing a plurality of fine-tuning layers of a neural network, and from Figure 2 (A) each fine-tuning layer corresponds to a respective reference layer of the reference neural network.) ,
each reference layer associated with a respective reference weight matrix; and for each fine-tuning layer of the plurality of fine-tuning layers: generating a fine-tuning weight matrix based on the reference weight matrix associated with the corresponding reference layer (Bae, Figure 2 (A), and page 243, column 2, paragraph 3, line 5 “The traditional knowledge transfer scheme is composed of two stages: hint training and KD training. First, hint training is achieved by minimizing the following l2 loss function [23]:
PNG
media_image3.png
54
524
media_image3.png
Greyscale
where WH are the weights of a teacher ResNet up to the selected hint layer, WG are the weights of a student ResNet up to the selected guided layer, and FmidH and FmidG represent Nl feature maps (∈ RNh X Nw ) generated from their respective hint and guided layers with WH and WG. Here, Nh and Nw are the height and width of the feature map.” In other words, hint layers are reference layers, guided layers are fine-tuning layers, WH is respective reference weight matrix, and WG is fine-tuning weight matrix, and Equation (1) shows generating the fine-tuning weight matrix based on the reference weight matrix associated with the corresponding reference layer.); and
[iteratively adjusting one or more weights of the fine-tuning weight matrix based on a comparison of output of the fine-tuning layer with output of the corresponding reference layer].
Thus far, Bae does not explicitly teach iteratively adjusting one or more weights of the fine-tuning weight matrix based on a comparison of output of the fine-tuning layer with output of the corresponding reference layer.
Kumar teaches iteratively adjusting one or more weights of the fine-tuning weight matrix based on a comparison of output of the fine-tuning layer with output of the corresponding reference layer (Kumar, abstract, line 10 “A comparator receives the outputs from the first and second machine learning networks based on a particular 2-D image. The
comparator compares the output of the first trained machine learning network with the output of the second machine learning network. A feedback mechanism is operative to alter the second machine learning network based at least in part on the output of the comparator.”, and paragraph, [0077], line 25 “In a second stage 800B, shown in FIG. SC, the learned weights
of teacher network 802 are frozen, and the output of the frozen teacher network 802 is used, at least in part, to train a student network 804.” And, paragraph [0079], line 1 “FIG. 8B shows loss function 816 being provided back into network 806A, where it is used to adjust the weights and biases in network 806 by, for example, minimizing the gradient of the loss function with respect to each weight and bias.” And, paragraph [0090], line 7 “A backward pass of total loss 836 through student depth prediction network 84 is then completed for each corresponding forward pass. The backward pass calculates the gradient of total loss function 836 with respect to the parameters of each node in depth prediction network 804 and utilizes an optimization algorithm to identify the values of those parameters for which the total loss function 836 is minimized for the particular input image. In the example embodiment, a gradient descent algorithm is utilized for this optimization.” And, paragraph [0081], line 1 “FIG. 8C illustrates stage 800B, wherein the output of the frozen teacher network 802 is used, at least in part, to train student network 804.” In other words, gradient descent algorithm is iteratively, adjust the weights is adjusting one or more weights, and compares the output of the first trained machine learning network with the output of the second machine learning network is based on a comparison of output of the fine-tuning layer with output of the corresponding reference layer.)
Both Bae and Kumar are directed to training a neural network from a reference neural network, among other things. Bae teaches a method for fine-tuning a neural network, comprising: executing a plurality of fine-tuning layers of a neural network, each fine-tuning layer corresponding to a respective reference layer of a reference neural network, each reference layer associated with a respective reference weight matrix; and for each fine-tuning layer of the plurality of fine-tuning layers: generating a fine-tuning weight matrix based on the reference weight matrix associated with the corresponding reference layer; but does not explicitly teach iteratively adjusting one or more weights of the fine-tuning weight matrix based on a comparison of output of the fine-tuning layer with output of the corresponding reference layer. Kumar teaches iteratively adjusting one or more weights of the fine-tuning weight matrix based on a comparison of output of the fine-tuning layer with output of the corresponding reference layer.
In view of the teaching of Bae, it would be obvious to one of ordinary skill in the art before the effectively filing date of the claimed invention to combine the teaching of Kumar into Bae. This would result in a method for fine-tuning a neural network, comprising: executing a plurality of fine-tuning layers of a neural network, each fine-tuning layer corresponding to a respective reference layer of a reference neural network, each reference layer associated with a respective reference weight matrix; and for each fine-tuning layer of the plurality of fine-tuning layers: generating a fine-tuning weight matrix based on the reference weight matrix associated with the corresponding reference layer; and iteratively adjusting one or more weights of the fine-tuning weight matrix based on a comparison of output of the fine-tuning layer with output of the corresponding reference layer.
One of ordinary skill in the art would be motivated to do this because supervised learning isn’t always available, and an effective way to perform fine tuning without using supervised learning is by having a second trained network for comparison, which involves an iterative adjustment of the weights. (Kumar, paragraph [0010], line 1 “An example method for training a machine learning framework to estimate the depths of objects captured in a two-dimensional image is disclosed. The example method includes providing a first trained machine learning network and a second untrained or minimally trained machine learning network…The example method additionally includes comparing the estimate of the depths of the objects in the space with the output corresponding to the 2-D image, and altering the second machine learning network based at least in part on the comparison.”)
Regarding claim 2,
The combination of Bao and Kumar teaches the method of claim 1, wherein iteratively adjusting the one or more weights of the fine-tuning weight matrix comprises
iteratively adjusting the one or more weights while keeping constant the reference weight matrix of the associated reference layer (Kumar, paragraph [0090], line 7 “A backward pass of total loss 836 through student depth prediction network 84 is then completed for each corresponding forward pass. The backward pass calculates the gradient of total loss function 836 with respect to the parameters of each node in depth prediction network 804 and utilizes an optimization algorithm to identify the values of those parameters for which the total loss function 836 is minimized for the particular input image. In the example embodiment, a gradient descent algorithm is utilized for this optimization.” And, paragraph [0081], line 1 “FIG. 8C illustrates stage 800B, wherein the output of the frozen teacher network 802 is used, at least in part, to train student network 804.” In other words, teacher network is reference network, gradient descent is iteratively adjusting the one or more weights, and frozen teacher network is while keeping constant the reference weight matrix. Examiner notes that training a neural network is training the weight matrix that comprises the parameters of the neural network) .
Regarding claim 3,
Bao teaches the method of claim 2, wherein iteratively adjusting the one or more weights of the fine-tuning weight matrix comprises
iteratively adjusting the one or more weights while keeping constant the reference weight matrices of one or more preceding reference layers of the reference neural network (Kumar, See above mapping. In other words, teacher network is reference network, gradient descent is iteratively adjusting the one or more weights, and frozen teacher network is while keeping constant the reference weight matrices of one or more preceding reference layers of the reference neural network.).
Regarding claim 5,
The combination of Bae and Kumar teaches the method of claim 1, further comprising
training the reference neural network to generate the respective reference weight matrices (Kumar, paragraph [0061], line 4 “In the example, the teacher network is a deep neural network trained using a regression algorithm. In alternative embodiments, any suitable network/algorithm can be used, including those yet to be invented.” In other words, teacher network is reference neural network, and teacher network is a deep neural network trained using a regression algorithm is training the reference neural network. Examiner notes that training a neural network is training the weight matrix that comprises the parameters of the neural network.) .
Claims 8-10, and 12, are system claims corresponding to method claims 1-3, and 5, respectively. Otherwise, they are not patentably distinct. The combination of Bae and Kumar teaches a system (Kumar, claim 11, line 3 “… said system comprising: at least one hardware processor electrically coupled to execute code, said code including a set of native instructions for causing said hardware processor to perform a corresponding set of operations responsive to executing said set of native instructions; and memory for storing data and said code…” In other words, system is system.). Therefore, claims 8-10, and 12, are rejected for the same reasons as claims 1-3, and 5, respectively.
Claims 15-17, and 19 are non-transitory, computer-readable medium claims corresponding to method claims 1-3, and 5, respectively. Otherwise, they are not patentably distinct. The combination of Bae and Kumar teaches a non-transitory, computer-readable medium (Kumar, claim 11, line 3 “… said system comprising: at least one hardware processor electrically coupled to execute code, said code including a set of native instructions for causing said hardware processor to perform a corresponding set of operations responsive to executing said set of native instructions; and memory for storing data and said code…” In other words, memory for storing data is non-transitory, computer-readable medium.). Therefore, claims 15-17, and 19 are rejected for the same reasons as claims 1-3, and 5, respectively.
Claims 4, 11, and 18 are rejected under 35 U.S.C. § 103 as being unpatentable over Bae, Kumar, O’Neill, et al (Layer-Wise Neural Network Compression via Layer Fusion, herein O’Neill), and Goldt, et al (Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup, herein Goldt).
Regarding claim 4,
The combination of Bao and Kumar teaches the method of claim 1, wherein iteratively adjusting the one or more weights of the fine-tuning weight matrix comprises:
[comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer];
[determining an error rate based on the comparing] ; and
adjusting the one or more weights of the fine-tuning weight matrix (Kumar, See mapping of claim 1. In other words, teacher network is reference network, gradient descent is iteratively adjusting the one or more weights, and frozen teacher network is while keeping constant the reference weight matrices of one or more preceding reference layers of the reference neural network.)
[based on the determined error rate based on the comparing].
Thus far, the combination of Bae and Kumar does not explicitly teach comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer.
O’Neill teaches comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer (O’Neill, page 4, paragraph 2, line 1 “Ideally, we seek a measure that can compare weight matrices that are permutable and of varying length. Formally, the main challenges with aligning weight matrices
W := {W0,…,Wl,…WL} of different layers is that, when vectorized as vec (Wl) Є Rnl(nl+1), Wl can be permuted and still exhibit the same behavior at the output.” In other words, compare weight matrices is comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer.);
Both O ‘Neill, and the combination of Bae and Kumar are directed to training a neural network by using a reference network, among other things. The combination of Bae and Kumar teaches the method of claim 1, but does not explicitly teach comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer. O’Neill teaches comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer.
In view of the teaching of the combination of Bae and Kumar, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of O’Neill into the combination of Bae and Kumar. This would result in the method of claim 1, and comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer.
One of ordinary skill in the art would be motivated to do this in order to reduce computing time, training time and storage space. (O’Neill, page 1, paragraph 2, line 13 “However, these large overparameterized networks require more compute, training time, storage and leave a larger carbon footprint (Strubell et al., 2019). While prior work on model compression has mainly focused on deploying compressed models to mobile devices (Han et al., 2015a; Wu et al., 2016), moving models from multi-GPU training to single-GPU training is now to a salient challenge. If achieved, this relaxes the resource requirements for ML practitioners and allow a wider adoption of larger pretrained CNNs and Transformers.”)
Thus far, the combination of Bae, Kumar, and O’Neill does not explicitly teach determining an error rate based on the comparing.
Goldt teaches determining an error rate based on the comparing (Goldt, page 1, paragraph 4, line 1 “In this paper, we analyse the dynamics of one-pass (or online) SGD in two-layer neural networks. We focus in particular on the influence of over-parameterisation on the final generalisation error. We use the teacher-student framework17,18, where a training data set is generated by feeding random inputs through a two-layer neural network with M hidden units called the teacher. Another neural network, the student, is then trained using SGD on that data set. The generalisation error is defined as the mean squared error between teacher and student outputs, averaged over all of input space.” In other words, generalisation error is error rate, and mean squared error between teach and student is determining an error rate based on the comparing.)
Both Goldt and the combination of Bae, Kumar, and O’Neill are directed to using a reference neural network, among other things. The combination of Bae, Kumar, and O’Neill teaches the method of claim 1 and comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer; but does not explicitly teach determining an error rate based on the comparing. Goldt teaches determining an error rate based on the comparing.
In view of the teaching of the combination of Bae, Kumar, and O’Neill, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Goldt into the combination of Bae, Kumar, and O’Neill. This would result in the method of claim 1, comparing the fine-tuning weight matrix with the reference weight matrix associated with the corresponding reference layer; and determining an error rate based on the comparing.
One of ordinary skill in the art would be motivated to do this to better understand the dynamics of teacher-student frameworks to improve their effectiveness and generalization. (Goldt, page 2, paragraph 2, line 1 “The teacher-student framework is also interesting in the wake of the need to understand the effectiveness of neural networks and the limitations of the classical approaches to generalisation11. Traditional approaches to learning and generalisation are data agnostic and seek worst-case type bounds19.”)
Claim 11 is a system claim that corresponds to method claim 4. Otherwise, they are not patentably distinct. Therefore, claim 11 is rejected for the same reasons as claim 4.
Claims 18 is a non-transitory, computer-readable medium claim that corresponds to method clam 4. Otherwise, they are not patentably distinct. Therefore, claim 18 is rejected for the same reasons as claim 4.
Claims 6-7, 13-14, and 20-21 are rejected under 35 U.S.C. § 103 as being unpatentable over Bae, Kumar, and Kim, et al (SqueezeLLM: Denes-And-Sparse Quantization, herein Kim).
Regarding claim 6,
The combination of Bae and Kumar teaches the method of claim 1, wherein generating the fine-tuning weight matrix comprises
Thus far, the combination of Bae and Kumar does not explicitly teach applying a quantization process to the reference weight matrix associated with the corresponding reference layer.
Kim teaches applying a quantization process to the reference weight matrix associated with the corresponding reference layer (Kim, abstract, line 13 “Our framework incorporates two novel ideas: (i) sensitivity-based non-uniform quantization, which searches for the optimal bit precision assignment based on second-order information; and (ii) the Dense-and-Sparse decomposition that stores outliers and sensitive weight values in an efficient sparse format.” And, page 5, paragraph 4, line 4 “To make the optimization objective in Eq. 3 more
feasible, we further approximate the Fisher information matrix as a diagonal matrix by assuming that the cross-weight interactions are negligible.” In other words, information matrix is weight matrix, and sensitivity-based non-uniform quantization is applying a quantization process to the reference weight matrix. ).
Both Kim and the combination of Bae and Kumar are directed to neural networks and using a reference neural network, among other things. The combination of Bae and Kumar teaches the method of claim 1, but does not explicitly teach applying a quantization process to the reference weight matrix associated with the corresponding reference layer. Kim teaches applying a quantization process to the reference weight matrix associated with the corresponding reference layer.
In view of the teaching of the combination of Bae and Kumar, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Kim into the combination of Bae and Kumar. This would result in the method of claim 1, and applying a quantization process to the reference weight matrix associated with the corresponding reference layer.
One of ordinary skill in the art would be motivated to do this to reduce performance degradation. (Kim, abstract, line 8 “While quantization has emerged as a promising solution by
representing weights with reduced precision, previous efforts have often resulted in notable performance degradation. To address this, we introduce SqueezeLLM, a post-training quantization framework that not only enables lossless compression to ultra-low precisions of up to 3-bit, but also achieves higher quantization performance under the same memory constraint.”)
Regarding claim 7,
The combination of Bae, Kumar, and Kim teaches the method of claim 1, wherein generating the fine-tuning weight matrix includes
applying a sparsification process to the reference weight matrix associated with the corresponding reference layer (Kim, abstract, line 13 “Our framework incorporates two novel ideas: (i) sensitivity-based non-uniform quantization, which searches for the optimal bit precision assignment based on second-order information; and (ii) the Dense-and-Sparse decomposition that stores outliers and sensitive weight values in an efficient sparse format.” In other words, incorporates…the Dense-and-Sparse decomposition is applying a sparsification process to the reference weight matrix. ).
Claims 13-14 are system claims that correspond to method claims 6-7, respectively. Otherwise, they are not patentably distinct. Therefore, claims 13-14 are rejected for the same reasons as claims 6-7, respectively.
Claims 20-21 are non-transitory, computer-readable medium claims that correspond to method claims 6-7, respectively. Otherwise, they are not patentably distinct. Therefore, claims 20-21 are rejected for the same reasons as claims 6-7, respectively.
The prior art made of record and not used is considered pertinent to applicant’s disclosure.
Blakeney, et al “Craft Distillation: Layer-wise Convolutional Neural Network Distillation” discloses craft distillation, a novel model compression approach that leverages both depthwise separable convolutions and knowledge distillation to significantly reduce the size of a highly complex model.
el-Bouri, et al “Student-Teacher Curriculum Learning via Reinforcement Learning: Predicting Hospital Inpatient Admission Location” discloses a representation of the weights of the student network that is treated as the state and is fed as an input to the teacher network. The teacher network’s action is to select the most appropriate batch of data to train the student network on from a training set sorted according to entropy.
Huang, T., et al “Uncertainty-Driven Knowledge Distillation for Language Model Compression” discloses a novel and efficient uncertainty-driven knowledge distillation compression method for transformer-based pretrained language models.
Kozlov, A., et al “Neural Network Compression Framework for Fast Model Inference” discloses a new PyTorch-based framework for neural network compression with fine-tuning named Neural Network Compression Framework (NNCF) that leverages recent advances of various network compression methods and implements some of them, namely quantization, sparsity, filter pruning and binarization.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BART RYLANDER whose telephone number is (571)272-8359. The examiner can normally be reached Monday - Thursday 8:00 to 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Bart I Rylander/Examiner, Art Unit 2124
Prosecution Insights