Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 7/11/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 6-8, 13, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Weizhu Chen et al; US 20220383126 A1 filed on May 19, 2021 (hereinafter “Chen”) in view of Longteng Zhang et al; “LORA-FA: MEMORY-EFFICIENT LOW-RANK ADAPTA TION FOR LARGE LANGUAGE MODELS FINE-TUNING” published Aug 7, 2023 (hereinafter “Zhang”)
Regarding claim 1, Chen teaches One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising: (Chen Paragraph 0012; “The functions or algorithms described herein may be implemented in software in one embodiment. The software may consist of computer executable instructions stored on computer readable media or computer readable storage device such as one or more non-transitory memories or other type of hardware based storage devices, either local or networked… The software may be executed on a digital signal processor, ASIC, microprocessor, or other type of processor operating on a computer system”)
accessing a neural network model with a pre-trained matrix at a layer of the neural network model; (Chen Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120.” Chen Paragraph 0030; “Method 200 begins with operation 210 by obtaining neural network-based language model base model weight matrices for each of multiple neural network layers.” Examiner notes that a neural network model (neural network-based language model) is accessed to obtain pre-trained matrix at a layer of the neural network (weight matrices for each of multiple neural network layers))
adding a shared matrix and a task matrix to the pre-trained matrix at the layer of the neural network model; and (Chen Fig 1 and Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120... The input vector is also processed by a pair of rank decomposition matrices, matrix A 125 and matrix B 130… where it is combined with the result of the pretrained weight 110 matrix to provide f(x), the input to the next layer in the neural network.” Examiner notes that a shared matrix (matrix A) and a task matrix (matrix b) is added to the pretrained matrix (pretrained weight matrix) at the layer of the neural network model (dense layer 100))
PNG
media_image1.png
473
465
media_image1.png
Greyscale
training the neural network model for a plurality of tasks by updating the task matrix for each task to provide a trained task matrix for each task [while maintaining the pre-trained matrix and the shared matrix the same for all tasks]. (Chen Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120… Matrices A and B may be referred to as adaptation matrices, as they adapt the general model to the specific task or domain.” Chen Paragraph 0019; “LoRA possesses several key advantages. A single pretrained model can be shared and used to build many small adaptations for different tasks.” Examiner notes that training the neural network model (language model neural network) for a plurality of tasks (different tasks) by updating/training the task matrix (matrix B) for each task to provide a trained task matrix for each task)
Chen does not teach while maintaining the pre-trained matrix and the shared matrix the same for all tasks
However, Zhang does teach while maintaining the pre-trained matrix and the shared matrix the same for all tasks (Zhang Figure 1 and Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B.” Examiner notes that the pre-trained matrix (pre-trained W) and the shared matrix (matrix A) is maintained the same/frozen for all tasks)
PNG
media_image2.png
226
528
media_image2.png
Greyscale
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Regarding claim 6, Chen teaches The one or more computer storage media of claim 1, wherein parameters of the shared matrix are initialized with random Gaussian values. (Chen Paragraph 0028; “the simple factorization can be applied to every dense layer using a random Gaussian initialization for A and zero for B, so ΔW is zero at the beginning of training.” Examiner notes that parameters of the shared matrix (matrix A) are initialized with random Gaussian values (random Gaussian initialization for A))
Regarding claim 7, Chen teaches The one or more computer storage media of claim 1, wherein parameters of the task matrix are initialized with zero values. (Examiner refers to previous mapping to show that parameters of the task matrix (matrix B) are initialized with zero values (zero for B))
Regarding claim 8, Chen teaches The one or more computer storage media of claim 1, wherein the neural network model comprises a second pre-trained matrix at a second layer, and wherein the operations further comprise: (Chen Paragraph 0018; “LoRA allows the training of each of multiple dense layers in the neural network indirectly by injecting and optimizing their rank decomposition matrices A and B, while keeping the original matrices of pretrained weights 110, unchanged.” Examiner notes that the neural network model comprises a second pre-trained matrix (original matrices of pretrained weights 110 corresponding to each layer) at a second layer (each of multiple dense layers suggests a second layer); operations done to first layer can be similarly performed on second layer)
adding a second shared matrix and a second task matrix to the second pre-trained matrix at the second layer of the neural network model; and (Chen Fig 1 and Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120... The input vector is also processed by a pair of rank decomposition matrices, matrix A 125 and matrix B 130… where it is combined with the result of the pretrained weight 110 matrix to provide f(x), the input to the next layer in the neural network.” Examiner notes that a second shared matrix (matrix A) and a second task matrix (matrix b) is added to the pretrained matrix (pretrained weight matrix) at the layer of the neural network model (dense layer 100); Examiner interprets second task matrix as a task matrix that corresponds to the second layer)
wherein the neural network model is trained for the plurality of tasks by also updating the second task matrix for each task to provide a trained second task matrix for each task [while maintaining the second pre-trained matrix and the second shared matrix for all tasks]. (Chen Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120… Matrices A and B may be referred to as adaptation matrices, as they adapt the general model to the specific task or domain.” Chen Paragraph 0019; “LoRA possesses several key advantages. A single pretrained model can be shared and used to build many small adaptations for different tasks.” Examiner notes that training the neural network model (language model neural network) for a plurality of tasks (different tasks) by updating/training the second task matrix (matrix B) for each task to provide a trained second task matrix for each task)
Chen does not teach while maintaining the second pre-trained matrix and the second shared matrix the same for all tasks
However, Zhang does teach while maintaining the second pre-trained matrix and the second shared matrix the same for all tasks (Zhang Figure 1 and Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B.” Examiner notes that the pre-trained matrix (pre-trained W) and the shared matrix (matrix A) is maintained the same/frozen for all tasks)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Regarding claim 13, Chen teaches A computer system comprising: one or more processors; and one or more computer storage media storing computer-useable instructions that, when used by the one or more processors, causes the computer system to perform operations comprising: (Chen Paragraph 0012; “The functions or algorithms described herein may be implemented in software in one embodiment. The software may consist of computer executable instructions stored on computer readable media or computer readable storage device such as one or more non-transitory memories or other type of hardware based storage devices, either local or networked… The software may be executed on a digital signal processor, ASIC, microprocessor, or other type of processor operating on a computer system”)
accessing a neural network model having a first-layer pre-trained matrix for a first layer of the neural network model; (Chen Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120.” Chen Paragraph 0030; “Method 200 begins with operation 210 by obtaining neural network-based language model base model weight matrices for each of multiple neural network layers.” Examiner notes that a neural network model (neural network-based language model) is accessed to obtain pre-trained matrix at a layer of the neural network (weight matrices for each of multiple neural network layers))
supplementing, by a matrix supplementation component, the first-layer pre-trained matrix with a first-layer shared matrix and a first-layer task matrix to provide a supplemented neural network model; (Chen Fig 1 and Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120... The input vector is also processed by a pair of rank decomposition matrices, matrix A 125 and matrix B 130… where it is combined with the result of the pretrained weight 110 matrix to provide f(x), the input to the next layer in the neural network.” Examiner notes that a shared matrix (matrix A) and a task matrix (matrix b) is added/supplemented to the pretrained matrix (pretrained weight matrix) at the first layer of the neural network model (dense layer 100) to provide a supplemented neural network model (layer has been supplemented meaning model is supplemented as well); matrix supplementation component is interpreted as processor performing the supplementing)
fine-tuning, by the task-specific model training component, the supplemented neural network model for a first task by updating parameters of the first-layer task matrix using a first set of training data to provide a trained first-layer first-task matrix. (Chen Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120… Matrices A and B may be referred to as adaptation matrices, as they adapt the general model to the specific task or domain.” Chen Paragraph 0033; “The second low-rank factorization matrices were obtained in a manner similar first low-rank factorization matrices by training with second domain specific training data without modifying base model weight matrices.” Examiner notes that training/fine-tuning the neural network model (language model neural network) for a first task (specific task) by updating parameters of the first task matrix (matrix B) using a first set of training data (specific training data) to provide a trained first-layer first-task matrix)
Chen does not teach freezing, by a task-specific model training component, parameters of the first-layer pre-trained matrix and parameters of the first-layer shared matrix; and
However, Zhang does teach freezing, by a task-specific model training component, parameters of the first-layer pre-trained matrix and parameters of the first-layer shared matrix; and (Zhang Figure 1 and Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B.” Examiner notes that the pre-trained matrix (pre-trained W) and the shared matrix (matrix A) is maintained the same/frozen for all tasks; task-specific model training component is interpreted as processor performing the freezing)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Regarding claim 19, Chen teaches The computer system of claim 13, wherein the parameters of the first-layer shared matrix are initialized with random Gaussian values, and wherein the parameters of the first-layer task matrix are initialized with zero values. (Chen Paragraph 0028; “the simple factorization can be applied to every dense layer using a random Gaussian initialization for A and zero for B, so ΔW is zero at the beginning of training.” Examiner notes that parameters of the shared matrix (matrix A) are initialized with random Gaussian values (random Gaussian initialization for A) and parameters of the task matrix (matrix B) are initialized with zero values (zero for B))
Regarding claim 20, Chen teaches The computer system of claim 13, wherein the neural network model comprises a second-layer pre-trained matrix for a second layer, and wherein the operations further comprise: (Chen Paragraph 0018; “LoRA allows the training of each of multiple dense layers in the neural network indirectly by injecting and optimizing their rank decomposition matrices A and B, while keeping the original matrices of pretrained weights 110, unchanged.” Examiner notes that the neural network model comprises a second pre-trained matrix (original matrices of pretrained weights 110 corresponding to each layer) at a second layer (each of multiple dense layers suggests a second layer); operations done to first layer can be similarly performed on second layer)
supplementing the second-layer pre-trained matrix with a second-layer shared matrix and a second-layer task matrix to provide the supplemented neural network model; (Chen Fig 1 and Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120... The input vector is also processed by a pair of rank decomposition matrices, matrix A 125 and matrix B 130… where it is combined with the result of the pretrained weight 110 matrix to provide f(x), the input to the next layer in the neural network.” Examiner notes that a second-layer shared matrix (matrix A) and a second-layer task matrix (matrix b) is added/supplemented to the second-layer pretrained matrix (pretrained weight matrix) at the second-layer of the neural network model (dense layer 100) to provide a supplemented neural network model (layer has been supplemented meaning model is supplemented as well))
and wherein the supplemented neural network model is fine-tuned for the first task by also updating parameters of the second-layer task matrix using the first set of training data to provide a trained second-layer first-task matrix. (Chen Paragraph 0017; “FIG. 1 is a block diagram illustrating the training of a dense layer 100 of a language model neural network to adapt a general model with a matrix of pretrained weights 110 for processing an input vector x at 115 with a function f(x) at 120… Matrices A and B may be referred to as adaptation matrices, as they adapt the general model to the specific task or domain.” Chu Paragraph 0033; “The second low-rank factorization matrices were obtained in a manner similar first low-rank factorization matrices by training with second domain specific training data without modifying base model weight matrices.” Examiner notes that the first-layer task matrix is updated/fine-tuned using a first set of training data (domain specific training data) to provide a trained first-layer first-task matrix (second low-rank factorization matrices that includes matrix B))
Chen does not teach freezing parameters of the second-layer pre-trained matrix and the second-layer shared matrix;
However, Zhang does teach freezing parameters of the second-layer pre-trained matrix and the second-layer shared matrix; (Zhang Figure 1 and Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B.” Examiner notes that the second-layer pre-trained matrix (pre-trained W) and the second-layer shared matrix (matrix A) is maintained the same/frozen for all tasks)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Claim(s) 2-5, 9-12, and 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Weizhu Chen et al; US 20220383126 A1 filed on May 19, 2021 (hereinafter “Chen”) in view of Longteng Zhang et al; “LORA-FA: MEMORY-EFFICIENT LOW-RANK ADAPTA TION FOR LARGE LANGUAGE MODELS FINE-TUNING” published Aug 7, 2023 (hereinafter “Zhang”) in further view of Hongyang Yu; US 12481677 B1 filed on Dec 27, 2023 (hereinafter “Yu”).
Regarding claim 2, Chen teaches performing the first task using the pre-trained matrix, the shared matrix, and the trained task matrix for the first task at the layer of the neural network model. (Chen Paragraph 0032; “At operation 240, inferencing on first domain language input is performed using the trained first domain language model that includes the base model weight matrices and corresponding first low-rank factorization matrices.” Examiner notes that that the first task (inferencing on first domain language) is performed using the pretrained matrix (base model weight matrices), the shared matrix, and the trained task matrix for the first task at the layer of the neural network (corresponding first-low rank factorization matrices include matrices A and B respectively))
Chen does not teach The one or more computer storage media of claim 1, wherein the operations further comprise: receiving an input for a first task from the plurality of tasks; and
However, Yu does teach The one or more computer storage media of claim 1, wherein the operations further comprise: receiving an input for a first task from the plurality of tasks; and (Yu Column 8 Line 56; “The predictive model 100 is configured to receive input data 110 and output overall inference 170.” Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes an input (input data 110) is received for a first task from the plurality of tasks (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 3, Chen teaches accessing the trained task matrix in response to determining the input corresponds to the first task. (Chen Paragraph 0032; “At operation 240, inferencing on first domain language input is performed using the trained first domain language model that includes the base model weight matrices and corresponding first low-rank factorization matrices.” Examiner notes that the trained task matrix (matrix B within the corresponding first low-rank factorization matrices) is accessed in response to determining the input corresponds to the first task)
Chen does not teach The one or more computer storage media of claim 2, wherein the operations further comprise: determining the input corresponds to the first task; and
However, Yu does teach The one or more computer storage media of claim 2, wherein the operations further comprise: determining the input corresponds to the first task; and (Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes that the input (input data 110) is determined to correspond to the first task (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 4, Chen teaches performing the second task using the pre-trained matrix, the shared matrix, and the trained task matrix for the second task at the layer of the neural network model. (Chen Paragraph 0034; “Operation 330 performs inferencing on second domain language input using the base model weight matrices and corresponding second low-rank factorization matrices.” Examiner notes that the second task (inferencing on second domain language) is performed using the pretrained matrix (base model weight matrices), the shared matrix, and the trained task matrix for the second task at the layer of the neural network (corresponding second-low rank factorization matrices include matrices A and B respectively))
Chen does not teach The one or more computer storage media of claim 2, wherein the operations further comprise: receiving an input for a second task from the plurality of tasks; and
However, Yu does teach The one or more computer storage media of claim 2, wherein the operations further comprise: receiving an input for a second task from the plurality of tasks; and (Yu Column 8 Line 56; “The predictive model 100 is configured to receive input data 110 and output overall inference 170.” Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes an input (input data 110) is received for a second task from the plurality of tasks (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 5, Chen teaches The one or more computer storage media of claim 4, wherein the second task is performed after performing the first task by: subtracting the trained task matrix for the first task [to obtain the pre-trained matrix and the shared matrix]; and (Chen Paragraph 0033; “FIG. 3 is a flowchart illustrating a method 300 of switching between domain models that utilize low-rank factorization matrices. Method 300 begins with removing the first low-rank factorization matrices at operation 310. Second low-rank factorization matrices are added to the base model weight matrices at operation 320.” Examiner notes that the trained task matrix (matrix B of the first low-rank factorization matrices) is subtracted/removed for the first task)
adding the trained task matrix for the second task. (Examiner refers to previous mapping to show that the trained task matrix for the second task (matrix B of the second low-rank factorization matrices) is added)
Chen does not teach to obtain the pre-trained matrix and the shared matrix
However, Zhang does teach to obtain the pre-trained matrix and the shared matrix (Zhang Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B. Thus, the change of weight during model adaptation will be constrained in a low-rank space” Examiner notes that matrix A (of the low-rank factorization in Chen) and pre-trained W is frozen, meaning when subtracting/updating matrix B (projection up weight B), the pre-trained matrix and the shared matrix is leftover during the updating of matrix B)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Regarding claim 9, Chen teaches retrieving, from a data store, a task matrix for a layer of a neural network model based on the task type, the layer of the neural network model including a pre-trained matrix and a shared matrix, [the shared matrix being shared by one or more other tasks]; (Chen Paragraph 0032; “At operation 240, inferencing on first domain language input is performed using the trained first domain language model that includes the base model weight matrices and corresponding first low-rank factorization matrices.” Chen Paragraph 0039; “During deployment, the service loads the pre-trained model into memory and store (potentially hundreds of) LoRA modules, each corresponding to a particular task, on stand-by. A task can also be specialized to different customers and stores in different LoRA modules.” Examiner notes that a LoRa module is retrieved from a data store (memory); LoRA module includes a task matrix for a layer of a neural network model based on the task type (matrix B), the layer of the neural network model including a pre-trained matrix (pretrained weights) and a shared matrix (matrix A))
and performing the task for the input using the neural network model with the pre-trained matrix, the shared matrix, and the task matrix at the layer of the neural network model. (Chen Paragraph 0032; “At operation 240, inferencing on first domain language input is performed using the trained first domain language model that includes the base model weight matrices and corresponding first low-rank factorization matrices.” Examiner notes that that the first task (inferencing on first domain language) is performed using the pretrained matrix (base model weight matrices), the shared matrix, and the trained task matrix for the first task at the layer of the neural network (corresponding first-low rank factorization matrices include matrices A and B respectively))
Chen does not teach the shared matrix being shared by one or more other tasks
However, Zhang does teach the shared matrix being shared by one or more other tasks (Zhang Figure 1 and Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B.” Examiner notes that by keeping the shared matrix (matrix A) it means the shared matrix is the same/shared by one or more other tasks)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Chen in view of Zhang does not teach A computer-implemented method comprising: receiving, at a model inference component, an input for a task;
However, Yu does teach A computer-implemented method comprising: receiving, at a model inference component, an input for a task; (Yu Column 8 Line 56; “The predictive model 100 is configured to receive input data 110 and output overall inference 170.” Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes an input (input data 110) is received at a model inference component (router layer) for a first task from the plurality of tasks (determine which one or more machine learning methods to select for classification tasks))
determining, by the model inference component, a task type for the input; (Examiner refers to previous mapping to show that the input (input data 110) is determined, by the model inference component (router layer), to correspond to the first task (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 10, Chen teaches The computer-implemented method of claim 9, wherein the neural network model includes a second layer having a second pre-trained matrix and a second shared matrix, [the second shared matrix being shared by the one or more other tasks], and wherein the method further comprises: (Chen Paragraph 0018; “LoRA allows the training of each of multiple dense layers in the neural network indirectly by injecting and optimizing their rank decomposition matrices A and B, while keeping the original matrices of pretrained weights 110, unchanged.” Examiner notes that the neural network model comprises a second pre-trained matrix (original matrices of pretrained weights 110 corresponding to each layer) at a second layer (each of multiple dense layers suggests a second layer); operations done to first layer can be similarly performed on second layer)
retrieving, from the data store, a second task matrix for the second layer of the neural network model based on the task type; and wherein the task is performed for the input using the second pre-trained matrix, the second shared matrix, and the second task matrix at the second layer of the neural network model. (Chen Paragraph 0032; “At operation 240, inferencing on first domain language input is performed using the trained first domain language model that includes the base model weight matrices and corresponding first low-rank factorization matrices.” Examiner notes that that the first task (inferencing on first domain language) is performed using the pretrained matrix (base model weight matrices), the shared matrix, and the trained task matrix for the first task at the layer of the neural network (corresponding first-low rank factorization matrices include matrices A and B respectively); a second task matrix for the second layer of the neural network based on the task type (matrix B of the low-rank factorization matrices) must be retrieved to me used)
Chen does not teach the second shared matrix being shared by the one or more other tasks
However, Zhang does teach the second shared matrix being shared by the one or more other tasks (Zhang Figure 1 and Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B.” Examiner notes that by keeping the shared matrix (matrix A) it means the shared matrix is the same/shared by one or more other tasks)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Regarding claim 11, Chen teaches retrieving, from the data store, a second task matrix for the layer of the neural network model based on the second task type; and performing the second task for the second input using the neural network model with the pre-trained matrix, the shared matrix, and the second task matrix at the layer of the neural network model. (Chen Paragraph 0034; “Operation 330 performs inferencing on second domain language input using the base model weight matrices and corresponding second low-rank factorization matrices.” Examiner notes that the second task (inferencing on second domain language) is performed using the pretrained matrix (base model weight matrices), the shared matrix, and the trained task matrix for the second task at the layer of the neural network (corresponding second-low rank factorization matrices include matrices A and B respectively); a second task matrix for the layer of the neural network based on the second task type (matrix B of the low-rank factorization matrices) must be retrieved to me used; Examiner interprets the second task matrix as the task matrix corresponding to the second task)
Chen does not teach The computer-implemented method of claim 9, wherein the method further comprises: receiving a second input for a second task;
determining, by the model inference component, a second task type for the second input;
However, Yu does teach The computer-implemented method of claim 9, wherein the method further comprises: receiving a second input for a second task; (Yu Column 8 Line 56; “The predictive model 100 is configured to receive input data 110 and output overall inference 170.” Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes an input (input data 110) is received for a second task from the plurality of tasks (determine which one or more machine learning methods to select for classification tasks))
determining, by the model inference component, a second task type for the second input; (Examiner refers to previous mapping to show that the second input (input data 110) is determined to correspond to the second task (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 12, Chen teaches The computer-implemented method of claim 11, wherein performing the second task for the second input comprises: subtracting the task matrix [to obtain the pre-trained matrix and shared matrix]; (Chen Paragraph 0033; “FIG. 3 is a flowchart illustrating a method 300 of switching between domain models that utilize low-rank factorization matrices. Method 300 begins with removing the first low-rank factorization matrices at operation 310. Second low-rank factorization matrices are added to the base model weight matrices at operation 320.” Examiner notes that the trained task matrix (matrix B of the first low-rank factorization matrices) is subtracted/removed for the first task)
and adding the second task matrix. (Examiner refers to previous mapping to show that the trained task matrix for the second task (matrix B of the second low-rank factorization matrices) is added)
Chen does not teach to obtain the pre-trained matrix and the shared matrix
However, Zhang does teach to obtain the pre-trained matrix and the shared matrix (Zhang Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B. Thus, the change of weight during model adaptation will be constrained in a low-rank space” Examiner notes that matrix A (of the low-rank factorization in Chen) and pre-trained W is frozen, meaning when subtracting/updating matrix B (projection up weight B), the pre-trained matrix and the shared matrix is leftover during the updating of matrix B)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Regarding claim 14, Chen teaches and performing the first task using the first-layer pre-trained matrix, the first-layer shared matrix, and the trained first-layer first-task matrix. (Chen Paragraph 0032; “At operation 240, inferencing on first domain language input is performed using the trained first domain language model that includes the base model weight matrices and corresponding first low-rank factorization matrices.” Examiner notes that that the first task (inferencing on first domain language) is performed using the pretrained matrix (base model weight matrices), the shared matrix, and the trained task matrix for the first task at the layer of the neural network (corresponding first-low rank factorization matrices include matrices A and B respectively))
Chen does not teach The computer system of claim 13, wherein the operations further comprise: receiving an input for the first task;
However, Yu does teach The computer system of claim 13, wherein the operations further comprise: receiving an input for the first task; (Yu Column 8 Line 56; “The predictive model 100 is configured to receive input data 110 and output overall inference 170.” Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes an input (input data 110) is received for a first task from the plurality of tasks (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 15, Chen teaches accessing the trained first-layer first-task matrix in response to determining the input corresponds to the first task. (Chen Paragraph 0032; “At operation 240, inferencing on first domain language input is performed using the trained first domain language model that includes the base model weight matrices and corresponding first low-rank factorization matrices.” Examiner notes that the trained task matrix (matrix B within the corresponding first low-rank factorization matrices) is accessed in response to determining the input corresponds to the first task)
Chen does not teach The computer system of claim 14, wherein the operations further comprise: determining the input corresponds to the first task; and
However, Yu does teach The computer system of claim 14, wherein the operations further comprise: determining the input corresponds to the first task; and (Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes that the input (input data 110) is determined to correspond to the first task (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 16, Chen teaches The computer system of claim 13, wherein the operations further comprise: fine-tuning the supplemented neural network model for a second task by updating the first-layer task matrix using a second set of training data to provide a trained first-layer second-task matrix. (Chen Paragraph 0033; “The second low-rank factorization matrices were obtained in a manner similar first low-rank factorization matrices by training with second domain specific training data without modifying base model weight matrices.” Examiner notes that the first-layer task matrix is updated using a second set of training data (second domain specific training data) to provide a trained first-layer second-task matrix (second low-rank factorization matrices that includes matrix B))
Regarding claim 17, Chen teaches and performing the second task using the first-layer pre-trained matrix, the first-layer shared matrix, and the trained first-layer second-task matrix. (Chen Paragraph 0034; “Operation 330 performs inferencing on second domain language input using the base model weight matrices and corresponding second low-rank factorization matrices.” Examiner notes that the second task (inferencing on second domain language) is performed using the pretrained matrix (base model weight matrices), the shared matrix, and the trained task matrix for the second task at the layer of the neural network (corresponding second-low rank factorization matrices include matrices A and B respectively))
Chen does not teach The computer system of claim 16, wherein the operations further comprise: receiving an input for the second task;
However, Yu does teach The computer system of claim 16, wherein the operations further comprise: receiving an input for the second task; (Yu Column 8 Line 56; “The predictive model 100 is configured to receive input data 110 and output overall inference 170.” Yu Column 9 Line 26; “In various embodiments, router layer 140 may determine which one or more machine learning methods to select for classification tasks based on the processing of input data 110.” Examiner notes an input (input data 110) is received for a second task from the plurality of tasks (determine which one or more machine learning methods to select for classification tasks))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen, Zhang, and Yu. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. Yu teaches low rank adaptation in multiclass deep learning classifiers. One of ordinary skill would have motivation to combine Chen, Zhang, and Yu to reduce the computation burden on resources and devices “This approach results in a prediction model that significantly advances data classification capabilities while reducing the computational burden on resources and devices. Leveraging this prediction model improves the efficiency of computing resources, including memory and processing power.” (Yu Column 5 Line 13).
Regarding claim 18, Chen teaches The computer system of claim 17, wherein the second task is performed after performing the first task by: subtracting the trained first-layer first-task matrix [to obtain the first-layer pre-trained matrix and the first-layer shared matrix]; (Chen Paragraph 0033; “FIG. 3 is a flowchart illustrating a method 300 of switching between domain models that utilize low-rank factorization matrices. Method 300 begins with removing the first low-rank factorization matrices at operation 310. Second low-rank factorization matrices are added to the base model weight matrices at operation 320.” Examiner notes that the trained task matrix (matrix B of the first low-rank factorization matrices) is subtracted/removed for the first task)
and adding the trained first-layer second-task matrix. (Examiner refers to previous mapping to show that the trained task matrix for the second task (matrix B of the second low-rank factorization matrices) is added)
Chen does not teach to obtain the pre-trained matrix and the shared matrix
However, Zhang does teach to obtain the pre-trained matrix and the shared matrix (Zhang Page 4 Paragraph 6; “During the fine-tuning process, as shown in Figure 1(c), we freeze the initialized A and the pre-trained W, and update the projection-up weight B. Thus, the change of weight during model adaptation will be constrained in a low-rank space” Examiner notes that matrix A (of the low-rank factorization in Chen) and pre-trained W is frozen, meaning when subtracting/updating matrix B (projection up weight B), the pre-trained matrix and the shared matrix is leftover during the updating of matrix B)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Chen and Zhang. Chen teaches low-rank adaptation of neural network models. Zhang teaches LoRA-FA where they choose to freeze the projection-down weight of A and update the projection-up weight of B in each LoRA layer. One of ordinary skill would have motivation to combine Chen and Zhang to reduce the overall memory cost “Our results show that LoRA-FA can always achieve close fine-tuning accuracy across different tasks compared to full parameter fine-tuning and LoRA. Furthermore, LoRA-FA can reduce the overall memory cost by up to 1.4× compared to LoRA.” (Zhang Abstract).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL DUC TRAN whose telephone number is (571)272-6870. The examiner can normally be reached Mon-Fri 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.D.T./Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
