NON-FINAL REJECTION, FIRST DETAILED ACTION
Status of Prosecution
The present application 18/571,616, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The application was filed in the Office on Dec. 18, 2023 and is a 371 national stage application of PCT/CN2021/135129 filed on Dec. 2, 2021.
Claims 1-19 and 26 are pending and are all rejected in this rejection. Claims 1, 13 and 26 are independent claims. Claims 20-25 are canceled by preliminary amendment.
Status of Claims
Claims 20-25 are canceled by preliminary amendment.
Claim 7 and 19 are objected to.
Claims 1-19 and 26 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1-3, 8-10, 12-15 and 26 are rejected under 35 USC § 103 as being unpatentable over Chen et al. (“Chen”), United States Patent Application Publication 2019/0130275 published on May 2, 2019 in view of non-patent literature, Liu et al. (“Liu”), “End-end-end Multi-Task Learning with Attention” published in 2019 and in further view of non-patent literature, Verboven et al. (“Verboven”), “HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxilary Tasks,” published in 2020.
Claims 4-6 and 16-18 are rejected under 35 USC § 103 as being unpatentable over Chen in view of Liu in view Verboven and in further view of Shayan et al. (“Shayan”), “Multi-step Forecasting via Multi-task Learning,” published in 2019.
Claim 11 is rejected under 35 USC § 103 as being unpatentable over Chen in view of Liu in view Verboven and in further view of Liu et al. (“Liu ‘164”), United States Patent Application Publication 2021/0142164, published on May 13, 2021.
Objection – Allowable Subject Matter
Notwithstanding non-prior art rejections, Claims 7 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections – 35 USC § 101 – Subject Matter Eligibility
Claims 1-19 and 26 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding representative claim 1, at step 1, the claim recites a computer system, and therefore is a maufacture, which is a statutory category of invention. See MPEP § 2106.03.
At step 2A, prong one, the claim recites a system that is capable of making assessments.
The following limitations are the abstract idea of a mathematical calculation. See MPEP § 2106.04(a)(2)(I)(C):
determine a custom interval including a designated number of mini-batch training operations and a designated window of N custom intervals, wherein N is an integer greater than 1;
calculate, for each task, a loss change rate between each pair of N−1 pairs of neighboring custom intervals within a designated window prior to a present custom interval;
calculate, for each task, a gradient magnitude with respect to selected shared weights within the designated window prior to the present custom interval; and
adjust, for each task, a weight of the task, based on the loss change rate between each pair of the N−1 pairs of neighboring custom intervals within the designated window prior to the present custom interval for the task, and the gradient magnitude with respect to selected shared weights within the designated window prior to the present custom interval for each task.
At step 2A prong 2, the claim language is analyzed to determine whether it recites additional elements that integrate the judicial exception into a practical application. See MPEP § 2106.04(d).
The limitations:
interface circuitry to receive a pre-trained neural network;
instructions; and
processor circuitry to execute the instructions to:
initialize parameters of shared layers of a deep neural network for MTL using the pre-trained neural network;
This that are related to a user interface display which is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g).
Next, at step 2B of the analysis, the claim is considered if it recites additional elements that amount to significantly more than the judicial exception. See MPEP § 2106.05.
The additional element of a generic computer processing is one the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is therefore directed to an abstract idea.
Therefore, claim 1 is ineligible.
As to dependent claims 2-8 the analysis of the respective parent claim is incorporated. In the step 2A, prong one analysis, the additional limitations deal with calculations which are all the abstract idea of a mathematical calculation or are mental processes. See MPEP § 2106.04(a)(2).
The claims are also ineligible.
As to dependent claims 9-12, the analysis of the respective parent claim is incorporated. In the step 2A, prong two analysis, the additional limitation deals with details of how the weights, pre-training of the neural networks are either calculated or set; these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is therefore directed to an abstract idea.
The claim is also ineligible.
As to independent claims 13 and 26, they are rejected for similar reasons as claim 5. Their dependent claims are rejected similarly to their corresponding dependent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
-A.
Claims 1-3, 8-10, 12-15 and 26 are rejected under 35 USC § 103 as being unpatentable over Chen et al. (“Chen”), United States Patent Application Publication 2019/0130275 published on May 2, 2019 in view of non-patent literature, Liu et al. (“Liu”), “End-end-end Multi-Task Learning with Attention published in 2019 and in further view of non-patent literature, Verboven et al. (“Verboven”), “HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxilary Tasks,” published in 2020.
As to Claim 1, Chen teaches: An apparatus for loss balancing in multi-task learning (MTL), comprising:
interface circuitry to receive a pre-trained neural network;
instructions; and
processor circuitry (Chen: par. 0006, processor in communication with memory, the processor programmed with executable instructions) to execute the instructions to:
initialize parameters of shared layers of a deep neural network for MTL using the pre-trained neural network (Chen: par. 0022, multiple tasks ban be learned in parallel using a shared representation among tasks, for instance sharing hidden layers among all tasks and thus parameters);
calculate, for each task, a gradient magnitude with respect to selected shared weights (Chen: par. 0037, GradNorm can establish a common scale gradient magnitude and then balance trainings for different tasks); and
adjust, for each task, a weight of the task, based on the gradient magnitude with respect to selected shared weights for each task (Chen: par. 0037, eq. 1, the weights are adjusted accordingly).
Chen may not explicitly teach: calculate, for each task, a loss change rate between each pair of N−1 pairs of neighboring custom intervals;
adjust, for each task, a weight of the task, based on the loss change rate between each pair of the N−1 pairs of neighboring custom intervals, and the gradient magnitude with respect to selected shared weights.
Liu teaches in general concepts related to a multi-task learning architecture that uses learning of task-features from global features and allowing the features to be shared across different tasks (Liu: Abstract). Specifically, Liu teaches that a dynamic weight average (DWA) approach loss change rate is considered for each task (Liu: sec. 4.1.3). The weighting for a task is considered for a neighboring time (Liu: sec. 4.1.3, equation (7) notes for the weighting at a certain iteration index, the neighboring indices are considered). The weight for each is adjusted accordingly (Liu: Sec. 4.1.3).
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have modified Chen’s disclosures and teachings by adjusting for each task the weight also based on the loss change rate as taught by Liu. Such a person would have been motivated to do so with a reasonable expectation of success to combine both the benefits of a gradient magnitude adjustment with the loss rate task-focused approach for a blended approach.
Chen and Liu may not explicitly teach: determine a custom interval including a designated number of mini-batch training operations and a designated window of N custom intervals, wherein N is an integer greater than 1;
calculate, for each task, a loss change rate between each pair of N−1 pairs of neighboring custom intervals within a designated window prior to a present custom interval;
adjust, for each task, a weight of the task, based on the loss change rate between each pair of the N−1 pairs of neighboring custom intervals within the designated window prior to the present custom interval for the task, and the gradient magnitude with respect to selected shared weights within the designated window prior to the present custom interval for each task.
Verboven teaches in general concepts related to multi-task learning and an intelligent weighting algorithm that connects multi-task gain to the individual task gradients (Verboven: Abstract). Specifically Verboven, teaches that small mini-batch learning may be used over various iterations (intervals) (Verboven: Fig. 1, “Small Mini-Batch Learning” et seq. discussion).
PNG
media_image1.png
342
484
media_image1.png
Greyscale
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have modified Chen-Liu’s disclosures and teachings by adjusting for each task the weight also within custom intervals of mini-batches as required by the claim and as taught and suggested by Verboven. Such a person would have been motivated to do so with a reasonable expectation of success to to determine the optimal loss weights for each mini-batch separately (Verboven: HydaLearn, Proposed Solution).
As to Claim 2, Chen, Liu and Verboven teach the elements of claim 1.
Chen, Liu and Verboven as combined further teaches: wherein the processor circuitry is to
calculate, for each task, an adjustment factor of the task, which changes along a total loss change rate within the designated window prior to the present custom interval for the task, and
adjust the weight of the task by the adjustment factor of the task (Chen: par. 0026, “the multitask loss function is a weighted linear combination of the single task losses … where the sum runs over all T tasks. An adaptive method is disclosed herein to vary w, at one or more training steps or iterations (e.g., each training step t:w,=w,(t)).”).
While Chen, Liu and Verboven may not explicitly teach: a reciprocal of a proportion of the gradient magnitude with respect to selected shared weights within the designated window prior to the present custom interval for the task to gradient magnitudes with respect to selected shared weights within the designated window prior to the present custom interval for all tasks.,
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have further modified the Chen-Liu-Verboven disclosures and teachings by also calculate the reciprocal of a proportion of the gradient magnitude accordingly. Such a person would have been motivated to do so to mathematically capture the impact on the weights with that measure.
As to Claim 3, Chen, Liu and Verboven teach the elements of claim 2.
Liu further teaches: wherein the processor circuitry is to adjust, for each task, the weight of the task, according to equations:
PNG
media_image2.png
74
229
media_image2.png
Greyscale
where K denotes a total number of tasks, k=1, . . . , K denotes kth task, Wk(·) denotes a weight of kth task, t denotes the present custom interval, λk(·) denotes the adjustment factor for kth task, T is a scaling factor to control softness of task weighting, and a larger T results in a more even weight distribution among tasks (Liu: Sec. 4.1.3, the softmax temperature equation is used).
As to Claim 8, Chen, Liu and Verboven teach the elements of claim 1.
Chen, Liu and Verboven as combined further teaches: wherein the interface circuitry is to record, during each mini-batch training operation, a task loss and a task gradient with respect to selected shared weights for each task (Chen: par. 0038, GradNorm is implemented a loss function between actual and target gradient norms at each time step for each task), wherein a loss and a gradient magnitude with respect to selected shared weights within a custom interval for each task are an average of task losses and task gradients with respect to selected shared weights for the task recorded during the designated number of mini-batch training operations within the custom interval (Examiner asserts that an averaging of task losses and task gradients would be a way to minimize the fluctuations from each training iteration over the mini-batches).
As to Claim 9, Chen, Liu and Verboven teach the elements of claim 1.
Chen further teaches: wherein the selected shared weights are weights of a last shared layer of the deep neural network for MTL (Chen: par. 0031, “W can be the last shared layer of weights to save on compute costs.”).
As to Claim 10, Chen, Liu and Verboven teach the elements of claim 1.
Chen further teaches: wherein the pre-trained neural network comprises pre-trained models for computer vision, natural language understanding, or vision and language learning (Chen: par. 0028: “the gradient normalization methods disclosed herein can have applications in computer vision, natural language processing, speech synthesis, domain-specific applications such as traffic prediction, general cross-domain applications, curriculum learning.”).
As to Claim 12, Chen and Liu teach the elements of claim 1.
Chen further teaches: wherein a gradient magnitude with respect to the selected shared weights is expressed by a Euclidean norm of a gradient of a weighted task-specific loss with respect to the selected shared weights (Chen: par. 0026, the gradient normalization method may be a best value optimization method with respect to the shared weights of the other tasks).
As to Claim 13, it is rejected for similar reasons as claim 1.
As to Claim 14, it is rejected for similar reasons as claim 2.
As to Claim 15, it is rejected for similar reasons as claim 3.
As to Claim 26, it is rejected for similar reasons as claims 1 and 13.
B.
Claims 4-6 and 16-18 are rejected under 35 USC § 103 as being unpatentable over Chen et al. (“Chen”), United States Patent Application Publication 2019/0130275 published on May 2, 2019 in view of non-patent literature, Liu et al. (“Liu”), “End-end-end Multi-Task Learning with Attention published in 2019 and in further view of non-patent literature, Verboven et al. (“Verboven”), “HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxilary Tasks,” published in 2020 and in further view of Shayan et al. (“Shayan”), “Multi-step Forecasting via Multi-task Learning,” published in 2019.
As to Claim 4, Chen, Liu and Verboven teach the elements of claim 2.
Chen, Liu and Verboven may not explicitly teach: wherein the processor circuitry is to
calculate, for each task, a decaying coefficient corresponding to the loss change rate between each pair of the N−1 pairs of neighboring custom intervals within the designated window prior to the present custom interval for the task, wherein the adjustment factor of the task changes along the total loss change rate weighted by corresponding decaying coefficients.
Shayan teaches in general concepts related to time series forecasting for multi-task learning in a multivariate setting (Shayan: Abstract). Specifically, Shayan teaches a decaying weighting scheme with regard to time series and time windows (Sec. III.C-D, equations 6-9, tau defines the decay).
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have further modified the Chen-Liu-Verboven disclosures and teachings by using a decay coefficient as taught by Shayan. Such a person would have been motivated to do so to mathematically allow for the variable adjustment of weights to relate to recency of the time series (Shayan: Sec. I, “Moreover, the tasks of a series would have to be weighted in a decaying scheme to take into account the model’s uncertainty with regard to distant vs near future.”).
As to Claim 5, Chen, Liu, Verboven and Shayan teach the elements of claim 4.
Chen, Liu, Verboven and Shayan as combined may not explicitly teach: wherein for each task, a sum of decaying coefficients corresponding to loss change rates of the N−1 pairs of neighboring custom intervals within the designated window prior to the present custom interval equal one, and a decaying coefficient corresponding to a pair of neighboring custom intervals closer to the present custom interval is greater.
However, Examiner asserts that normalizing the sum of decaying coefficient would have been obvious to a person having ordinary skill in the art at the time of the effective filing of the invention. Such a person would have been motivated to do so with a reasonable expectation for success to allow for a design choice of better understanding the mathematical relationship of the coefficients, if normalized.
As to Claim 6, Chen, Liu, Verboven and Shayan teach the elements of claim 4.
Chen, Liu, Verboven and Shayan as combined may not explicitly teach: wherein the processor circuitry is to calculate, for each task, the decaying coefficient, according to equations:
PNG
media_image3.png
62
314
media_image3.png
Greyscale
PNG
media_image4.png
54
358
media_image4.png
Greyscale
where t denotes the present custom interval, and αn is the decaying coefficient between (t−n)th interval and (t−(n+1))th interval.
However, Examiner asserts that normalizing the sum of decaying coefficient would have been obvious to a person having ordinary skill in the art at the time of the effective filing of the invention. Such a person would have been motivated to do so with a reasonable expectation for success to allow for a linear relationship with the given mathematical calculations as a matter of design choice.
As to Claim 16, it is rejected for similar reasons as claim 4.
As to Claim 17, it is rejected for similar reasons as claim 5.
As to Claim 18, it is rejected for similar reasons as claim 6.
C.
Claim 11 is rejected under 35 USC § 103 as being unpatentable over Chen et al. (“Chen”), United States Patent Application Publication 2019/0130275 published on May 2, 2019 in view of non-patent literature, Liu et al. (“Liu”), “End-end-end Multi-Task Learning with Attention published in 2019 and in further view of non-patent literature, Verboven et al. (“Verboven”), “HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxilary Tasks,” published in 2020 and in further view of Liu et al. (“Liu ‘164”), United States Patent Application Publication 2021/0142164, published on May 13, 2021.
As to Claim 11, Chen, Liu and Verboven teach the elements of claim 1.
Chen, Liu and Verboven may not explicitly teach: wherein the deep neural network for MTL is initialized with Bidirectional Encoder Representations from Transformers (BERT).
Liu teaches in general concepts related to systems and methods for employing knowledge distillation under a multi-task learning setting (Liu ‘164: Abstract). Specifically, Liu teaches that a teacher model may be initialized as a multi-task refined with a pre-trained BERT model (Liu ‘164: par. 0057).
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have modified the Chen-Liu-Verboven disclosures and teachings by initializing the model with BERT as taught by Liu ‘164. Such a person would have been motivated to do so with a reasonable expectation of success to use a well-known l for initialization.
Conclusion
Prior art made of the record:
Tomasev et al. (“Tomasev”), United States Patent Application Publication 2020/0152333 published on May 14, 2020.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES T TSAI whose telephone number is (571)270-3916. The examiner can normally be reached M-F 8-5 Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000./JAMES T TSAI
/JAMES T TSAI/ Primary Examiner, Art Unit 2147
Prosecution Insights