DETAILED ACTION
The action is in response to the original filing on January 27, 2024. Claims 1-20 are pending and have been considered below. Claims 1, 9, and 17 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhai et al. (“Scaling Vision Transformers,” 2022, hereinafter Zhai) in view of Raffel et al. (“Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” 2023, hereinafter Raffel) and further in view of Mattivi (US 20240211370 A1, hereinafter Mattivi).
Regarding claim 1:
Regarding the limitation a method of training a language model, the method comprising: receiving a training data set for training the language model, Zhai teaches a method of training a… model (Abstract: “Attention-based neural networks such as the Vision Trans former (ViT) have recently attained state-of-the-art results… we refine the architecture and training of ViT,” Page 3, Col. 2, ¶1 “Our results suggest that with sufficient data, training a larger model for fewer steps is preferable. This observation mirrors results in language modelling and machine translation”), the method comprising: receiving a training data set for training the… model (Page 8, Col. 1, Section 5, ¶1 “This work uses the proprietary JFT-3B dataset for the scaling laws study. To make our insights more reliable and generalizable, we verify that the scaling laws also apply on the public ImageNet-21k dataset”). However, Zhai fails to teach a language model.
Raffel, in the same field of endeavor, teaches a language model (Page 11, Section 3.3.1, ¶1 “For our model, we use a standard encoder-decoder Transformer,” Page 42, ¶2 “Data sets – We introduced the “Colossal Clean Crawled Corpus” (C4)”).
Regarding the limitation determining a number of iterations for which the language model should be trained, Zhai teaches determining a number of iterations for which the… model should be trained (Page 6, Col. 2, Section 3.5, ¶2 “Figure 7 (bottom) depicts several of these options, with a cooldown after approximately 200k… steps,” Page 7, Col. 1, Fig. 7 depicts wherein a “cooldown” happens after a predetermined 200k “Steps” or a number of iterations). However, Zhai fails to teach the language model.
Raffel teaches the language model (Page 11, Section 3.3.1, ¶1, see above).
Regarding the limitation initializing parameters of the language model, Zhai teaches initializing parameters of the… model (Page 4, Col. 1, Section 2.5, ¶1 “We trained a large Vision Transformer, ViT-G/14,” Page 5, Col. 1, Section 3.2, ¶3 “To choose the best head, we perform a side-by-side comparison of a [class] token and GAP/MAP heads… we opt for the MAP head, since it is the most expressive and results in the most uniform architecture,” Page 7, Col. 1, Section 3.6, ¶¶1-2 “ViT models have many parameters that control the model’s shape… these include the patch-size, the number of encoder blocks (depth), the dimensionality of patch embeddings and self-attention (width), the number of attention heads, and the hidden dimension of MLP blocks (MLP-width)… we run an extensive simulation, where we instantiate a large amount of ViTs of various shapes, and attempt to train them for a few steps… We vary the depth, width, heads, and MLP-width but keep the patch-size at 14 px”). However, Zhai fails to teach the language model.
Raffel teaches the language model (Page 11, Section 3.3.1, ¶1, see above, Page 14, ¶1 “we train our baseline model 10 times from scratch (i.e. with different random initializations and data set shuffling)”).
Regarding the limitation repeatedly training the language model for a set of iterations, wherein a learning rate of training is a piecewise function, the training comprising: training the language model for a first set of iterations for which the learning rate is linearly increasing in value with the number of iterations, Zhai teaches repeatedly training the… model for a set of iterations (Page 6, Col. 2, Section 3.5, ¶1 “we want to train each of the models for several different durations in order to measure the tradeoff between model size and training duration,” Page 7, Col. 1, Fig. 7 as explained above), wherein a learning rate of training is a piecewise function (Page 6, Col. 2, Section 3.5, ¶2 “we address this issue by exploring learning-rate schedules that, similar to the warmup phase in the beginning, include a cooldown phase at the end of training, where the learning-rate is linearly annealed toward zero. Between the warmup and the cooldown phases, the learning-rate should not decay too quickly to zero,” Page 7, Col. 1, Fig. 7, Caption: “Various ‘infinite’ learning-rate schedules, along with the finite linear one for reference,” Fig. 7 depicts various piecewise learning rates such as “Reciprocal,” “Trapezoidal,” and “Constant”), the training comprising: training the… model for a first set of iterations for which the learning rate is linearly increasing in value with the number of iterations (Page 11, Col. 1, Section B, ¶1 “We use reciprocal square-root schedule with a linear learning rate warmup of 10k steps” and Col. 2, Listing 1, Line 46 – “warmup_steps=10_000,” Page 7, Col. 1, Fig. 7 depicts a first set of iterations or “10k steps” for which the learning rate is linearly increasing in value with the number of iterations). However, Zhai fails to teach the language model.
Raffel teaches the language model (Page 11, Section 3.3.1, ¶1, see above).
Regarding the limitation training the language model for a second set of iterations after the first set of iterations, for which the learning rate comprises: a first term that varies as inverse square root of the number of iterations, and a second term that has a constant value with respect to the number of iterations, Zhai teaches training the… model for a second set of iterations after the first set of iterations (Page 6, Col. 2, Section 3.5, ¶2 “Between the warmup and the cooldown phases, the learning-rate should not decay too quickly to zero,” Page 7, Col. 1, Fig. 7 depict a second set of iterations after the first set of iterations for the “Reciprocal √” and “Constant” learning rate schedules), for which the learning rate comprises… inverse square root of the number of iterations, or… a constant value with respect to the number of iterations (Page 6, Col. 2, Section 3.5, ¶2 “This can be achieved by using either a constant, or a reciprocal square root schedule for the main part of training,” Page 7, Col. 1, Fig. 7 – “Reciprocal √” and “Constant,” Page 11, Col. 2, “Listing 1,” Line 46 – “decay_type=’rsqrt’”). However, Zhai fails to teach the language model and the learning rate comprises: a first term that varies as inverse square root of the number of iterations, and a second term that has a constant value with respect to the number of iterations.
Raffel teaches the language model (Page 11, Section 3.3.1, ¶1, see above) and a first term that varies as inverse square root of a number of iterations and a second term that has a constant value with respect to the number of iterations (Page 12, ¶2 “During pre-training, we use an “inverse square root” learning rate schedule: 1/[sqrt(max(n,k))] where n is the current training iteration and k is the number of warm-up steps (set to 104 in all of our experiments). This sets a constant learning rate of 0.01 for the first 104 steps, then exponentially decays the learning rate until the training is over,” wherein “n” encompasses a first term that varies as inverse square root of the number of iterations and “k” encompasses a second term that has a constant value with respect to the number of iterations).
Regarding the limitation and training the language model for a third set of iterations after the second set of iterations for which the learning rate is linearly decreasing in value with the number of iterations, Zhai teaches and training the… model for a third set of iterations after the second set of iterations for which the learning rate is linearly decreasing in value with the number of iterations (Page 6, Col. 2, Section 3.5, ¶2 “a cooldown phase at the end of training, where the learning-rate is linearly annealed toward zero,” Page 7, Col. 1, Fig. 7 depicts a third set of iterations after the second set of iterations for which the learning rate is linearly decreasing in value with the number of iterations at 200k “Steps” for the “Reciprocal,” “Trapezoidal,” and “Constant” learning rate schedules). However, Zhai fails to teach the language model.
Raffel teaches the language model (Page 11, Section 3.3.1, ¶1, see above).
Regarding the limitation evaluating the trained language model, Zhai teaches evaluating the trained… model (Page 6, Col. 2, Section 3.5, ¶2 “Figure 7 shows the validation score (higher is better) for each of these options and their cooldowns, together with two linear schedules for reference. While the linear schedule is still preferable when one knows the training duration in advance and does not intend to train any longer, all three alternatives come reasonably close, with the advantage of allowing indefinite training and evaluating multiple training iterations from just one run” Page 7, Col. 1, Fig. 7 depicts “Validation score”). However, Zhai fails to teach the trained language model.
Raffel teaches the trained language model (Page 11, Section 3.1.2, ¶2 “We pre-train each model… before fine-tuning”).
Zhai and Raffel are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the language model and learning schedule terms of Raffel with the second set of iterations and training methodology of Zhai. The motivation to do so is to “achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more” (Raffel, Abstract).
Regarding the limitation determining, based on the evaluation, whether the trained language model should be deployed, Zhai teaches the evaluation (Page 6, Col. 2, Section 3.5, ¶2, Page 7, Col. 1, Fig. 7 – “Validation score”). However, the combination of Zhai and Raffel fails to teach determining, based on the evaluation, whether the trained language model should be deployed.
Mattivi, in the same field of endeavor, teaches determining, based on performance, whether the trained language model (¶115 “The machine learning natural language model may include any type of natural language processor including… neural networks, transformers, and/or the like”) should be deployed (Fig. 9 – 906-908a, 910, ¶155 “The adapted machine learning model may be evaluated based on one or more training outputs generated by the adapted machine learning model. Evaluation data indicative of the performance of the adapted machine learning model may be provided to the user… the user may determine whether to perform another iteration of an iterative model training process based on the evaluation data,” ¶157 “deploying, at step/operation 910, the adapted machine learning mode”).
Zhai, Raffel, and Mattivi are analogous art to the claimed invention as all are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the model deployment of Mattivi and the language model and learning rate schedule terms of Raffel with the evaluation of the model and training methodology of Zhai. The motivation to do so is to “achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more” (Raffel, Abstract) and to design a method in which “machine learning models may be efficiently developed and continuously refined, as performance conditions change, in a time efficient manner without sacrificing the reliability and accuracy of the resulting machine learning models” (Mattivi, ¶18).
Regarding claim 2, Zhai in view of Raffel and further in view of Mattivi teaches the method of claim 1 (and thus the rejection of claim 1 is incorporated).
Regarding the limitation further comprising: determining based on the evaluation, that the trained language model is ready to be deployed, Zhai teaches the evaluation (Page 6, Col. 2, Section 3.5, ¶2, Page 7, Col. 1, Fig. 7 – “Validation score”). However, the combination of Zhai and Raffel fails to teach further comprising: determining based on the evaluation, that the trained language model is ready to be deployed.
Mattivi teaches further comprising: determining based on performance metrics, that the trained language model is ready to be deployed (¶115 see claim 1 above, Fig. 9 – 906, 908a, 908b, ¶155 “the user may determine whether to perform another iteration of an iterative model training process based on the evaluation data,” ¶156 “receiving, at step/operation 908b, evaluation data for the previous model… the evaluation data for the previous model may be indicative of one or more performance metrics and/or training outputs for the previous model. The evaluation data generated for the adapted machine learning model may be compared to the previous evaluation data for the previous model to illustrate a relative performance of the adapted machine learning model relative to the previous model… a user may continually track improvements and/or degradations between two versions of a machine learning model,” wherein it is implicit that a user may determine that the trained language model is ready to be deployed when no more training iterations are needed).
Mattivi further teaches and deploying the trained language model (Fig. 9 – 910, ¶157 “deploying, at step/operation 910, the adapted machine learning model”).
Zhai and Mattivi are analogous art to the claimed invention as all are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the model deployment determination of Mattivi with the methodology of Zhai. The motivation to do so is to design a method in which “machine learning models may be efficiently developed and continuously refined, as performance conditions change, in a time efficient manner without sacrificing the reliability and accuracy of the resulting machine learning models” (Mattivi, ¶18).
Regarding claim 3, Zhai in view of Raffel and further in view of Mattivi teaches the method of claim 1 (and thus the rejection of claim 1 is incorporated).
Zhai teaches wherein the third set of iterations has greater than 10% and less than 40% of a total number of iterations (Pages 23-31, Tables 14-22 depict this limitation; for example, the “Steps” column may have “30k” steps while the “Cooldown” column has “10k” steps or 33% of a total number of iterations).
Regarding claim 4, Zhai in view of Raffel and further in view of Mattivi teaches the method of claim 1 (and thus the rejection of claim 1 is incorporated).
Regarding the limitation further comprising: determining based on the evaluation, that the trained language model needs further training, Zhai teaches the evaluation (Page 6, Col. 2, Section 3.5, ¶2, Page 7, Col. 1, Fig. 7 – “Validation score”). However, the combination of Zhai and Raffel fails to teach further comprising: determining based on the evaluation, that the trained language model needs further training.
Mattivi teaches further comprising: determining based on performance metrics, that the trained language model needs further training (¶115 see claim 1 above, Fig. 9 – 906, 908a, 908b, ¶155 “the user may determine whether to perform another iteration of an iterative model training process based on the evaluation data,” ¶156 “receiving, at step/operation 908b, evaluation data for the previous model… the evaluation data for the previous model may be indicative of one or more performance metrics and/or training outputs for the previous model. The evaluation data generated for the adapted machine learning model may be compared to the previous evaluation data for the previous model to illustrate a relative performance of the adapted machine learning model relative to the previous model… a user may continually track improvements and/or degradations between two versions of a machine learning model,” wherein it is implicit that a user may determine that the trained language model needs further training when the user determines that further iterations are needed or when model performance degrades).
Zhai and Mattivi are analogous art to the claimed invention as all are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the iterative model training and determination of Mattivi with the evaluation of Zhai. The motivation to do so is to design a method in which “machine learning models may be efficiently developed and continuously refined, as performance conditions change, in a time efficient manner without sacrificing the reliability and accuracy of the resulting machine learning models” (Mattivi, ¶18).
Regarding claim 5, Zhai in view of Raffel and further in view of Mattivi teaches the method of claim 4 (and thus the rejection of claim 4 is incorporated).
Regarding the limitation wherein the number of iterations is a first number of iterations, the method further comprising: determining a second number of iterations for which the language model should be further trained, Zhai teaches wherein the number of iterations is a first number of iterations, the method further comprising: determining a second number of iterations for which the… model should be further trained (Page 6, Col. 2, Section 3.5, ¶2 “Figure 7 (bottom) depicts several of these options, with a cooldown after approximately 200k, 400k… steps,” Page 7, Col. 1, Fig. 7 depicts a learning rate cooldown happening at 200k “Steps” or a first number of iterations and a predetermined 400k “Steps” or a second number of iterations for which the… model should be further trained). However, Zhai fails to teach However, Zhai fails to teach the language model.
Raffel teaches the language model (Page 11, Section 3.3.1, ¶1, see claim 1 above).
Regarding the limitation and repeatedly training the language model for a second set of iterations, wherein the learning rate of training is a piecewise function comprising: a fourth set of iterations after the third set of iterations for which the learning rate comprises: the first term that varies as inverse square root of the number of iterations, and the second term that has the constant value with respect to the number of iteration, Zhai teaches and repeatedly training the… model for a second set of iterations (Page 6, Col. 2, Section 3.5, ¶1 “we want to train each of the models for several different durations in order to measure the tradeoff between model size and training duration,” Page 7, Col. 1, Fig. 7 depicts training the model for 400k “Steps” or repeatedly training the… model for a second set of iterations), wherein the learning rate of training is a piecewise function (Page 6, Col. 2, Section 3.5, ¶2, Page 7, Col. 1, Fig. 7 and Caption as explained above with respect to claim 1) comprising: a fourth set of iterations after the third set of iterations (Page 6, Col. 2, Section 3.5, ¶2 “Between the warmup and the cooldown phases, the learning-rate should not decay too quickly to zero… The upper half of Figure 7 shows the validation score (higher is better) for each of these options and their cooldowns… with the advantage of allowing indefinite training and evaluating multiple training durations from just one run,” Page 7, Col. 1, Fig. 7 depicts, for the set of training iterations in which the “cooldown” occurs at around 400k “Steps,” how its training steps “between the warmup and cooldown phases” or a fourth set of iterations occur after the training in which the “cooldown” or the third set of iterations occurs at around 200k “Steps,” wherein “evaluating multiple training durations from just one run” implies that the longer set of training iterations where the “cooldown” happens at 400k “Steps” occurs after the shorter 200k “Steps” within a single execution of a training job) for which the learning rate comprises… inverse square root of the number of iterations, or… the constant value with respect to the number of iterations (Page 6, Col. 2, Section 3.5, ¶2 “Between the warmup and the cooldown phases, the learning-rate should not decay too quickly to zero. This can be achieved by using either a constant, or a reciprocal square root schedule for the main part of training. Figure 7 (bottom) depicts several of these options,” Page 7, Col. 1, Fig. 7 – “Reciprocal √” and “Constant”). However, Zhai fails to teach the language model and the first term that varies as inverse square root of the number of iterations, and the second term that has the constant value with respect to the number of iterations.
Raffel teaches the language model (Page 11, Section 3.3.1, ¶1, see claim 1 above) and the first term that varies as inverse square root of the number of iterations, and the second term that has the constant value with respect to the number of iterations (Page 12, ¶2 “we use an “inverse square root” learning rate schedule: 1/[sqrt(max(n,k))] where n is the current training iteration and k is the number of warm-up steps (set to 104 in all of our experiments). This sets a constant learning rate of 0.01 for the first 104 steps,” as explained above with respect to claim 1).
Zhai further teaches and a fifth set of iterations for which the learning rate is linearly decreasing in value with the number of iterations (Page 6, Col. 2, Section 3.5, ¶2 “a cooldown phase at the end of training, where the learning-rate is linearly annealed toward zero … with a cooldown after approximately…400k… steps,” Page 7, Col. 1, Fig. 7 and Pages 23-31, Tables 14-22 depict “cooldown phases” or a fifth set of iterations happening after 400k “Steps”).
Zhai and Raffel are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the language model and learning rate schedule terms of Raffel with the training methodology of Zhai. The motivation to do so is to “achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more” (Raffel, Abstract).
Regarding claim 6, Zhai in view of Raffel and further in view of Mattivi teaches the method of claim 5 (and thus the rejection of claim 5 is incorporated).
Zhai further teaches evaluating the model (Page 6, Col. 2, Section 3.5, ¶2 “Figure 7 (bottom) depicts several of these options, with a cooldown after approximately 200k, 400k… steps. The upper half of Figure 7 shows the validation score (higher is better) for each of these options and their cooldowns,” Page 7, Col. 1, Fig. 7 depicts how “Validation score” further improves for the set of training iterations in which the “cooldown” takes place at around 400k “Steps” when compared to the training with 200k “Steps”).
Regarding the limitation and determining based on the evaluation, whether the trained language model should be deployed, Zhai teaches the evaluation (Page 6, Col. 2, Section 3.5, ¶2, Page 7, Col. 1, Fig. 7 as explained above). However, Zhai fails to teach and determining based on the evaluation, whether the trained language model should be deployed.
Mattivi teaches and determining based on performance metrics, whether the trained language model should be deployed (Fig. 9 – 908b, 910, ¶155 “Evaluation data indicative of the performance of the adapted machine learning model may be provided to the user… the user may determine whether to perform another iteration of an iterative model training process based on the evaluation data,” ¶156 “receiving, at step/operation 908b, evaluation data for the previous model… the evaluation data for the previous model may be indicative of one or more performance metrics and/or training outputs for the previous model. The evaluation data generated for the adapted machine learning model may be compared to the previous evaluation data for the previous model to illustrate a relative performance of the adapted machine learning model relative to the previous model… a user may continually track improvements and/or degradations between two versions of a machine learning model,” ¶157 “deploying, at step/operation 910, the adapted machine learning model,” wherein it is implicit that model deployment is based on user determination; while Zhai never explicitly states that its second number of iterations occurs after its first set of iterations in response to determining that a model needs further training, Mattivi’s training framework demonstrates how determining whether to continue training a model was already a known practice in the art, and that one of ordinary skill in the art may combine the retraining process of Mattivi with the sets of training iterations of various durations of Zhai so that the user of Mattivi may “track improvements” regarding “one or more performance metrics,” for example, the “validation score” of Zhai).
Zhai and Mattivi are analogous art to the claimed invention as all are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the iterative model training and determination of Mattivi with the evaluation of Zhai. The motivation to do so is to design a method in which “machine learning models may be efficiently developed and continuously refined, as performance conditions change, in a time efficient manner without sacrificing the reliability and accuracy of the resulting machine learning models” (Mattivi, ¶18).
Regarding claim 7, Zhai in view of Raffel and further in view of Mattivi teaches the method of claim 1 (and thus the rejection of claim 1 is incorporated).
Regarding the limitation determining an additional number of iterations for which the language model should be further trained, Zhai teaches determining an additional number of iterations for which the… model should be further trained (Page 6, Col. 2, Section 3.5, ¶1 “we want to train each of the models for several different durations in order to measure the tradeoff between model size and training duration,” ¶2 “Figure 7 (bottom) depicts several of these options, with a cooldown after approximately 200k, 400k… steps,” Page 7, Col. 1, Fig. 7 and Pages 23-31, Tables 14-22 depict training the model for a pre-determined 400k steps before its cooldown, or an additional number of iterations). However, Zhai fails to teach the language model.
Raffel teaches a language model (Page 11, Section 3.3.1, ¶1 see claim 1 above).
Regarding the limitation training the language model for the additional number of iterations, Zhai teaches training the… model for the additional number of iterations (Page 7, Col. 1, Fig. 7 and Pages 23-31, Tables 14-22 as explained above). However, Zhai fails to teach the language model.
Raffel teaches a language model (Page 11, Section 3.3.1, ¶1 see claim 1 above).
Regarding the limitation and evaluating the language model, Zhai further teaches and evaluating the… model (Page 6, Col. 2, Section 3.5, ¶2 “The upper half of Figure 7 shows the validation score (higher is better) for each of these options and their cooldowns,” Page 7, Col. 1, Fig. 7 depicts the rising “Validation score” for the training iteration in which the “cooldown” happens at around 400k “Steps”). However, Zhai fails to teach the language model.
Raffel teaches a language model (Page 11, Section 3.3.1, ¶1 see claim 1 above).
Zhai and Raffel are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the language model of Raffel with the training methodology of Zhai. The motivation to do so is to “achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more” (Raffel, Abstract).
Regarding claim 8, Zhai in view of Raffel and further in view of Mattivi teaches the method of claim 1 (and thus the rejection of claim 1 is incorporated).
Regarding the limitation further comprising repeating one or more times: determining an additional number of iterations for which the language model should be further trained, Zhai teaches further comprising repeating one or more times: determining an additional number of iterations for which the… model should be further trained (Page 6, Col. 2, Section 3.5, ¶1 “we want to train each of the models for several different durations in order to measure the tradeoff between model size and training duration,” ¶2 “Figure 7 (bottom) depicts several of these options, with a cooldown after approximately 200k, 400k, and 500k steps,” Page 7, Col. 1, Fig. 7 and Pages 23-31, Tables 14-22 depict repeating one or more times, training the model for a pre-determined 400k, 500k, or more steps before its cooldown, or an additional number of iterations). However, Zhai fails to teach the language model.
Raffel teaches a language model (Page 11, Section 3.3.1, ¶1 see claim 1 above).
Regarding the limitation training the language model for the additional number of iterations, wherein the learning rate of training is a piecewise function comprising: a fourth set of iterations after the third set of iterations for which the learning rate comprises: the first term that varies as inverse square root of the number of iterations, and the second term that has the constant value with respect to the number of iterations, Zhai teaches training the… model for the additional number of iterations, wherein the learning rate of training is a piecewise function (Page 6, Col. 2, Section 3.5, ¶2, Page 7, Col. 1, Fig. 7 and Caption as explained above with respect to claim 1) comprising: a fourth set of iterations after the third set of iterations for which the learning rate comprises… inverse square root of the number of iterations or… the constant value with respect to the number of iterations (Page 6, Col. 2, Section 3.5, ¶2 “we address this issue by exploring learning-rate schedules that, similar to the warmup phase in the beginning, include a cooldown phase at the end of training, where the learning-rate is linearly annealed toward zero. Between the warmup and the cooldown phases, the learning-rate should not decay too quickly to zero. This can be achieved by using either a constant, or a reciprocal square root schedule for the main part of training. Figure 7 (bottom) depicts several of these options, with a cooldown after approximately 200k, 400k, and 500k steps. The upper half of Figure 7 shows the validation score (higher is better) for each of these options and their cooldowns, together with two linear schedules for reference. While the linear schedule is still preferable when one knows the training duration in advance and does not intend to train any longer, all three alternatives come reasonably close, with the advantage of allowing in definite training and evaluating multiple training durations from just one run,” wherein “evaluating multiple training durations from just one run” implies that the longer training iteration sets occur after the shorter ones within a single training job execution, Page 7, Col. 1, Fig. 7 depicts a fourth set of iterations or “the main part of training” for the training sets with 400k or 500k “Steps” occurring after the third set of iterations or the “cooldown phase” of the training set with 200k “Steps” for both the “Reciprocal √” and “Constant” options). However, Zhai fails to teach the language model and the first term that varies as inverse square root of the number of iterations, and the second term that has the constant value with respect to the number of iterations.
Raffel teaches a language model (Page 11, Section 3.3.1, ¶1 see claim 1 above) and the first term that varies as inverse square root of the number of iterations, and the second term that has the constant value with respect to the number of iterations (Page 12, ¶2, as explained above with respect to claim 1).
Zhai further teaches and a fifth set of iterations for which the learning rate is linearly decreasing in value with the number of iterations (Page 6, Col. 2, Section 3.5, ¶2 “a cooldown phase at the end of training, where the learning-rate is linearly annealed toward zero… with a cooldown after approximately 200k, 400k, and 500k steps,” Page 7, Col. 1, Fig. 7 and Pages 23-31, Tables 14-22 all as explained above with respect to claim 5).
Regarding the limitation and evaluating the language model, Zhai teaches and evaluating the… model (Page 6, Col. 2, Section 3.5, ¶2 “The upper half of Figure 7 shows the validation score (higher is better) for each of these options and their cooldowns, together with two linear schedules for reference. While the linear schedule is still preferable when one knows the training duration in advance and does not intend to train any longer, all three alternatives come reasonably close, with the advantage of allowing in definite training and evaluating multiple training durations from just one run,” Page 7, Col. 1, Fig. 7 – “Validation score”). However, Zhai fails to teach the language model.
Raffel teaches a language model (Page 11, Section 3.3.1, ¶1 see claim 1 above).
Zhai and Raffel are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the language model and learning rate schedule terms of Raffel with the training methodology of Zhai. The motivation to do so is to “achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more” (Raffel, Abstract).
Regarding claim 9:
Regarding the limitation a non-transitory computer readable storage medium comprising stored instructions that when executed by one or more computer processors cause the one or more computer processors to: receive a training data set for training a language model, Zhai teaches receive a training data set for training a… model (Page 8, Col. 1, Section 5, ¶1 see claim 1 above). However, Zhai fails to teach a non-transitory computer readable storage medium comprising stored instructions that when executed by one or more computer processors cause the one or more computer processors to: receive… and a language model.
Raffel teaches a language model (Page 11, Section 3.3.1, ¶1 see claim 1 above). However, Raffel fails to teach a non-transitory computer readable storage medium comprising stored instructions that when executed by one or more computer processors cause the one or more computer processors to receive…
Mattivi teaches a non-transitory computer readable storage medium comprising stored instructions that when executed by one or more computer processors cause the one or more computer processors to perform the instructions (Fig. 1 – 104, 106, ¶71 “The memory element 106 may include a non-transitory computer-readable storage medium for implementing one or more aspects of the present disclosure including… instructions that when executed by a computer (e.g., processing element 104), cause the computer to perform one or more steps/operations of the present disclosure”).
Claims 9-16 are computer-readable medium claims with similar limitations to the methods of claims 1-8, respectively. Therefore, claims 9-14 are rejected under substantially the same rationale as claims 1-8, respectively.
Claims 17-20 are system claims with similar limitations to the computer-readable medium of claims 9, 12-13, 15, and 16, respectively. Therefore, claims 17-18 are rejected under substantially the same rationale as claims 9, 12-13, 15, and 16, respectively.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM M LEE whose telephone number is (571)272-4761. The examiner can normally be reached Mon-Fri. 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571)272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM M LEE/
Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
