Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are pending. Claims 1, 8, and 15 are independent.
This Application was published as U.S. 20260134215.
Apparent priority: 13 November 2024
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tay (U.S. 20240256964).
Regarding Claim 1, Tay teaches:
1. A computer-implemented method comprising:
generating from a plurality of tokens interpretable by a large language model: [Tay, Figure 3A shows tokens and the training examples and Figure 3B shows examples of corrupted training examples. Figures 4A and 4B shows the various configurations of corrupted examples (sets of tokens/words) using actual text instead of circles in place of tokens. These are examples that are generated for retraining of the model according to different objectives. Such that “corrupted spans” are determined according to several different criteria/ “objective configurations 206, 208, 210” shown in Figure 2: “[0119] The objective framework 204 can provide for selection from the plurality of objective configurations based on one or more parameter values. One parameter value can include a span length parameter….”]
a set of context tokens comprising tokens with bidirectional attention; and [Tay, Figure 4B, e.g., the first and second objective configurations include context tokens that are bidirectional: “[0061] … For example, presenting the same examples in a differently transformed or corrupted manner can expose the model to new challenges that cause it to learn different aspects of the information contained within the examples….” “[0117] … For example, bounding a model by bidirectional context (or the future) (e.g., span corruption) can make the task easier and can become more akin to fact completion. …” See also [0137] below. “Bidirectional context” is when text on both sides of the corrupted token is used by the model to predict the missing token. Model looks to the past (before) and the future (after) and is therefore bidirectional but not causal. Causality requires dependence only on the past.]
a set of span tokens comprising tokens with causal attention and bidirectional attention; [Tay, Figure 4B, e.g., third objective configuration includes span tokens that work with causal attention because the span 50 depends only on the preceding text; the other two objective configurations, first and second, have bidirectional attention that can look to the past and the future both. “[0137] In some embodiments, configuration parameters of the objective framework can be selected to interpolate between, for example, language modeling objectives (e.g., to unidirectionally predict subsequent word(s) based on preceding word(s)) and in-place reconstruction (e.g., fill in gaps bidirectionally based on surrounding context). …” Predicting unidirectionally based on preceding words is “Causal attention” and bidirectional use of text on both sides of a token is “bidirectional attention.” “[0019] … For sequence-based objectives, the span length parameter can be augmented by constraining the span to the end of the input sequence, such that no uncorrupted tokens appear after the corrupted span.” This is a causal training set: nothing after.]
modifying parameters of the large language model at a first training stage by utilizing a first loss function that incorporates the set of context tokens and a second loss function that incorporates the set of span tokens; and [Tay, Figure 7, a pretrained model is obtained at 702 and its parameters are updated at 712 which is the end of a “further pretraining 704.” Figure 2, the “Objective Configurations 206, 208, 210” each represent a different loss function and each can use one of the corrupted training examples of Figures 3A/3B or 4A/4B each of which corresponds to a different objection configuration, as shown in Figure 4B, and causes re-training of a model for a different task. Figure 2, “evaluator 220” that causes the retraining operates using a “loss function” and the type of “loss function” determines the retraining of step 200b also shown in Figure 8: “[0193] At 806, example method 800 can include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various determinations of loss can be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, contrastive loss, or various other loss functions. he evaluation signal can be computed using known ground-truth labels (e.g., supervised learning), predicted or estimated labels (e.g., semi- or self-supervised learning), or without labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). The reward can be computed using a machine-learned reward model configured to generate rewards based on output(s) received. The reward can be computed using feedback data describing human feedback on the output(s).”]
modifying the parameters of the large language model at a second training stage by utilizing the first loss function, the second loss function, and a third loss function that incorporates the set of context tokens. [Tay, Figure 7, a pretrained model is obtained at 702 and its parameters are updated at 712 which is the end of a “further pretraining 704.” There are several of these pretraining stages that can teach the first, second, third, etc “training stages” of this Claim. See Figure 8 where the “trained instance” is updated at 808 based on the evaluation results at 806. See also Figure 2 showing the “evaluator 220” as part of the “further pretraining 200b.” “[0059] Based on these outputs, a training computing system can update the parameters of the model. Evaluation of the parameters can be based on various metrics, such as the accuracy of the reconstructed text or the fluency and coherence of the generated content.” Each of the various metrics would correspond to a different loss function for a different mode of training: “[0060] In further example implementations, this process of further pretraining can cause the model to learn to recognize and respond to mode-switching tokens. These tokens can act as signals that cause internal reasoning pathways within the model to switch between different modes of operation (e.g., performing a task-specific function). For example, a mode-switching token might prompt the model to shift from generating narrative prose to solving a mathematical equation embedded within a story.” “[0123] Multiple different objective configurations can be used. For instance, a first objective configuration can be used for training example. A second objective configuration can be used for a second training example. A third objective configuration can be used for a third training example. Alternatively, multiple different objective configurations can be used for each training example.”]
Regarding Claim 2, Tay teaches:
2. The computer-implemented method of claim 1, wherein generating the set of span tokens comprises
assigning, utilizing a causal-bidirectional hybrid attention mask, a contiguous span of tokens of the plurality of tokens interpretable by the large language model to have causal attention with one another. [Tay: “[0062] For instance, a text passage used during initial pretraining in a left-to-right, causal learning approach could be reintroduced with certain words or phrases randomly masked in different configurations. …” Examples of corrupting/masking the tokens are shown in Figure 3A/3B and 4A/4B.]
Regarding Claim 3, Tay teaches:
3. The computer-implemented method of claim 1, wherein generating the set of span tokens comprises
assigning, utilizing a causal-bidirectional hybrid attention mask, a contiguous span of tokens of the plurality of tokens interpretable by the large language model to have bidirectional attention with the set of context tokens. [Tay: “[0069] … Another approach includes pretraining with a masked language objective which identifies masked text based on surrounding text (e.g., bidirectionally). …” “[0140] … For instance, for the first objective configuration, an example output can be “<B> ability <a> emotion or <b> copied. <c> Noughts & <d> Ellis, <E>.” In this manner, for instance, example implementations can effectively provide a fill-in-the-blank solution to masked-out subportions of the input sequence.” Figures 4A and 4B show examples of contiguous spans that are masked and must be predicted by the language model during its retraining.]
Regarding Claim 4, Tay teaches:
4. The computer-implemented method of claim 1, wherein generating the set of context tokens comprises
assigning, utilizing a causal-bidirectional hybrid attention mask, non-contiguous tokens of the plurality of tokens interpretable by the large language model to have bidirectional attention with one another. [Tay, Figures 4A and 4B show examples of non-contiguous spans that are masked and must be predicted by the language model during its retraining.]
Regarding Claim 5, Tay teaches:
5. The computer-implemented method of claim 1, wherein the second loss function enables the large language model to perform missing span generation by modifying the parameters of the large language model using the set of span tokens, [Tay: “missing span” is taught by “infilling”: “[0053] In addition to improved flexibility and overall performance, example techniques according to the present disclosure can imbue pretrained models with emergent abilities that were previously only associated with much larger models. By integrating a mixture of prefix language modeling and diverse span corruption denoising tasks, example methods can allow the model to leverage bidirectional attention and infilling capabilities, leading to substantial improvements in downstream tasks-even without any “new” knowledge in the training data. …” “[0057] … raining on these severely corrupted examples can challenge the model to infer and reconstruct information from very limited context, thereby improving its reasoning and infilling capabilities.”]
the large language model comprising a decoder-only large language model. [Tay: “[0027] In some implementations of the one or more example non-transitory, computer-readable media, the example pretrained machine-learned model includes a decoder-only model….”]
Regarding Claim 6, Tay teaches:
6. The computer-implemented method of claim 5, wherein the first loss function enables the large language model to perform masked next token prediction by modifying the parameters of the large language model using the set of context tokens. [Tay, Figures 2 and 8. Depending on the “objective configuration 206, 208, 214” that is selected, the “corrupted training data 214” is masked/corrupted in a different way, as shown in Figures 3A/3B and 4A/4B and each type of “corrupted training data 214” corresponds to its own “loss function.” The “masked next token prediction” is one type of training: “[0273] … As another example, machine-learned model(s) 901 can process the natural language data to generate a prediction output (e.g., one or more predicted next portions of natural language content).” “[0219] Output sequence 1007 can be generated autoregressively. For instance, for some applications, an output of one or more prediction layer(s) 1006 can be passed through one or more output layers (e.g., softmax layer) to obtain a probability distribution over an output vocabulary (e.g., a textual or symbolic vocabulary) conditioned on a set of input elements in a context window. In this manner, for instance, output sequence 1007 can be autoregressively generated by sampling a likely next output element, adding that element to the context window, and re-generating the probability distribution based on the updated context window, and sampling a likely next output element, and so forth.”]
Regarding Claim 7, Tay teaches:
7. The computer-implemented method of claim 1, wherein the third loss function enables the large language model to perform self-supervised contrastive learning by modifying the parameters of the large language model using the set of context tokens, [Tay, Figures 2 and 8: [0193] At 806, example method 800 can include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various determinations of loss can be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, contrastive loss, or various other loss functions. he evaluation signal can be computed using known ground-truth labels (e.g., supervised learning), predicted or estimated labels (e.g., semi- or self-supervised learning), or without labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). …”]
the large language model comprising a decoder-only large language model. [Tay: “[0027] In some implementations of the one or more example non-transitory, computer-readable media, the example pretrained machine-learned model includes a decoder-only model….”]
Claims 8-20 have different scopes but are mix and matches of elements of the Claims of the first set of 1-7 and therefore the more detailed mapping of Claims 1-8 should be consulted for Claims 8-20.
Regarding Claim 8, Tay teaches:
8. A system comprising: [Tay: Figure 1A, “training computing system 150.”]
one or more memory devices; and [Tay: Figure 1A, “memory 154.”]
one or more processors configured to cause the system to: [Tay: Figure 1A, “processors 152.”]
generate, from a plurality of tokens interpretable by a large language model, a set of context tokens capturing bidirectional attention and a set of span tokens capturing causal attention; and [Tay: Figure 2, “Corrupted Training Data 214.” The training data are corrupted according to different masking/removing schemes some of which are provided bidirectional spans on both sides of a missing portion (infilling) and require bidirectional attention and other ones of which occur at the end of a span and rely only on previously occurred text (causal attention) to predict/generate the masked/missing portion. “[0053] … By integrating a mixture of prefix language modeling and diverse span corruption denoising tasks, example methods can allow the model to leverage bidirectional attention and infilling capabilities, leading to substantial improvements in downstream tasks-even without any “new” knowledge in the training data….” “[0182] I… For example, the first set of one or more training examples can correspond to a PrefixLM-type objective in which bidirectional attention is applied over an initial prefix sequence to autoregressively generate, with causal attention, the corrupted portion.”]
modify parameters of the large language model according to: [Tay: Figure 2, “Further pretraining 200b” also shown in Figure 7 as “updating one or more parameters of the pretrained machine-learned model based on an evaluation of the plurality of inputs 712.” Figure 8 showing the process of retraining based on a training instance.]
a first loss function that incorporates the set of context tokens and that enables masked next token prediction by the large language model; [Tay: Figure 2, “Objective configuration 206.” Figures 3A/3B and 4A/4B show the masking/corrupting of data so the masked portion can be predicted during the training of the model using a loss function. “[0219] … In this manner, for instance, output sequence 1007 can be autoregressively generated by sampling a likely next output element, adding that element to the context window, and re-generating the probability distribution based on the updated context window, and sampling a likely next output element, and so forth.” “[0273] …As another example, machine-learned model(s) 901 can process the natural language data to generate a prediction output (e.g., one or more predicted next portions of natural language content).”]
a second loss function that incorporates the set of span tokens and that enables missing span generation by the large language model; and [Tay: Figure 2, “Objective configuration 208.” “[0053] … By integrating a mixture of prefix language modeling and diverse span corruption denoising tasks, example methods can allow the model to leverage bidirectional attention and infilling capabilities, leading to substantial improvements in downstream tasks-even without any “new” knowledge in the training data. …” “[0057] … Training on these severely corrupted examples can challenge the model to infer and reconstruct information from very limited context, thereby improving its reasoning and infilling capabilities.” See [0161]-[0183].]
a third loss function that incorporates the set of context tokens and that enables self-supervised contrastive learning by the large language model. [Tay: Figure 2, “Objective configuration 210.” “[0193] At 806, example method 800 can include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various determinations of loss can be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, contrastive loss, or various other loss functions. The evaluation signal can be computed using known ground-truth labels (e.g., supervised learning), predicted or estimated labels (e.g., semi- or self-supervised learning), or without labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). The reward can be computed using a machine-learned reward model configured to generate rewards based on output(s) received. The reward can be computed using feedback data describing human feedback on the output(s).”]
Regarding Claim 9, Tay teaches:
9. The system of claim 8, wherein the one or more processors are further configured to cause the system to:
generate the set of span tokens capturing causal attention by assigning, utilizing a causal-bidirectional hybrid attention mask, a contiguous span of tokens of the plurality of tokens interpretable by the large language model to have causal attention with one another; and [Tay: for the hybrid bidirectional and causal mask see [0182]. For contiguous span of tokens see examples of masked tokens in Figures 4A/4B.]
generate the set of context tokens capturing bidirectional attention by assigning, utilizing the causal-bidirectional hybrid attention mask, additional tokens of the plurality of tokens flanking the set of span tokens and attending to one another. [Tay: for tokens flanking a span see the examples of Figures 4A/4B and infilling. “[0161] … With U-PaLM it is observed that, by extending pretraining with a small amount of UL2 denoising steps, the model is also able to pick up infilling abilities—where the model is given a location in the middle of a prompt to fill in….”]
Regarding Claim 10, Tay teaches:
10. The system of claim 9, wherein the one or more processors are further configured to cause the system to generate the set of span tokens capturing bidirectional attention by assigning, utilizing the causal-bidirectional hybrid attention mask, one or more tokens of the set of span tokens to have bidirectional attention to the set of context tokens. [Tay: masking/corruption is taught in Figures 3A/3B and 4A/4B. see [0182] for the hybrid model. “[0117] In some embodiments, the plurality of configurations can effectively interpolate between long-range generative language modeling objectives and local prefix-based modeling objectives. Advantageously, each of the plurality of object configurations can test the performance of the model 12216 in different ways. For example, bounding a model by bidirectional context (or the future) (e.g., span corruption) can make the task easier and can become more akin to fact completion. Meanwhile, language modeling objectives can be more open ended. This behavior can be observed, for example, by monitoring cross entropy losses of different objective configurations.”]
Regarding Claim 11, Tay teaches:
11. The system of claim 8, wherein modifying the parameters of the large language model according to the first loss function comprises modifying the parameters of the large language model at a first training stage that involves modifying the parameters of the large language model over a number of iterations before a second training stage. [Tay: Figure 2 and 7-8 show the stages of training using different Objective Configurations and the iteration of the model based on the results of the Evaluator 220 before going to a different training stage for a different mode/task. “[0065] A technical effect of example implementations of the present disclosure is more energy-efficient training operations or model updates. In some scenarios, increased energy efficiency can provide for less energy to be used to perform a given number of update iterations (e.g., less energy expended to maintain the model in memory, less energy expended to perform calculations within the model, such as computing gradients, backpropagating a loss, etc.)….” “[0194] … Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations. …”]
Regarding Claim 12, Tay teaches:
12. The system of claim 8, wherein modifying the parameters of the large language model according to the third loss function comprises modifying the parameters of the large language model at a first training stage that involves modifying the parameters of the large language model over a number of iterations before a second training stage. Tay: Figure 2 and 7-8 show the stages of training using different Objective Configurations and the iteration of the model based on the results of the Evaluator 220 before going to a different training stage for a different mode/task. This Claim is like Claims 11 and 13 only that the “loss function” / “Objective Configuration” changes in these Claims. The process is the same for any “loss function” / “Objective Configuration” and the iterations of the previous stage need to be concluded before a next stage of training begins.]
Regarding Claim 13, Tay teaches:
13. The system of claim 8, wherein modifying the parameters of the large language model according to the second loss function comprises modifying the parameters of the large language model at a second training stage that involves modifying the parameters of the large language model over a number of iterations after a first training stage. [Tay: Figure 2 and 7-8 show the stages of training using different Objective Configurations and the iteration of the model based on the results of the Evaluator 220 before going to a different training stage for a different mode/task. This Claim is like Claims 11 and 12 only that the “loss function” / “Objective Configuration” changes in these Claims. The process is the same for any “loss function” / “Objective Configuration” and the iterations of the previous stage need to be concluded before a next stage of training begins.]
Regarding Claim 14, Tay teaches:
14. The system of claim 13, wherein modifying the parameters of the large language model comprises:
modifying parameters at a first training stage that incorporates the first loss function and the second loss function and omits the third loss function; and [Tay: Figure 2 and 7-8 show the stages of training using different Objective Configurations and the iteration of the model based on the results of the Evaluator 220 before going to a different training stage for a different mode/task. Depending on which modes/tasks the model is being trained for, the Objective Configurations / Loss Functions are selected and used for the training in order to cause modified parameters as shown in Figure 7, 712.]
modifying parameters at a second training stage that incorporates the first loss function, the second loss function, and the third loss function. [Tay: Figure 7, 712. “[0060] In further example implementations, this process of further pretraining can cause the model to learn to recognize and respond to mode-switching tokens. These tokens can act as signals that cause internal reasoning pathways within the model to switch between different modes of operation (e.g., performing a task-specific function). For example, a mode-switching token might prompt the model to shift from generating narrative prose to solving a mathematical equation embedded within a story.”]
Regarding Claim 15, Tay teaches:
15. A non-transitory computer readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising: [Tay: “[0026] Example aspects of the present disclosure provide one or more example non-transitory, computer-readable media storing an example pretrained machine-learned model….”]
receiving a prompt to a decoder-only large language model, the prompt comprising at least one of an encoding request or a text infilling request; [Tay: “[0094] … As an example, the machine-learned model(s) can process the natural language data to generate a language encoding output….” “[0097] In some implementations, the input to the machine-learned model(s) of the present disclosure can be latent encoding data (e.g., a latent space representation of an input, etc.). The machine-learned model(s) can process the latent encoding data to generate an output. As an example, the machine-learned model(s) can process the latent encoding data to generate a recognition output. As another example, the machine-learned model(s) can process the latent encoding data to generate a reconstruction output. As another example, the machine-learned model(s) can process the latent encoding data to generate a search output. As another example, the machine-learned model(s) can process the latent encoding data to generate a reclustering output. As another example, the machine-learned model(s) can process the latent encoding data to generate a prediction output.” “[0161] Qualitative Analysis: New Prompting Capabilities: Left-to-right casual language model pretraining has typically allowed models to provide meaningful continuations of prompts. With U-PaLM it is observed that, by extending pretraining with a small amount of UL2 denoising steps, the model is also able to pick up infilling abilities—where the model is given a location in the middle of a prompt to fill in. Notably, with U-PaLM it can be possible to query both the infill style and the traditional style via the usage of extra ID tokens (as it is used in denoising) or without, respectively.”
extracting, from the prompt, a plurality of tokens by using the decoder-only large language model to process the prompt according to parameters modified based on a loss function that incorporates causality and bidirectionality; and [Tay: “[0027] In some implementations of the one or more example non-transitory, computer-readable media, the example pretrained machine-learned model includes a decoder-only model….” Figure 2, “corrupted training data 214” are modified based on Objective Configurations / loss functions determined by the “objective framework 204” and include causality and bidirectionality and depending on the Objective Configuration / loss function the model parameters are modified during this retraining as shown in Figure 7 at 712: “updating one or more parameters of the pretrained machine-learned model ….” “[0016] In some implementations of the example method, the initial pretraining was based on a causal language modeling objective.” “[0053] … By integrating a mixture of prefix language modeling and diverse span corruption denoising tasks, example methods can allow the model to leverage bidirectional attention and infilling capabilities, leading to substantial improvements in downstream tasks-even without any “new” knowledge in the training data. …”]
generating, using the decoder-only large language model with the parameters modified based on the loss function, at least one of a token embedding from the plurality of tokens based on the encoding request or an infill text based on the text infilling request. [Tay: the updated model is used to generate text that is masked according to patterns of Figures 4A or 4B which include both infilling and prediction. “[0094] … As another example, the machine-learned model(s) can process the text or natural language data to generate a latent text embedding output. …” See Figure 11 and [0222]-[0226] for generation of embeddings.]
Regarding Claim 16, Tay teaches:
16. The non-transitory computer readable medium of claim 15, wherein the operations further comprise processing the plurality of tokens using the decoder-only large language model according to parameters modified based on the loss function incorporating causality and bidirectionality captured by a causal-bidirectional hybrid attention mask. [Tay: “17. The one or more non-transitory, computer-readable media of claim 16, wherein the pretrained machine-learned model comprises a decoder-only model.” “[0182] In some implementations of example method 700, the first set of one or more training examples are characterized by corrupted spans following an initial prefix at a start of an input sequence of a corresponding training example. For example, the first set of one or more training examples can correspond to a PrefixLM-type objective in which bidirectional attention is applied over an initial prefix sequence to autoregressively generate, with causal attention, the corrupted portion.” Tay says that previous methods included either bidirectionality or unidirectional causality and the approach of its embodiment combines/hybrids both such that both of bidirectionality and causality can exist together: “[0069] Furthermore, pretraining a model according to example embodiments of the present disclosure can help provide a “universal” model effective to perform a variety of different downstream tasks with respect to sequenced data (e.g., the same or different sequenced data), optionally with or without subsequent fine-tuning. Traditional techniques, in contrast, point to model selection based on the downstream tasks. The plethora of distinct model arrangements, architectures, training recipes, training datasets, etc. can be overwhelming, leading to uninformed choices or otherwise suboptimal model implementations. Furthermore, even if a model may be appropriately selected for a given task, that model may need to be reconfigured or even replaced if the tasks or other requirements change. For example, traditional approaches to processing sequenced data have often relied on different categories of pretraining approaches. For instance, in the context of natural language processing, one prior approach includes pretraining with a language-modeling objective which unidirectionally generates sequences of text based on preceding textual content. Another approach includes pretraining with a masked language objective which identifies masked text based on surrounding text (e.g., bidirectionally). But these pretraining objectives have generally proved inadequate for diverse implementations: for example, open-text generation and prompt-based learning can be an unfavorable setting for traditional masked language objectives, whereas traditional language modeling approaches can be unduly inhibited by purely unidirectional causality.” “[0117] In some embodiments, the plurality of configurations can effectively interpolate between long-range generative language modeling objectives and local prefix-based modeling objectives. Advantageously, each of the plurality of object configurations can test the performance of the model 12216 in different ways. For example, bounding a model by bidirectional context (or the future) (e.g., span corruption) can make the task easier and can become more akin to fact completion. Meanwhile, language modeling objectives can be more open ended. This behavior can be observed, for example, by monitoring cross entropy losses of different objective configurations.” “[0137] In some embodiments, configuration parameters of the objective framework can be selected to interpolate between, for example, language modeling objectives (e.g., to unidirectionally predict subsequent word(s) based on preceding word(s)) and in-place reconstruction (e.g., fill in gaps bidirectionally based on surrounding context). For instance, as the corrupted subportion length increases, the objective can, in some embodiments, approximate a language modeling objective locally within the corrupted subportion. Accordingly, a diverse mixture of pretraining objectives can be generated by implementing a plurality of configurations of a pretraining objective framework according to example aspects of the present disclosure.”]
Regarding Claim 17, Tay teaches:
17. The non-transitory computer readable medium of claim 16, wherein the operations further comprise processing the plurality of tokens using the decoder-only large language model according to parameters modified based on the loss function incorporating causality via span tokens captured by the causal-bidirectional hybrid attention mask and bidirectionality via context tokens captured by the causal-bidirectional hybrid attention mask. [Tay: “17. The one or more non-transitory, computer-readable media of claim 16, wherein the pretrained machine-learned model comprises a decoder-only model.” “[0182] In some implementations of example method 700, the first set of one or more training examples are characterized by corrupted spans following an initial prefix at a start of an input sequence of a corresponding training example. For example, the first set of one or more training examples can correspond to a PrefixLM-type objective in which bidirectional attention is applied over an initial prefix sequence to autoregressively generate, with causal attention, the corrupted portion.”]
Regarding Claim 18, Tay teaches:
18. The non-transitory computer readable medium of claim 15, wherein generating the token embedding comprises using the decoder-only large language model with parameters modified based on a loss sub-function of the loss function that incorporates a set of context tokens and that enables self-supervised contrastive learning. [Tay: “17. The one or more non-transitory, computer-readable media of claim 16, wherein the pretrained machine-learned model comprises a decoder-only model.” Figure 2, “Objective configuration 210.” “[0193] At 806, example method 800 can include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various determinations of loss can be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, contrastive loss, or various other loss functions. The evaluation signal can be computed using known ground-truth labels (e.g., supervised learning), predicted or estimated labels (e.g., semi- or self-supervised learning), or without labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). The reward can be computed using a machine-learned reward model configured to generate rewards based on output(s) received. The reward can be computed using feedback data describing human feedback on the output(s).”]
Regarding Claim 19, Tay teaches:
19. The non-transitory computer readable medium of claim 15, wherein generating the infill text comprises using the decoder-only large language model with parameters modified based on a loss sub-function of the loss function that incorporates a set of span tokens and that enables missing span generation. [Tay: “17. The one or more non-transitory, computer-readable media of claim 16, wherein the pretrained machine-learned model comprises a decoder-only model.” Missing span generation is infilling. Figures 4A and 4B show the scenarios. “[0053] In addition to improved flexibility and overall performance, example techniques according to the present disclosure can imbue pretrained models with emergent abilities that were previously only associated with much larger models. By integrating a mixture of prefix language modeling and diverse span corruption denoising tasks, example methods can allow the model to leverage bidirectional attention and infilling capabilities, leading to substantial improvements in downstream tasks-even without any “new” knowledge in the training data….”]
Regarding Claim 20, Tay teaches:
20. The non-transitory computer readable medium of claim 15, wherein the operations further comprise generating, using the decoder-only large language model and in response to a text generation request, predicted text based on the loss function comprising three loss sub-functions that enable causal attention and bidirectional attention. [Tay: “17. The one or more non-transitory, computer-readable media of claim 16, wherein the pretrained machine-learned model comprises a decoder-only model.” Three different “Objective Configurations 206, 208, 210” are shown in Figure 2 each of which can have a different loos function and Tay is based on mixture of modes/objectives for language models which means different loss functions for different types of training. “[0182] … For example, the first set of one or more training examples can correspond to a PrefixLM-type objective in which bidirectional attention is applied over an initial prefix sequence to autoregressively generate, with causal attention, the corrupted portion.” “[0053] …The model can become more adept at handling a wider range of prompts and can develop new capabilities, such as mode-switching, which enables it to trigger specific downstream behaviors in response to certain tokens. This flexibility can allow an example model to adapt to various tasks without the need for further fine-tuning (e.g., on more expensive, labeled data), making it more versatile and practical for real-world applications. Additionally, or alternatively, such a robust pretraining technique can allow a more efficient fine-tuning stage (e.g., less labeled data, fewer iterations, etc.) to achieve satisfactory results.” “[0117] In some embodiments, the plurality of configurations can effectively interpolate between long-range generative language modeling objectives and local prefix-based modeling objectives. Advantageously, each of the plurality of object configurations can test the performance of the model 12216 in different ways. For example, bounding a model by bidirectional context (or the future) (e.g., span corruption) can make the task easier and can become more akin to fact completion. Meanwhile, language modeling objectives can be more open ended. This behavior can be observed, for example, by monitoring cross entropy losses of different objective configurations.” See [0137] also.]
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wei (U.S. 20230244938): also uses a set of corrupted training examples to retrain pretrained language models.
Sun (U.S. 20240045994): generates training data by perturbing the ground truth for different modalities and purposes and uses bidirectional language model encoders and autoregressive language model decoders.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FARIBA SIRJANI whose telephone number is (571)270-1499. The examiner can normally be reached 9 to 5, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Fariba Sirjani/
Primary Examiner, Art Unit 2659
Prosecution Insights