Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-8, 10-15, 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable by Poesia, G., Polozov, O., Le, V., Tiwari, A., Soares, G., Meek, C., & Gulwani, S. (2022). Synchromesh: Reliable code generation from pre-trained language models. ArXiv.org. https://arxiv.org/abs/2201.11227 (Year: 2022), hereinafter “Poesia” and further in light of Yager, N. (2018, May 4). Neural text generation: How to generate text using conditional language models. Medium: Pharasee. Neural text generation: How to generate text using conditional language models, hereinafter “Yager”.
Claim 1:
Poesia teaches a method for controllable text generation by a neural network model (i.e. pg. 1, SYNCHROMESH feeds the examples to a pre-trained language model and samples programs using Constrained Semantic Decoding(CSD): a general framework for constraining the output to a set of valid programs in the target language) the method comprising: receiving, via a communication interface, an input request for generating a natural language output (i.e. pg. 4, Fig. 3, “Figure 3 shows a case where the model cannot simply adapt one example from the prompt. Here, the user’s query is “Which city has the highest number of departing flights?” This query is similar to the previous one”, wherein the BRI for an input encompasses a user entered prompt); encoding, by an encoder of the neural network model, the input request into a vector representation (i.e. pg. pg. 3, Fig. 2, “Example of Target Similarity Tuning improving example selection for synthesizing a SQL query… With TST, we retrieve a relevant example which GPT-3 successfully adapts to answer the user’s question (c).”, wherein it is noted that in pg. 3, Fig. 2 shows an example of the user input “Which city has the highest number of airports?”, being encoded into a vector representation for the GPT-3 model);
generating, by a decoder of the neural network model, a conditional (i.e. pg. 1, “CSD leverages constraints on partial outputs to sample complete correct programs”, wherein the BRI for previously decoded output tokens encompasses the partial outputs of valid previous tokens) based on the vector representation (i.e. pg. 2, 6, “Using the decision procedure for Lc, we can now describe the Constrained Semantic Decoding algorithm. Suppose s ∈ Lc is the language model’s output so far (we start with ). If ΣM is the model’s vocabulary, we can compute the set of valid next tokens VM(s) = {t ∈ ΣM : st ∈ Lc} by using our decision procedure for each token in the vocabulary ΣM. In other words, we maintain the invariant that the model’s current partial output s is in Lc, and make progress by using the model to sample from VM(s), instead of the unconstrained ΣM. Once we have a complete program, we are guaranteed that it will satisfy all constraints enforced by the CE.”, wherein the BRI for a conditional probability distribution encompasses how the Constrained Semantic Decoding (CSD) algorithm takes enforces a rich syntactic and semantic constraints that aligns these constraints with the language model’s token vocabulary so that it guarantees that the model’s token distribution satisfies the rules of the target language. These constraints eliminate the probability of invalid programs and improve prediction accuracy of the model as the distribution of the next token depends on the language model’s output so far. Thus the model is further constrained by the previously decoded tokens); adjusting the conditional (i.e. pg. 12, “The CSD procedure uses the model M on the partial program to generate a distribution on the next token, but constrains it to belong to the set of valid tokens determined by the completion engine CL.”, wherein the previously decoded tokens are partial valid tokens and the predictive probability distribution for the model adjusted to be constrained based on the partial and previously decoded valid tokens) and a user-provided language description of a constraint (i.e. pg. 15, “If we find a violation, we (a) use CSD to determine all valid next tokens, (b) make a request asking from just a single token, applying a logit bias to constrain it to the valid tokens, and then (c) continue generation after appending the new token”, wherein the BRI for a constraint term encompasses how valid next tokens are based on previously decoded valid tokens, which are based on natural language descriptions of a user made in prompts, such as text from a user question) to logits of the conditional probability distribution (i.e. pg. 7, Our only access to the models was through the public OpenAI HTTP API, which allowed us to apply constraints by adding a bias to logits); generating, by the decoder, the next output token for the natural language output based on the adjusted conditional (i.e. pg. 8, Fig. 4, “(a) Accuracy and (b) validity of Codex predictions with and without SYNCHROMESH on SMCalFlow as a function of the ground-truth program length… Error bands correspond to standard error. (c) Evaluation of the “generate-then-test” approach with Codex, showing the probability of at least one prediction being a valid program (Valid@K) for up to 5 samples.”, wherein it is noted that with Synchromesh the prediction accuracy and validity improves due to the constraints) of the next output token (i.e. pg. 12, The CSD procedure uses the model M on the partial program to generate a distribution on the next token, but constrains it to belong to the set of valid tokens determined by the completion engine CL).
While Poesia teaches the concept of having output tokens decoded iteratively according the previously generated output tokens and adjusting the distribution of possible valid tokens by constraining the distribution to only tokens that are valid as defined by the previously decoded tokens, Poesia may not explicitly teach
a conditional probability distribution.
However, Yager teaches
generating, by a decoder of the neural network model, a conditional (i.e. pg. 2-3, “. A language model is at the core of many NLP tasks, and is simply a probability distribution over a sequence of words:
PNG
media_image1.png
45
163
media_image1.png
Greyscale
It can also be used to estimate the conditional probability of the next word in a sequence:
PNG
media_image2.png
46
231
media_image2.png
Greyscale
Let’s assume we have the sequence [my, cat's, breath, smells, like, cat, ____] and we want to guess the final word….
Sampling: Sample from the conditional word probability distribution.
Words that are a better fit are more likely to be selected. For the example
above, we would select the word “food” with probability 62%, “toys” with
probability 14%, “aardvark” with probability 0.001%, etc.”, ; adjusting the conditional (i.e. pg. 3, “A beam search keeps track of several probable variants at each step to avoid being led astray by local maxima. Select “food” and “toys”, and reassess what is better when more words have been added) .
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add a conditional probability distribution., to Poesia’s system that uses a logit bias in an autoencoding LLM to constrain the generation of tokens by previous tokens, with how a conditional probability distribution of every word in the vocabulary by be adjusted by generated constraint terms that correspond to previously generated output tokens, as taught by Yager. One would have been motivated to combine Yager with Poesia, and would have had a reasonable expectation of success in doing so as using techniques may improve the performance of the LLM by using a method called “teacher forcing” which provides an advantage to the overall accuracy of a model trying to predict the word that comes next.
Claim 3:
Poesia and Yager teach the method of claim 1.
Poesia further teaches wherein the user-provided language description of the constraint comprise a sequence of tokens in the input request (i.e. pg. 4, Fig. 3, “A CE CL is a partial function from Σ∗ L to a set of tokens. We use a regular expression over Σ to represent a set of tokens”, wherein it is noted that a prompt may be a series of tokens).
Claim 4:
Poesia and Yager teach the method of claim 1.
Poesia further teaches wherein the user-provided language description of the constraint comprises one or more of: an indication of one or more desired keywords to be used in the natural language output (i.e. pg. 3, Suppose a data analyst has a relational database of airports and wants to answer the following question: “Which city has the highest number of airports?”); and an indication of one or more undesired keywords not to be included in the natural language output (i.e. pg. 14, “These tend to happen when the model blindly copies portions of the examples in the prompt without adaptation. For instance, whenever the utterance contains exactly one of “a.m.” or “p.m”, this is usually represented in SMCalFlow by a call to a corresponding SMCalFlow function that constructs a Time object (e.g., NumberAM(5)). However, if the examples retrieved for the prompt tend to only have times in the opposite half of the day, GPT-3 might call the wrong function, and translate the time sub-expression in “Schedule it for 5pm” into NumberAM(5). To avoid this, if we detect exactly one of “a.m.” or “p.m.” in the utterance, we remove the time construction functions associated with the opposite pattern from the candidates. We do the same filtering with days of the week and months, which are also constructed by specific functions.”, wherein it is noted that a logit bias will cause the LLM to discard the undesired keywords that are not to be blindly copied into the request for output).
Claim 5:
Poesia and Yager teach the method of claim 1.
Poesia further teaches wherein the user-provided language description of the constraint comprises one or more of: one or more retrieved documents relevant to the input request (i.e. pg. 1, “TST learns to recognize utterances that describe similar target programs despite differences in surface natural language features”, wherein the BRI for documents encompasses target programs); and one or more distilled concepts relevant to the input request (i.e. pg. 2, “TST mit igates conceptual errors by learning to select examples with similar intent, even when their natural language descriptions seem unrelated in form”, wherein the BRI for a distilled concept encompasses a user intent).
Claim 6:
Poesia and Yager teach the method of claim 1.
Poesia further teaches wherein the generating, by the decoder, the next output token comprises: selecting a number of top candidate tokens from a candidate set based on the adjusted conditional probability distribution at a first decoding step, wherein the candidate set comprises one or more desired keywords (i.e. pg. 12, “The CSD procedure uses the model M on the partial program to generate a distribution on the next token, but constrains it to belong to the set of valid tokens determined by the completion engine CL.”, wherein it is noted that the top candidate tokens may only be selected based on belonging to a set of valid tokens that are based on prompt keywords).
Claim 7:
Poesia and Yager teach the method of claim 1.
Poesia further teaches wherein the neural network model is a pretrained large language model, and wherein the natural language output is generated for a variety of natural language processing (NPL) tasks without finetuning the pretrained large language model (i.e. pg. 1, Then, SYNCHROMESH feeds the examples to a pre-trained language model and samples programs using Constrained Semantic Decoding(CSD): a general framework for constraining the output to a set of valid programs in the target language. CSD leverages constraints on partial outputs to sample complete correct programs, and needs neither re-training nor fine-tuning of the language model.).
Claim 8:
Claim 8 is the system claim similar to claim 1 and is rejected for similar reasons.
Poesia also teaches
the system comprising: a communication interface that is configured to receive an input request for generating a natural language output (i.e. pg. 9, we argue that improving validity is especially important for user-facing applications. Users of language-to-code systems might need to rephrase their request or to edit the system’s output); a memory storing an encoder and a decoder of the neural network model, and a plurality of processor-executable instructions; and one or more processors to execute the plurality of processor-executable instructions (i.e. pg. 5, “Our only access to the models was through the public OpenAI HTTP API, which allowed us to apply constraints by adding a bias to logits”, wherein it is noted that the neural network would be stored on the processing devices memory).
Claim 10:
Claim 10 is the system claim similar to claim 3 and is rejected for similar reasons.
Claim 11:
Claim 11 is the system claim similar to claim 4 and is rejected for similar reasons.
Claim 12:
Claim 12 is the system claim similar to claim 5 and is rejected for similar reasons.
Claim 13:
Claim 13 is the system claim similar to claim 6 and is rejected for similar reasons.
Claim 14:
Claim 14 is the system claim similar to claim 7 and is rejected for similar reasons.
Claim 15:
Claim 15 is the medium claim similar to claim 1 and is rejected for similar reasons.
Claim 17:
Claim 17 is the medium claim similar to claim 3 and is rejected for similar reasons.
Claim 18:
Claim 18 is the medium claim similar to claim 4 and is rejected for similar reasons.
Claim 19:
Claim 19 is the medium claim similar to claim 5 and is rejected for similar reasons.
Claim 20:
Claim 20 is the medium claim similar to claim 6 and is rejected for similar reasons.
Claim(s) 2, 9, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Poesia, G., Polozov, O., Le, V., Tiwari, A., Soares, G., Meek, C., & Gulwani, S. (2022). Synchromesh: Reliable code generation from pre-trained language models. ArXiv.org. https://arxiv.org/abs/2201.11227 (Year: 2022), hereinafter “Poesia”, in light of Yager, N. (2018, May 4). Neural text generation: How to generate text using conditional language models. Medium: Pharasee. Neural text generation: How to generate text using conditional language models (Year: 2018), hereinafter “Yager”, and further in light of Gao, L. (2021, October 11). Multiple Choice Normalization in LM Evaluation. EleutherAI Blog; EleutherAI. https://blog.eleuther.ai/multiple-choice-normalization, hereinafter “Gao”.
Claim 2:
Poesia and Yager teach the method of claim 1.
Poesia further teaches
wherein the constraint term is computed as a logit of a conditional (i.e. pg. 6, “Suppose s ∈ Lc is the language model’s output so far (we start with ). If ΣM is the model’s vocabulary, we can compute the set of valid next tokens VM(s) = {t ∈ ΣM : st ∈ Lc} by using our decision procedure for each token in the vocabulary ΣM”, wherein the previously decoded valid tokens and user language are applied as a logits bias and thus eliminates the probability of invalid programs and improves prediction accuracy of the model as the distribution of the next token depends on the language model’s output so far. Thus the conditional probability for accurate predictions for the model is constrained by the conditions set by the previously decoded tokens and user language).
Yager further teaches the concept of a conditional probability (i.e. pg. 2-3,Sample from the conditional word probability distribution. Words that are a better fit are more likely to be selected. For the example above, we would select the word “food” with probability 62%, “toys” with probability 14%, “aardvark” with probability 0.001%, etc.”).
While Poesia-Yager teach that the constraint term is computed as a logit of a conditional probability, Poesia may not explicitly teach that the constraint term is
divided by a length of the user-provided language description.
However Gao teaches, a logit of a conditional probability that is
divided by a length of the user-provided language description (i.e. pg. 1, “
Let
PNG
media_image3.png
32
54
media_image3.png
Greyscale
be the prompt, and
PNG
media_image4.png
34
64
media_image4.png
Greyscale
be the ith possible continuation with a token length of
PNG
media_image5.png
28
90
media_image5.png
Greyscale
There are several ways to use a language model to rank multiple possible continuations to a prompt. Since the language model only gives (log) probabilities for the next token given the context
PNG
media_image6.png
34
194
media_image6.png
Greyscale
.
The following are several possible ways to resolve this problem:
Token-length normalized: The score of continuation i is determined using
PNG
media_image7.png
32
248
media_image7.png
Greyscale
This approach attempts to normalize for length by computing average log probability per token; however, this approach is not tokenization agnostic, and as such two models with different tokenization that assign the same log likelihood to every single input string will have different token-length normalized scores. This approach is used by GPT-3 in most tasks”, wherein it is noted that the BRI for a length of the user-provided language description would encompass a token-length. While a logit is not specifically referenced, the BRI for a logit equates to a sum of log probability of tokens as being divided by a token-length, as the concept of applying token-length normalization to a GPT-3 LLM solves a similar problem to the source by dividing a log probability by a token length of a prompt).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add that a conditional probability that is divided by a length of the user-provided language description, to Poesia-Jager’s system that uses a logit bias in an autoencoding LLM, with how a value for a log probability for a next token for a LLM may be dividing by a length of a prompt, as taught by Gao. One would have been motivated to combine Gao with Poesia, and would have had a reasonable expectation of success in doing so as using techniques may improve the performance of the LLM as a normalization reduces the ambiguity in handling scoring for arbitrary continuations
Claim 9:
Claim 9 is the system claim similar to claim 2 and is rejected for similar reasons.
Claim 16:
Claim 16 is the medium claim similar to claim 2 and is rejected for similar reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Patent No. 11934781 “He”, teaches in Col. 6, lines 17-26, the conditional probability distribution p(y|x, z) of generated summaries from the summarization model is compared with the ground-truth summary to compute a cross-entropy loss, which may be used to update the summarization model via backpropagation
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID H TAN whose telephone number is (571)272-7433. The examiner can normally be reached M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.T./Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights