DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of US Provisional Application No. 63487642, filed march 1, 2023. Claims 1-4,9, 11-14, and 19 have been afforded the benefit of this filing date.
Applicant claims the benefit of US Provisional Application No. 63487642, filed march 1, 2023 but claims 5-8, 10, 15-18, and 20 contain subject matter not disclosed in the provisional application, thus the effective filing date of February 29, 2024 has been used.
If applicant intends to have these claims afforded the benefit of the earlier filing date, applicant may: (1) amend the claim limitation(s) to avoid it/them containing subject matter not in the provisional application; or (2) present a sufficient showing that the claim limitation(s) recite(s) material from specific sections of the provisional application.
Response to Amendment and Arguments.
Applicant’s arguments, see page 6, filed 2/9/2026, with respect to claims 1-20 rejection have been fully considered and are not persuasive. Applicant argues “Paragraph 3 of the present specification describes a technical challenge in the use language models. Fine-tuning a pretrained language model for a particular task "can be inefficient, as such models may have many billions or trillions of parameters." Paragraph 14 outlines a solution to this problem, "Tuning a language model in a particular task can be performed by using prompt tuning, whereby a prompt is added to a given input, causing the language model to provide results that are adapted to the task. A dynamic prompt may be which may vary by position, length, and prompt pool to provide superior results from a pretrained language model." In this manner the performance of the language model may improve without incurring the heavy computational cost needed to fine-tune it. The remainder of the specification provides specific details for how the prompt may be tuned,”. However, examiner disagrees, since claim one does not reflect the improvement of allegedly reducing computing cost, such as keeping the pretrained model fixed, adjusting the prompt parameters only, and using soft prompt vectors. The claim as it stands now only recite dynamically, tailoring a prompt for a task, rather than a technological improvement. Applicant further argues that “The claims reflect this improvement. The independent claims in particular recite steps that create a. prompt which is added to an input query and then applying the combined input to a language model. This reflects the described improvement, whereby the input query is modified to create the dynamic prompt that provides superior results from the language model”. The Examiner respectfully disagrees the language models is an additional element (see paragraph 32) which is recited to be a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component. Furthermore, this would still be a mental process of a person receiving the query and a topic along with it. An example would be an astrology question about the solar system then a question for that topic. Additionally, the group of people answering those questions, additionally categorizing them and scoring them. Further, this individual would be the final editor and can work alongside them and choose the final answer. Hence being a mental process. The Examiner respectfully disagrees this does not solve problems within the computer technology. Since the language models are being used to combine a topic and query together. Having a combined input is not an improvement of a computer. Applicant further argues for claims 2,4,9,10,12,14,19, and 20 ” The dependent claims go further, for example specifying in claims 9 and 19 that the language model is a pretrained language model based on transformers, and specifying in claims i10 and 20 that the tuning function may be trained using supervised learning based on a set of training examples. Claims 2--4 and 1 2-14'- recite substantive details on how the prompt position, prompt length, and prompt pool are implemented in the prompt text. These provide substantive technical derails that. further establish how the improvement is reflected in the claims.”. However the examiner disagrees, since the dependent claims just add the implementation, having pretrained transformer and supervised learning just shows the technological environment and training approach. Further prompt position, length, and pool, describe the prompt characteristic not a particular unconventional Menckenism that improves the transformer operation or cost usage. The claims are not patent eligible.
Therefore, the 101 rejection of claims 1- 20 are maintained.
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are not persuasive. Applicant states that “The rejection cites Lester as teaching the features of claim 3, particularly in a discussion of using a loss function to learn the tokens of a soft prompt. However, Lester never attempts to learn the length of the soft prompt. which is described as "a fixed-length sequence of vectors." Its learning process appears to be limited to determining the values of those vectors.”. However, the examiner strongly disagrees see paragraph 113 which teaches using a loss function of the language model to train prompt parameters. while, paragraph 138 and 152 teaches treating the prompt length as a variable to be selected based on the performance. Taken together these paragraphs teach selecting a prompt length by training/evaluating candidate prompt length using the model’s loss or task performance. Therefore, applicants’ argument for claim 1 is not persuasive and rejection is till maintained.
Applicant further argues claim two “The rejection cites Peng's background, which acknowledges that a soft prompt may be prepended or appended to an input sequence. However, the combination of references fails to read on the actual claim language, which recites that the prompt text includes a prefix part and a postfix part. The body of Peng's specification only ever describes prepending soft prompts and never says that a given prompt may be split into a prefix part and a postfix part that are both added to the input query. It is therefore respectfully asserted that Lester and/or Peng, taken alone or in combination, fail to disclose or suggest selecting a prompt prefix length and a prompt postfix length such that the prompt text includes a prefix part that is prepended to the input query and a postfix part that is appended to the input query.”. However, examiner disagrees, the claim states “The method of claim 1, wherein training the tuning function selects a prompt prefix length and a prompt postfix length, such that the prompt text is includes a prefix part that is prepended to the input query and a postfix part that is appended to the input query.”. The claim requires training a tuning function that selects prefix length , and a postfix length. Those selected length cause the prompt text to be split so that one portion is placed before or after the query. Further everything after “such that” is an intended result. Since the claim recites selection of a "prefix length" and "postfix length". The claim does not indicate that both have to be used at the same time or how both are utilized. The reference although teaches the prefix OR postfix it does teach using both as an option (see paragraph 16) and thus the selection thereof. Therefore, applicants’ argument for claim 2 is not persuasive and rejection is till maintained.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1,2,4-12,14-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1 and 11 Further claim 1 recites A computer-implemented method for prompt tuning, comprising:
training a tuning function to set prompt position, prompt length, or prompt pool based on a language processing task including selecting the prompt length by minimizing a loss function of a language model;
applying the tuning function to an input query to generate a combined input, with prompt text having the prompt length, being selected according to the prompt pool, and being added to the input query at the prompt position; and
applying the combined input to a language model.
Further claim 11 states A system for prompt tuning, comprising:
a hardware processor; and
a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to:
The limitation of “training…”, “applying…”, and “applying…”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person receiving a query from a user and assigning themselves a topic to answer it. More specifically a query asking about the solar system and the person thinking he’s an astrologist and answering the question
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are computer components “language model” (paragraph 32) “processor” (paragraph 40) and “memory” (paragraphs 42) recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using the computer components amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are not patent eligible.
Claims 2 and 12 additionally recite the method of claim 1, wherein training the tuning function selects a prompt prefix length and a prompt postfix length, such that the prompt text is includes a prefix part that is prepended to the input query and a postfix part that is appended to the input query. However, this limitation does not prevent a human from performing the steps mentally as described above. Further, the person updating the topic to be before or after the query. Thus, these claims are directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claims are not patent eligible.
Claims 4 and 14 additionally recites the method of claim 1, wherein training the tuning function selects the prompt text according to a weighted sum of prompts in a prompt pool. However, these limitations encompass a person receiving a query from a user and assigning themselves a topic from a topic pool. While assigning it seeing which ones the best based upon the query. Thus, these claims are directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claims are not patent eligible.
Claims 5 and 15 additionally recites the method of claim 1, wherein the language model implements a chatbot that has access to patient information for medical decision making in a healthcare setting. However, these limitations encompass a person receiving a query from a patient and answering it based on the patient’s information. Thus, the claim is directed towards a mental process. In particular, the claim only recites additional elements that are computer components “language model” (paragraph 32) and “chatbot” (paragraphs 32) recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
Claims 6 and 16 additionally recites the method of claim 5, wherein the input query is from a healthcare professional regarding a patient’s condition or treatment, further comprising performing an action responsive to an output of the language model. However, these limitations encompass a doctor asking another doctor in regards for a patient’s future treatment plan. Thus, the claim is directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claims 7 and 17 additionally recites the method of claim 5, wherein the input query is from a patient regarding a condition or treatment of the patient, further comprising automatically altering the patient’s treatment based on an output of the language model. However, these limitations encompass a person asking in regards f his condition and while asking about his condition new information came in and a new treatment plan was given. Thus, the claim is directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claims 8 and 18 additionally recites the method of claim 5, wherein the patient information includes medical history information and treatment information. However, these limitations encompass a person having the patients’ medical history and past treatment operations. Thus, the claim is directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claims 9 and 19 additionally recite he method of claim 1, wherein the language model is a pretrained language model based on transformers. However, these limitations encompass a person training or being trained. Thus, the claim is directed towards a mental process. Similar to above, in particular, the claim only recites additional elements that are computer components “language model” (paragraph 32) and “transformers” (paragraphs 19) recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
Claim 10 and 20 additionally recites the method of claim 1, wherein training the tuning function performs supervised learning based on a set of training examples for the language processing task. However, these limitations encompass a person when receiving the query labeling the data. Thus, the claim is directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1,9,10,11,19, and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by US Patent US 20230325725 A1, (Lester; Brian David).
Claim 1 and 11
Regarding Claim 1 and 11, Lester teaches
1. A computer-implemented method for prompt tuning, comprising:
(Paragraph 1 "The present disclosure relates generally to generating and using prompts for utilizing pre-trained machine-learned models. More particularly, the present disclosure relates to prompt tuning in order to generate prompts associated with particular tasks to enable the use of pre-trained machine-learned models without retraining the large pre-trained machine-learned model.")
training a tuning function to set prompt position, prompt length, or prompt pool based on a language processing task;
(Paragraph 34 " In some implementations, prompt tuning can involve inputting parameters with the input data into the frozen model such that only those parameters are updated. Additionally and/or alternatively, the systems and methods may involve only training an initial learnable layer that either precedes the pre-trained machine-learned model or is an initialization layer of the pre-trained machine-learned model. Therefore, only the initial block (e.g., a small set of parameters at the beginning) may be written and/or overwritten, not the entire model. In some implementations, prompt tuning can include learning vectors for new words and tasks. The parameters may be learned directly based on the label comparison. The prompts can include a plurality of values and/or functions."
Paragraph 54 "To create a soft prompt for a given task, the system may first initialize the prompt as a fixed-length sequence of vectors (e.g., 20 tokens long). In some implementations, the systems and methods can attach these vectors to the beginning of each embedded input and feed the combined sequence into the model. Alternatively and/or additionally, the systems and methods can put the prompts at different parts of the input and analyze the effect of the different positions. The model's prediction can be compared to the target to calculate a loss, and the error can be back-propagated to calculate gradients, however the system may only apply these gradient updates to our new learnable vectors—keeping the core model frozen. While soft prompts learned in this way may not be immediately interpretable, at an intuitive level, the soft prompt can be extracting evidence about how to perform a task from the labeled dataset, performing the same role as a manually written text prompt, but without the need to be constrained to discrete language."
Paragraph 37 "The systems and methods can include storing the prompt in a prompt database. The prompt database can include a plurality of prompts associated with a plurality of different tasks. The prompt and the respective task may be paired for storage such that the association can be utilized for obtaining the prompt based on a selection by a user of a desired task.")
including selecting the prompt length by minimizing a loss function of a language model;
(paragraph 54 "To create a soft prompt for a given task, the system may first initialize the prompt as a fixed-length sequence of vectors (e.g., 20 tokens long). In some implementations, the systems and methods can attach these vectors to the beginning of each embedded input and feed the combined sequence into the model. Alternatively and/or additionally, the systems and methods can put the prompts at different parts of the input and analyze the effect of the different positions. The model's prediction can be compared to the target to calculate a loss, and the error can be back-propagated to calculate gradients, however the system may only apply these gradient updates to our new learnable vectors—keeping the core model frozen. While soft prompts learned in this way may not be immediately interpretable, at an intuitive level, the soft prompt can be extracting evidence about how to perform a task from the labeled dataset, performing the same role as a manually written text prompt, but without the need to be constrained to discrete language.")
applying the tuning function to an input query to generate a combined input, with prompt text having the prompt length, being selected according to the prompt pool, and being added to the input query at the prompt position; and
(Paragraph 38 "Additionally and/or alternatively, the systems and methods can include obtaining input text data, processing the prompt and the input text data with the pre-trained machine-learned model to generate output text data, and providing the output text data as an output. In some implementations, the input text data can include one or more words. The output text data can include a plurality of text characters (e.g., a text response, a text classification, a text completion, and/or a text augmentation)."
Paragraph 54 "To create a soft prompt for a given task, the system may first initialize the prompt as a fixed-length sequence of vectors (e.g., 20 tokens long). In some implementations, the systems and methods can attach these vectors to the beginning of each embedded input and feed the combined sequence into the model. Alternatively and/or additionally, the systems and methods can put the prompts at different parts of the input and analyze the effect of the different positions. The model's prediction can be compared to the target to calculate a loss, and the error can be back-propagated to calculate gradients, however the system may only apply these gradient updates to our new learnable vectors—keeping the core model frozen. While soft prompts learned in this way may not be immediately interpretable, at an intuitive level, the soft prompt can be extracting evidence about how to perform a task from the labeled dataset, performing the same role as a manually written text prompt, but without the need to be constrained to discrete language."
Paragraph 37 "The systems and methods can include storing the prompt in a prompt database. The prompt database can include a plurality of prompts associated with a plurality of different tasks. The prompt and the respective task may be paired for storage such that the association can be utilized for obtaining the prompt based on a selection by a user of a desired task.")
applying the combined input to a language model.
(Paragraph 38 "Additionally and/or alternatively, the systems and methods can include obtaining input text data, processing the prompt and the input text data with the pre-trained machine-learned model to generate output text data, and providing the output text data as an output. In some implementations, the input text data can include one or more words. The output text data can include a plurality of text characters (e.g., a text response, a text classification, a text completion, and/or a text augmentation)."
Paragraph 41 "The systems and methods can include processing the input data and the prompt with a pre-trained machine-learned model to generate output data. The output data can be associated with the particular task associated with the prompt. In some implementations, the prompt and the pre-trained machine-learned model may be trained separately. Additionally and/or alternatively, the pre-trained machine-learned model can include a generative pre-trained transformer model. The pre-trained machine-learned model can include an autoregressive language model. In some implementations, the pre-trained machine-learned model may be originally trained with text masking and may be re-trained for auto-completion.")
Regarding claim 11, Lester further teaches
11. A system for prompt tuning, comprising:
a hardware processor; and
(Paragraph 5 "One example aspect of the present disclosure is directed to a computing system for prompt tuning. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a training dataset. The training dataset can include a plurality of training examples and a plurality of training labels for the respective training examples. The operations can include processing, with a pre-trained machine-learned model, one or more training examples of the plurality of training examples and a prompt to generate a training output. In some implementations, a plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning. The prompt can be associated with a particular task. In some implementations, the particular task can be associated with the one or more training examples. The operations can include determining a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples and adjusting one or more prompt parameters of the prompt based on the prompt gradient.")
a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to:
(Paragraph 5 "One example aspect of the present disclosure is directed to a computing system for prompt tuning. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a training dataset. The training dataset can include a plurality of training examples and a plurality of training labels for the respective training examples. The operations can include processing, with a pre-trained machine-learned model, one or more training examples of the plurality of training examples and a prompt to generate a training output. In some implementations, a plurality of pre-trained parameters for the pre-trained machine-learned model can be fixed during prompt tuning. The prompt can be associated with a particular task. In some implementations, the particular task can be associated with the one or more training examples. The operations can include determining a prompt gradient based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples and adjusting one or more prompt parameters of the prompt based on the prompt gradient.")
Claim 9 and 19
Regarding Claim 9 and 19, Lester teaches
The method of claim 1, wherein the language model is a pretrained language model based on transformers.
(Paragraph 41 "The systems and methods can include processing the input data and the prompt with a pre-trained machine-learned model to generate output data. The output data can be associated with the particular task associated with the prompt. In some implementations, the prompt and the pre-trained machine-learned model may be trained separately. Additionally and/or alternatively, the pre-trained machine-learned model can include a generative pre-trained transformer model. The pre-trained machine-learned model can include an autoregressive language model. In some implementations, the pre-trained machine-learned model may be originally trained with text masking and may be re-trained for auto-completion.")
Claim 10 and 20
Regarding Claim 10 and 20, Lester teaches
The method of claim 1, wherein training the tuning function performs supervised learning based on a set of training examples for the language processing task.
(Paragraph 40 "More specifically, the systems and methods can include obtaining input data and a prompt. The prompt can include one or more learned parameters associated with a particular task. In some implementations, the prompt can prime a pre-trained machine-learned model for the particular task. The prompt may be a prompt obtained from a prompt database based on one or more user selections. Additionally and/or alternatively, the prompt may be a prompt generated based on a training dataset that includes a plurality of training examples and a plurality of respective labels. In some implementations, the input data can include text data, image data, video data, audio data, and/or latent encoding data.")
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2,4,12,and 14 are rejected under 35 U.S.C. 103 as obvious over US Patent US 20230325725 A1, (Lester; Brian David) in view of US Patent US 20240070394 A1, (Peng; Xiangyu).
Claim 2 and 12
Regarding Claim 2 and 12, Lester do not explicitly teach all of the method of claim 1, wherein training the tuning function selects a prompt prefix length and a prompt postfix length, such that the prompt text is includes a prefix part that is prepended to the input query and a postfix part that is appended to the input query.
However, Peng teach
The method of claim 1, wherein training the tuning function selects a prompt prefix length and a prompt postfix length, such that the prompt text is includes a prefix part that is prepended to the input query and a postfix part that is appended to the input query.
(Paragraph 3 "Machine learning systems have been widely used in natural language processing tasks, such as question answering, summarization, intent classification, and/or the like. Textual, or discrete/hard, prompts, which are pre-designed templates (e.g., “the sentence is about [what]”), can often be used to make predictions with language models. However, language models are sensitive to the choice of textual prompts. Alternatively, soft/continuous prompts can be used. A soft/continuous prompt is a sequence of additional task-specific tunable tokens prepended (i.e., placed at the beginning) or appended (i.e., placed at the end) to the input sequence and are learned on the task-specific data, a process sometimes called prompt tuning. Prompt tuning can be an efficient and effective paradigm for large-scale language models because during tuning, only the prompts are being updated while the relatively large language model remains frozen. In this way, parameters that need to be updated are relatively small-scale as compared to updating the entire language model. However, under few-shot settings (e.g., when training samples may be scarce for a particular task), prompt tuning may not achieve desirable fine-tuning performance."
Paragraph 23 "In one embodiment, as shown in FIG. 1, for each source task S.sub.1, . . . , S.sub.T, a randomly initialized task-specific soft P.sub.j (j∈[1, T ]) (e.g., 110a-b, 120a-b, 150a-b) may be prepended to the input sequence embedding X (e.g., 112a-d, 122a-d, 152a-d) corresponding to the respective source task. Then a pre-trained language model 100 generates a task-specific output logit (e.g., 114, 124, 154) from a concatenation of the input data and soft prompt. Normalizing the task-specific output logit with a softmax operation 116 generates a predicted source output Y (e.g., 118, 126, 156) corresponding to the respective source task. Finally, the predicted source output 118, 126, 156 and the corresponding source output label from the source training data can be compared to compute a cross-entropy loss. The cross-entropy loss may then be used to backpropagate to update the soft prompt P.sub.j (j∈[1, T ]) (e.g., 110a-b, 120a-b, 150a-b), respectively, while keeping the PLM 100 frozen."
Paragraph 77 "At step 604, a training input sequences is generated (e.g. by a processor 410 of FIG. 4) by prepending one or more soft prompts (e.g., 110a,110b in FIG. 2) to the input (e.g., 212a-d in FIG. 2). For example, the soft prompts may be a sequence of tokens or vectors and the input may also be a sequence of different tokens or vectors. Thus, the training input sequences will be a series of tokens or vectors longer than the sequences that constitute the soft prompts and input. Furthermore, the prepending may occur for each of the source tasks.")
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Lester to incorporate the teachings of Peng to provide a “The method of claim 1, wherein training the tuning function selects a prompt prefix length and a prompt postfix length, such that the prompt text is includes a prefix part that is prepended to the input query and a postfix part that is appended to the input query.” Doing so would Make it computationally efficient, as recognized by Peng. (Paragraph 17).
Claim 4 and 14
Regarding Claim 4 and 14, Lester do not explicitly teach all of the method of claim 1, wherein training the tuning function selects the prompt text according to a weighted sum of prompts in a prompt pool.
However, Peng teach
The method of claim 1, wherein training the tuning function selects the prompt text according to a weighted sum of prompts in a prompt pool.
(Paragraph 31 "FIG. 2 is a simplified block diagram illustrating an aspect of using ensembled soft prompts trained as described in FIG. 1 to train an attention module that generates a classification logit for a target input from a target training dataset, according to embodiments described herein. Given a collection of source prompts [P.sub.1; . . . ; P.sub.T] (e.g., 110a-b, 120a-b, 150a-b) from source tasks [S.sub.1; . . . ; S.sub.T] trained as described in FIG. 1, or not trained, and the PLM 100 with parameters θ, the ensembled soft prompt tuning framework shown in FIG. 2 may be used to train an attention module 200 so that the framework can make predictions for a target task. Specifically, the ensembled soft prompt tuning framework is operated with the PLM 100 paired with a prompt for a source task to train the attention module 200. The prompt from a source task together with the PLM 100 is jointly referred to as a source model, represented as [P.sub.j; θ]. Thus, each source model [P.sub.j; θ] is trained with the few-shot target data in a prompt-tuning manner. This enforces the source models to generate target task's verbalizers given the target input sample 212a-d."
Paragraph 32 "In one embodiment, given a labeled instance (X, y) from the few-shot target training dataset corresponding to a target task T.sub.target, trained or untrained (e.g., randomly initialized) soft prompts (e.g. 110a-b, 120a-b, or 150a-b) may be prepended to the target input data sample X, referred to as [P.sub.j; X]. The input data sequence may comprise one or more embeddings 212a-d. The prepended input [P.sub.j; X] is then fed into the corresponding source model [P.sub.j; θ], i.e, the PLM 100, to generate one or more pre-softmax logits l.sub.x,j (e.g., 214, 224, 254) from the input data and soft prompts [P.sub.j; X]. The generated logits l.sub.x,j (e.g., 214, 224, 254) and a representation 202 of the input (e.g., 212a-d) are sent to the attention module 200 that generates sample-specific attention weights 260 representing the competence of the source model [P.sub.j; θ] for the given input X (e.g., 212a-d)."
Paragraph 33 " In one embodiment, a final logit 262 is generated from a linear combination of the one or more logits L.sub.x=[l.sub.x,1; . . . ; l.sub.x,T]∈custom-character.sup.T×v (e.g., 214, 224, 254) across source tasks 1, . . . , T, where v is the vocabulary size of the pre-trained model. The weight of each logit in the linear combination is given by the attention weights/scores 260. The final logit can be used to make the prediction for input sample X."
Paragraph 47 "Next, normalization layer 370 is applied to the results of up projection, producing {h.sub.l,j}.sub.j=1.sup.T, projected representations of all the logits generated from source tasks. Thus, given h.sub.x and the projected representations of all source logits {h.sub.l,j}.sub.j=1.sup.T the attention score 260 may be computed as:"
See PGPub for equation )
See claim 2 and 12 for rationale.
Claims 5,6,8,15,16, and 18 are rejected under 35 U.S.C. 103 as obvious over US Patent US 20230325725 A1, (Lester; Brian David) in view of US Patent US 20250200331 A1, (KUUSELA; Esa).
Claim 5 and 15
Regarding Claim 5 and 15, Lester do not explicitly teach all of the method of claim 1, wherein the language model implements a chatbot that has access to patient information for medical decision making in a healthcare setting.
However, KUUSELA teach
The method of claim 1, wherein the language model implements a chatbot that has access to patient information for medical decision making in a healthcare setting.
(Fig 4 shows the LLM implementing the AI Agent and having accesses to patient data.
paragraph 105 "With reference to FIG. 4, in some configurations, the LLM 418 may be instructed to describe additional or alternative types of actions than only natural language comments meant to be shown for the experts. The LLM prompt may include descriptions for types of actions or formats of outputs implemented by the LLM 418 and AI agent 416, such that the LLM 418 and AI agent 416 is instructed or configured to ingest inputs and/or produce outputs beyond only text-based natural language comments written by or intended for display to the participating experts. Non-limiting examples of the types of actions of the AI agent 416 include: displaying multimedia data (e.g., diagnostic images) or other forms of data (e.g., laboratory test results) to the Tumor Board members; querying the patient database 404 and identifying similar patient cases in the patient database 404; applying statistical models to evaluate expected outcomes based on the current information in the patient data or other data sources; preparing a discussion continuation plan or filling structured information describing the decisions made or RTTP generated by the Tumor Board; querying a database of current resources available in a clinic or care facility; and determining and displaying possible alternative timelines or locations for the patient treatment.")
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Lester to incorporate the teachings of KUUSELA to provide a “The method of claim 1, wherein the language model implements a chatbot that has access to patient information for medical decision making in a healthcare setting.” Doing so would Increase efficiency of analyzing patient data, as recognized by KUUSELA. (Paragraph 5).
Claim 6 and 16
Regarding Claim 6 and 16, further KUUSELA teaches
The method of claim 5, wherein the input query is from a healthcare professional regarding a patient’s condition or treatment, further comprising performing an action responsive to an output of the language model.
(paragraph 3 "A Tumor Board typically involves a multidisciplinary group of professionals who review and consider treatment options (e.g., RT, chemotherapy, surgery) and related healthcare efforts, such as follow-ups, continued care, and psychosocial support. It is possible that the patient history is reviewed (including aspects related to the diagnosis, such pathology statement and images). In many cases, the stage of the cancer may be defined by the tumor board. The Tumor Board may review and discuss patient-specific factors (e.g., age, other medical conditions) to determine how such factors could affect the treatment possibilities. In some circumstances, for example, the Tumor Board may determine that the RTTP is an appropriate component of the patient care. The RTTP creation typically involves a collaboration of multiple professionals who collectively function as the Tumor Board who diagnose the patients and develop the RTTP for the patient. Especially important in this process is the interaction between the members of the tumor board, and providers (e.g., radiation oncologist). It is common that after the oncologist has first generated a plan for treatment using such a computer model, the oncologist may perform multiple rounds of additional interaction with the computer model because there were ambiguities in the original request, aspects of the plan needed to be improved, or the oncologist wanted to verify that better plans are not available. The information discussed by the members of the tumor board is typically memorialized as the RTTP or tasks of the RTTP."
Paragraph 55 "In some implementations, the interaction interface can simulate an instant messaging application conversation between the experts and the machine learning language processing model. For example, a user can input text into the interaction interface. The interaction interface can be a feed (e.g., a user feed). The text can be or include patient attributes for a patient receiving radiotherapy treatment. For example, the user can select a send button to cause the text to be transmitted to the analytics server. The analytics server can input the text including the patient attributes into the machine-learning language processing model and execute the machine-learning language processing model. Based on the input, the language machine learning model may output a response in text or additional types of information, such as media image data containing image scans, charts/graphs, and the like. The analytics server may present the response in the interaction interface. For instance, the response text may be presented in the interaction interface below the most recent input text by the user, thereby displaying the inputs and responses as a running “chat” interaction within in the interaction interface. The user can respond to the response from the machine learning language processing model with text following the response and submit the user input response. This process can repeat any number of times to simulate an instant message application conversation."
Paragraph 56 " The interaction interface can cover a portion of the user interface of the collaboration software or TBA or be a widget of the user interface of the collaboration software or TBA. The user interface can include other portions or widgets that offer differing functionality, such as external software tools. While the interaction interface can enable communication between the Tumor Board participants and the AI agent(s) implementing corresponding machine learning language processing model(s), the analytics server can also include a portion of the user interface that displays, for example, different patient attributes of patients, treatment options mentioned in the Tumor Board discussion, or other types of outputs generated by a machine learning language processing model. For example, the TBA may capture an audio signal in which a participating member mentions a particular patient attribute, treatment option, or other aspect of the patient's treatment planning, and convert the audio signal into a text input for the AI agent. The machine learning language processing model and the AI agent may be trained to, for example, provide additional information about the particular patient attribute, treatment option, or other aspect of the patient's treatment planning mentioned by the participating member. The AI agent may include this additional information as an output to the user interface presenting the ongoing Tumor Board discussion."
Paragraph 105 "With reference to FIG. 4, in some configurations, the LLM 418 may be instructed to describe additional or alternative types of actions than only natural language comments meant to be shown for the experts. The LLM prompt may include descriptions for types of actions or formats of outputs implemented by the LLM 418 and AI agent 416, such that the LLM 418 and AI agent 416 is instructed or configured to ingest inputs and/or produce outputs beyond only text-based natural language comments written by or intended for display to the participating experts. Non-limiting examples of the types of actions of the AI agent 416 include: displaying multimedia data (e.g., diagnostic images) or other forms of data (e.g., laboratory test results) to the Tumor Board members; querying the patient database 404 and identifying similar patient cases in the patient database 404; applying statistical models to evaluate expected outcomes based on the current information in the patient data or other data sources; preparing a discussion continuation plan or filling structured information describing the decisions made or RTTP generated by the Tumor Board; querying a database of current resources available in a clinic or care facility; and determining and displaying possible alternative timelines or locations for the patient treatment.")
See claim 5 and 15 for rationale.
Claim 8 and 18
Regarding Claim 8 and 18, further KUUSELA teaches
The method of claim 5, wherein the patient information includes medical history information and treatment information.
(paragraph 57 "The TBA may receive an input indicating an identifier (e.g., a name) of the patient into the user interface or the platform provided by the analytics server. Responsive to receiving the input, the analytics server can retrieve patient data (e.g., one or more patient attributes) regarding the patient from non-transitory memory containing database records of the patient database. Examples of patient attributes the analytics server may retrieve include computed tomography (CT) scans of the patient or a tumor of the patient, images of the patient or a tumor of the patient, previously collected patient attributes of the patient, such as data collected from previous health tests, among others. The analytics server can present the retrieved patient data on the user interface in another widget or a portion of the user interface separate or adjacent to the interaction interface. In some cases, the portion or widget of the user interface including the retrieved patient data can be separated from the interaction interface on the user interface by a line (e.g., a vertical line going across the width or length of the user interface (e.g., along the x-axis or the y-axis) of the user interface)."
Paragraph 83 "In step 210, in response to receiving an indication of approval, transmitting, by the processor, the treatment attribute, the first input, and the second input to a radiation therapy plan optimizer as an instruction to generate a radiotherapy treatment plan for the patient. The radiotherapy plan optimizer can be a computer model (e.g., an optimization computer model) that is configured to generate one or more treatment attributes for a radiotherapy treatment plan that comply with the radiation therapy plan objectives for a patient (e.g., objectives in patient attributes of the patient) based on patient attributes of the patient for which the radiotherapy treatment plan is being generated. The radiotherapy plan optimizer can generate the one or more treatment attributes, for example, by iteratively calculating the one or more treatment attributes where, with each iteration, the radiotherapy plan optimizer revises the one or more attributes of the radiotherapy treatment plan in accordance with a cost value. The radiotherapy plan optimizer can receive the first patient attribute of the patient and the second patient attribute of the patient, in some cases in combination with other patient attributes that were used to satisfy a template or plan condition. Based on the patient attributes, the radiotherapy plan optimizer can generate or determine one or more attributes (e.g., radiation dose amounts or other information, field geometry settings, arc settings, treatment frequency, type of treatment, radiation parameters, etc.) of a radiotherapy treatment plan for the patient to generate a radiotherapy treatment plan."
Paragraph 60 "IReferring again to FIG. 2, at step 204, the analytics server may receive, from the interaction interface, a first input comprising a first patient attribute of the patient and a second input corresponding to the radiation therapy treatment of the patient. The analytics server can receive the first input from the interaction interface of the user interface. The first input can include a first patient attribute of a patient. The first input can include the patient attribute and any number of other patient attributes. The first patient attribute and the other patient attributes can be any type of patient attribute, such as, for example, height, gender, weight, treatment options for the patient, treatment attributes (e.g., gantry movements, gantry positions, etc.), treatment objectives, attributes regarding a tumor (e.g., size or shape), images of the patient or tumor, tumor stage, the primary site of treatment, endpoints, whether the tumor has been extended, body mass index, blood pressure, medical history (e.g., previous medical treatments received by the patient, etc.). A user can provide the first input and select a send or submit button on the user interface, for example. The analytics server can receive the first input from the computing device presenting the user interface.")
See claim 5 and 15 for rationale.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as obvious over US Patent US 20230325725 A1, (Lester; Brian David) in view of US Patent US 20250200331 A1, (KUUSELA; Esa) in further view of US Patent US 20220037030 A1, (Kozloski; James R.).
Claim 7 and 17
Regarding Claim 7 and 17, Lester in view of KUUSELA do not explicitly teach all of the method of claim 5, wherein the input query is from a patient regarding a condition or treatment of the patient, further comprising automatically altering the patient’s treatment based on an output of the language model.
However, Kozloski teach
The method of claim 5, wherein the input query is from a patient regarding a condition or treatment of the patient, further comprising automatically altering the patient’s treatment based on an output of the language model.
(paragraph 38 "One or more entities can use the one or more input devices 106 to interact with the reception component 110 (e.g., via the one or more networks 104) and provide data to the control component 108. The received data can regard one or more communications regarding a subject entity (e.g., a patient) and/or can be stored in one or more datasets 122. The one or more datasets 122 can be located in the memory 116 and/or in another location in a cloud computing environment (e.g., accessible via the one or more networks 104). For example, the one or more chatbots can inquire into one or more commitments involving the entity (e.g., the patient). For instance, the one or more chatbots can communicate regarding, for example: past experiences with a physician; past experiences with a subject medicine; past experiences with a treatment; expectations regarding a physician, medication, and/or treatment; a level of satisfaction with a subject physician, medication and/or treatment; and/or a perceived reputation of a subject physician, medication and/or treatment."
Paragraph 64 " In various embodiments, the assessment component 302 can send a command code (e.g., via the one or more networks 104) to the distribution component 402, thereby instructing the distribution component 402 to facilitate distribution of the one or more treatment services based on the computed susceptibility disposition values. For example, the assessment component 302 can send the command code in response to determining a susceptibility disposition value that is greater than or equal to a predefined threshold. In response to receiving the command code, the distribution component 402 can distribute one or more treatment services to one or more entities associated with the subject susceptibility disposition value. For example, the distribution component 402 can distribute one or more treatment services via the one or more networks 104. Further, the distribution component 402 can distribute the one or more treatment services to the one or more input devices 106 to facilitate presentation to the one or more entities. In addition, and/or alternatively, the command code can instruct the distribution component 402 to generate one or more recommendations regarding whether to distribute the one or more treatment services to the subject one or more entities based on the one or more susceptibility disposition values. For example, the one or more generated recommendations can: encourage distribution of the one or more treatment services to the one or more entities based on the one or more susceptibility disposition values being greater than or equal to a predefined threshold; or discourage distribution of the one or more treatment services to the one or more entities based on the one or more susceptibility disposition values being lower than a predefined threshold. The generated recommendation (e.g., a message conveyed via text, audio, and/or video) can be sent (e.g., via the one or more networks 104) by the distribution component 402 to one or more medical professionals and/or the one or more entities."
Paragraph 67 " In various embodiments, the system 100 (e.g., via the control component 108) can autonomously: collect data regarding trust dynamics of an entity (e.g., data regarding communications that indicate the fulfillment or violation of one or more commitments); determine one or trust disposition values based on the collected data; maintain the currency of the one or more trust disposition values using, for example, one or more machine learning technologies; generate one or more trust graphs to predict one or more trust disposition values that can characterize indirect relationships involving the subject entity; determine one or more susceptibility disposition values based on the one or more trust disposition values; and/or distribute one or more treatment services based on the one or more susceptibility disposition values. Therefore, the system 100 tailor one or more treatment services associated to an entity based on recent experience involving the entity that can affect the entity's susceptibility towards the one or more treatment services. Further, the system 100 can determine an entity's susceptibility towards one or more treatment services expeditiously by negating intervention by a medical professional (e.g., a physician). Thus, an individual subject to the analyses and/or evaluations of the system 100 can benefit from up-to-date treatment services distributed during periods of optimal expected effectiveness based on the individual's current disposition of trust.")
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Lester in view of KUUSELA to incorporate the teachings of Kozloski to provide a “The method of claim 5, wherein the input query is from a patient regarding a condition or treatment of the patient, further comprising automatically altering the patient’s treatment based on an output of the language model.” Doing so would Make it up to date treatment services, as recognized by Kozloski. (Paragraph 67).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALI M HASSAN whose telephone number is (571)272-5331. The examiner can normally be reached Monday - Friday 8:00am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras Shah can be reached at (571)270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALI M HASSAN/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
05/30/2026
Prosecution Insights