DETAILED ACTION
This final office action is responsive to the amendment filed on May 20, 2026. Claims 1-5, 7-13, and 15-22 are pending. Claims 1, 9, and 17 are independent. Claims 6 and 14 are canceled. Claims 21 and 22 are added.
Specification objections are withdrawn in light of applicant’s amendment to the specification.
Claim rejections under 35 USC §112(a) of claims 1-5, 8-13, and 16-20 are withdrawn in light of applicant’s amendment and remarks. However, rejection of claims 7 and 15 are maintained.
Claim rejections under 35 USC §112(b) of claims 1-2, 8-10, and 16-18 are withdrawn in light of applicant’s amendments and remarks. However, rejections of claims 3-5, 7, 11-13, 15, and 19-20 are maintained.
Claim rejections under 35 USC §103 have been updated in light of applicant’s amendments. See sections Claim Rejections – 35 USC §103 and Response to Arguments below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 7, 15, and 22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 7, 15, and 22, the first limitation reads “selecting a sequence of candidate prompt vectors from N prompt vectors, wherein the N prompt vectors have serial number values 1, 2, … N respectively; and a third difference between serial number values corresponding to each pair of two adjacent candidate prompt vectors in the sequence of candidate prompt vectors is K, where K is a positive integer.” While the claim has been amended to further clarify the serial number values, this is not supported within the original disclosure.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3-5, 7, 11-13, 15, and 19-22 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 3-5, 11-13, and 19-21, the claims state “determining first differences, each between first scores corresponding to each pair of two adjacent prompt vectors of the L prompt vectors.” It is unclear whether the first scores are corresponding to a pair of adjacent vectors or if there is a first score for each vector wherein the vectors are adjacent. For prior art purposes, the limitation is being interpreted as involving many prompt vectors 1, …, L indexed next to each other in sequential order and that each prompt vector instance has a corresponding first score, and this limitation is looking at the differences between the first scores of adjacent prompt vectors, e.g., the first score of prompt vector 1 compared to the first score of prompt vector 2, etc.
Regarding claims 7, 15, and 22, the claim states “a third difference between serial number values corresponding to each pair of two adjacent candidate prompt vectors in the sequence of candidate prompt vectors is K, where K is a positive integer;” It is unclear on what the third difference is being based on. The claim says that it is a difference between serial number values corresponding to each pair of two adjacent candidate prompt vectors which would indicate that K is 1 as the vectors are adjacent. If K is greater than one then the adjacency of the vectors is not maintained and there would not be a difference between a pair of two adjacent vectors since they would no longer be adjacent. For instance, if K = 4 then it would be prompt vector 1 compared to prompt vector 5 and prompt vector 2 compared to prompt vector 6 which are no longer adjacent vectors.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8-9, and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Lester et al. (US20230325725), hereinafter Al-Rfou, in view of Liang et al. (Super Tickets in Pre-Trained Language Models), hereinafter Liang.
Regarding claim 1, Al-Rfou teaches:
obtaining a first one of prompt vectors and a first vector corresponding to sample data; (Al-Rfou, paragraph 0131: “The “soft prompt” can be trained end-to-end and can condense the signal from a full labeled dataset, which can allow the systems and methods to outperform “few-shot” prompts.” And paragraph 0136: “The soft-prompts can be represented as a parameter Peϵ
PNG
media_image1.png
42
79
media_image1.png
Greyscale
, where p is the length of the prompt. In some implementations, the prompt can then be concatenated to the embedded input forming a single matrix [Pe; Xe]
PNG
media_image2.png
46
146
media_image2.png
Greyscale
which then flows through the encoder-decoder as normal.” – The soft prompt being concatenated with the embedded input to form a single matrix is analogous to the first prompt vector and a first vector corresponding to sample data.)
obtaining N pruned models … where N is any integer greater than 1; (Al-Rfou, paragraph 0165: “By training N prompts on the same task, the system can create N separate “models” for a task, while still sharing the core language modeling parameters throughout.” – The N separate “models” is analogous to the N pruned models.)
obtaining a first score corresponding to the first one of the prompt vectors by fusing the first vector and the first one of the prompt vectors and inputting the fused first vector and first one of the prompt vectors into the N pruned models; (Al-Rfou, paragraph 0031: “A prompt gradient can then be determined based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples. In some implementations, the prompt gradient can be determined by evaluating a loss function that is evaluated based on a difference between the training output and the one or more training labels.” – The prompt gradient is analogous to the first score, where the output comes from the initial input of the soft prompt concatenated with the embedding input, e.g. the fusing of the first vector and the first prompt vector.)
determining a second one of the prompt vectors by modifying, based on the first score, the first one of the prompt vectors, wherein each element in the first one of the prompt vectors is added to the first score, to modify the first one of the prompt vectors, and the modified vector is determined as the second one of the prompt vectors; and (Al-Rfou, paragraph 0032: “One or more prompt parameters of a prompt can then be adjusted based on the prompt gradient.” And paragraph 0054: “The model's prediction can be compared to the target to calculate a loss, and the error can be back-propagated to calculate gradients, however the system may only apply these gradient updates to our new learnable vectors—keeping the core model frozen.” – The prediction being compared to the target for the loss function which is then propagated back through the gradients is analogous to the second prompt as the prompt parameters are what is being updated and not the model parameters.)
when the second one of the prompt vectors is not a target prompt vector corresponding to the sample data, returning to obtaining the first score, until the target prompt vector corresponding to the sample data is obtained, wherein the target prompt vector is an accurate prompt vector corresponding to the sample data, and is determined by performing forward inference on the pruned models and the prompt vectors; (Al-Rfou, paragraph 0152: “In some implementations, the systems and methods can train prompts for each model size while varying the prompt length in {1, 5, 20, 100, 150}, while fixing the rest of the model hyperparameters. Specifically, the system can use the 100K-step LM-adapted frozen model, and class-label initialization. In some implementations, the XXL model may give strong results with a single-token prompt, suggesting that the larger the model, the less conditioning signal may be used to achieve the target behavior.” – The different model sizes is analogous to the different pruned models, the prompts being trained for the different ones is analogous to determining if the second prompt vector is a target prompt vector and, if not, returning to obtaining the first score. The second prompt vector is being found through the forward inference of the model before being determined to be the target or not.)
wherein obtaining the first score corresponding to the first one of the prompt vectors by fusing the first vector and the first one of the prompt vectors and inputting the fused first vector and first one of the prompt vectors into the N pruned models comprises: obtaining predictive tags outputted by the N pruned models respectively by fusing the first vector and the first one of the prompt vectors and inputting the fused first vector and first one of the prompt vectors into the N pruned models; (Al-Rfou, paragraph 0032: “In some implementations, the prompt can be trained for a particular task associated with the one or more training examples and the one or more training labels such that the prompt is configured to be input with input data to the pre-trained machine-learned model to generate output data associated with the particular task.” – The loss function is generated based on the output, wherein the output is compared to the target (labels) in order to determine the gradient (first score).)
using a difference between a predictive tag outputted by a pruned model among the N pruned models and a tagging tag corresponding to the sample data, as a second score corresponding to the first one of the prompt vectors under the pruned model; and (Al-Rfou, paragraph 0077: “For example, a loss can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the loss function).” – The loss function is a difference between the predictive tag output and the tagging tag (e.g., the label).)
performing mean value processing on a plurality of second scores to determine the first score corresponding to the first one of the prompt vectors. (Al-Rfou, paragraph 0077: “Various loss functions can be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, a ranking loss, and/or various other loss functions. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations.” – The mean squared error is analogous to the mean value processing of the plurality of second scores (the iterative updates).)
Al-Rfou does not explicitly teach:
obtaining N pruned models by N different pruning processing on the pre-trained model, where N is any integer greater than 1;
However, Liang teaches:
obtaining N pruned models by N different pruning processing on the pre-trained model, where N is any integer greater than 1; (Liang, page 1, column 1, paragraph 2: “The Lottery Ticket Hypothesis (LTH, Frankle and Carbin (2018)) suggests that an over-parameterized network consists of “lottery tickets”, and training a certain collection of them (i.e., a subnetwork) can 1) match the performance of the full model; and 2) outperform randomly sampled subnetworks of the same size (i.e., “random tickets”).”) and page 5, column 1, paragraph 1: “Specifically, we prune BERT-base/large in unit of 10% heads and 10% feed-forward layers (FFN) at 8 different sparsity levels (10% heads and 10% FFN, 20% heads and 20% FFN, etc).” – The different sparsity levels is analogous to the N different pruning processes. Here, 8 is the number of pruned models which is N and therefore greater than 1.)
Liang is considered analogous to the claimed invention as it is in the same field of endeavor, machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified Al-Rfou, which already teaches the method of determining a prompt vector from N "models" but does not explicitly teach the N models are pruned by N different pruning processes on the pre-trained model, to include the teachings of Liang which does teach the N models are pruned by N different pruning processes on the pre-trained model since “generalization performance of the winning tickets selected at appropriate compression ratios can not only match, but also exceed that of the full model.” (Liang, page 1, column 2, paragraph 3)
Regarding claim 8, Al-Rfou and Liang teach the method of claim 1, as cited above.
Al-Rfou does not explicitly teach:
determining a number m of neurons to be pruned, where m is any positive integer; and
obtaining the N pruned models by the N different pruning processing on the pre- trained model based on the number m of neurons to be pruned, wherein at least one neuron between every two pruned models is different.
However, Liang further teaches:
determining a number m of neurons to be pruned, where m is any positive integer; and obtaining the N pruned models by the N different pruning processing on the pre- trained model based on the number m of neurons to be pruned, wherein at least one neuron between every two pruned models is different. (Liang, page 5, column 1, paragraph 1: “Specifically, we prune BERT-base/large in unit of 10% heads and 10% feed-forward layers (FFN) at 8 different sparsity levels (10% heads and 10% FFN, 20% heads and 20% FFN, etc).” – Wherein the different sparsity percentages indicates that each layer is being pruned and thus there are 10% of the neurons in the head and feed-forward layers being pruned which therefore indicates that there are m positive number of neurons pruned in each layer. The 8 different sparsity levels indicates that there are a number of different pruning processes being done (levels of sparsity).)
Regarding claim 9, Claim 9 has all the same limitations of claim 1 which are taught by Al-Rfou and Liang – see claim 1 above.
Al-Rfou further teaches:
An electronic device, comprising: a processor; and a memory communicatively coupled to the processor; wherein, the memory is configured to store instructions executable by the processor, and the processor is configured to execute the instructions to: (Al-Rfou, paragraph 0005: “The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations.”)
Regarding claim 16, Al-Rfou and Liang teach the device of claim 9, as cited above.
Claim 16 additionally has the same limitations of claim 8 which are taught by Al-Rfou and Liang – see claim 8 above.
Regarding claim 17, Claim 17 has all the same limitations of claim 1 which are taught by Al-Rfou and Liang – see claim 1 above.
Al-Rfou additionally teaches:
A non-transitory computer-readable storage medium storing thereon computer instructions, wherein the computer instructions, when executed by a computer, cause the computer to perform: (Al-Rfou, paragraph 0005: “The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations.”)
Claims rejected under 35 U.S.C. 103 as being unpatentable over Al-Rfou in view of Liang in view of Lester et al. (The Power of Scale for Parameter-Efficient Prompt Tuning), hereinafter Lester.
Lester was cited in applicant’s IDS dated 4/11/2024.
Regarding claim 2, Al-Rfou and Liang teach the method of claim 1, as cited above.
Al-Rfou and Liang do not explicitly teach:
wherein returning to obtaining the first score comprises:
obtaining L prompt vectors previously adjacent to a (N+1) th one of the prompt vectors and a first score corresponding to each of the L prompt vectors, where L is a positive integer less than or equal to N and greater than 1, and N is a positive integer greater than 1;
determining a modifying mode of the (N+1) th one of the prompt vectors based on the first score corresponding to each of the L prompt vectors; and
based on the modifying mode of the (N+1) th one of the prompt vectors, generating a (N+2) th prompt vector by modifying the (N+1) th one of the prompt vectors.
However, Lester teaches:
wherein returning to obtaining the first score comprises: obtaining L prompt vectors previously adjacent to a (N+1) th one of the prompt vectors and a first score corresponding to each of the L prompt vectors, where L is a positive integer less than or equal to N and greater than 1, and N is a positive integer greater than 1; determining a modifying mode of the (N+1) th one of the prompt vectors based on the first score corresponding to each of the L prompt vectors; and based on the modifying mode of the (N+1) th one of the prompt vectors, generating a (N+2) th prompt vector by modifying the (N+1) th one of the prompt vectors. (Lester, page 9, column 1, paragraph 2: “To test the interpretability of our learned soft prompts, we compute the nearest neighbors to each prompt token from the frozen model’s vocabulary. We use cosine distance between the vocabulary embedding vector and the prompt token representation as the similarity metric. We observe that for a given learned prompt token, the top-5 nearest neighbors form tight semantic clusters.” – The nearest neighbors is analogous to the adjacent vectors where they are being computed based the comparison using cosine distance. Therefore, the (N+2)th prompt vector is being obtained based off the comparison of the neighbors with the (N+1)th vector.)
Lester is considered analogous to the claimed invention as it is in the same field of endeavor, machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified Al-Rfou and Liang, which already teaches obtaining first scores of the prompt vectors but does not explicitly teach generating prompt vectors based off the adjacent prompt vector scores, to include the teachings of Lester which does teach generating prompts based off nearest neighbors in order to "store the expected output classes in the prompts as reference, and initializing the prompt to outputs classes makes this easier and more centralized." (Lester, page 9, column 1, paragraph 4)
Regarding claim 10, Al-Rfou and Liang teach the device of claim 9, as cited above.
Claim 10 additionally has the same limitations of claim 2 which are taught by Al-Rfou, Liang, and Lester – see claim 2 above.
Regarding claim 18, Al-Rfou and Liang teach the device of claim 17, as cited above.
Claim 18 additionally has the same limitations of claim 2 which are taught by Al-Rfou, Liang, and Lester -see claim 2 above.
Regarding claims 3-5, 11-13, and 19-21, based of the cited 35 USC §112 issues cited above, examiner generally understands these claims as teaching finding and modifying similar prompt vectors. Lester appears to teach this on page 9, section 7 when further discussing the neighboring prompts.
Claims 7, 15, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Al-Rfou in view of Liang in view of Ma et al. (XPrompt: Exploring the Extreme of Prompt Tuning), hereinafter Ma.
Regarding claims 7, 15, and 22, based off the cited 35 USC §112 issues cited above, examiner generally understand that a set of candidate prompt vectors are first determined before being input into the models to obtain a predicted tag to compare to a tagging tag in order to determine the target prompt vector. Ma and Al-Rfou appear to teach this as Ma discusses a collection of prompt tokens on page 11034, column 1, Al-Rfou teaches obtaining a predictive tag in the manner described in claim 1 above, with Ma then appearing to teach the determining a first score and determining a candidate prompt on page 11036, section 4.2.1.
Ma is analogous to the claimed invention as it is in the same field of endeavor, machine learning and prompt tuning. A person of ordinary skill in the art would have been motivated to combine the teachings of Ma with that of Al-Rfou and Liang in order to yield “a more parameter-efficient prompt yet with a competitive performance.” (Ma, abstract)
Response to Arguments
Regarding claim rejections under 35 USC §112(a) and (b), applicant’s amendments and remarks have overcome the rejections under 112(a) for all claims except claims 7, 15, and 22. While Applicant states “applicant has clearly defined the serial number values of N prompt vectors, as well as serial number values of the candidate prompt vectors,” Applicant has not noted where in the original disclosure this definition can be found and, therefore, the rejection stands. Further, the rejection under 112(b) has been clarified and further describes the issues presented in the previous office action.
Applicant’s amendments to claims 3-5, 11-13, and 19-21 has overcome the rejections under 35 USC §112(a) and previous rejection under 112(b), the amendment has not fully clarified the first limitation, as noted above.
Claim rejections under 35 USC §112(a) has been withdrawn for claims 1-5, 8-13, and 16-20. However, they are maintained for claims 7, 15, and 22. Claim rejections under 35 USC §112(b) has been withdrawn for claims 1-2, 8-10, and 16-18. However, they are maintained for claims 3-5, 7, 11-13, 15, and 19-22.
Applicant’s arguments with respect to claim(s) 1, 9 and 17 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Lester is no longer relied upon to teach the limitations of claims 1, 9, or 17. Regarding applicant’s argument that the backpropagation of Lester teaches away from the forward inference of the claimed invention, examiner disagrees. The claim does not state that backpropagation is not used to train or find the scores of the prompt vectors. The claim sates “the target prompt vector is an accurate prompt vector corresponding to the sample data, and is determined by performing forward inference on the pruned models and the prompt vectors” which indicates that forward inference is used to determine the second prompt vector (the output) and compare it to the target prompt vector wherein the iteration is complete if they match or starts over if they do not. Using backpropagation anywhere else in this process does not teach away from the inference being forward inference.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lv et al. (Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACQUELINE MEYER whose telephone number is (703)756-5676. The examiner can normally be reached M-F 8:00 am - 4:30 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571-272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.C.M./Examiner, Art Unit 2144
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144
Prosecution Insights