Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 rejected under 35 U.S.C. 103 as being unpatentable over Pena Pena: 11861884 hereinafter Pen further in view of Hu: LORA: Low Rank Adaptation of Large Language Models (copy provided by Examiner, copyright 10/2021 and hereinafter Hu) and further in view of Zhang: LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention (copy provided by Examiner, copyright 3/26 and hereinafter Zha).
Regarding claim 1
Pen teaches:
A method comprising:
using a first machine learning model, generating a pseudo label associated with domain- specific training data (Pen: Col 2:61-2:64, 3:28-3:45, 6:24-6:30; Fig 2: pseudo labels generated in association with domain specific entity training data to thereby adapt characteristics of a labelled domain upon an unlabeled target domain, in this case the second machine learning model 210 functions in the manner claimed with respect to the first model, such as for text information extraction)
comprising a pseudo label associated with a first domain- specific task and a pseudo label associated with a second domain-specific task (Pen: Abstract; Col 2:28-2:42: system performs key information extraction (KIE) for generation of the pseudo labels based on information classified upon a specific domain, tasks thereof such as utilizing classification types to reify the domain; e.g. the determination of domain tasks relevant to determined KIE such as goods purchased by vendor upon an accounting or tax reporting domain),
wherein the pseudo label comprises machine-generated text extracted from the domain-specific training data (Pen: Col 5:14-5:20, 9:44-9:52; Fig 1, 2: such as by utilizing prototype labels extracted from a receipt such as depicted in figure 1, such as a name, an address, an item type, etc.); and
fine-tuning a second machine learning model, wherein the fine-tuned model generates text using the first model, a domain-specific document and the pseudo labels(Pen: Col 3:45-3:60, 5:32-5:42, etc.; Fig 2: a model subsequent to and generated from the first model, domain specific data, pseudo labels, etc. thereof used to improve the subsequent model for adaptability on a target or subsequent domain, such as by using documents upon the domain, prototype labelling thereof) wherein the fine-tuned second machine learning model generates text of the content type from a domain-specific data (Pen: Col 1:40-1:45, 7:59-8:5, 8:19-25, 8:31-8:34; Fig 2, 3: such as by iteratively improving the textual content of pseudo labels of entities, characteristics thereof and thereby determining additional documents to which to apply pseudo labels).
Pen does not explicitly teach fine tuning the second machine learning model to generate text by applying a sequence prediction task to a domain specific document; nor updating the first and second low rank matrices of the second machine learning model associated with a first domain task using a first loss function and the pseudo label associated with the first domain specific task and updating the first and second low rank matrices of the second machine learning model associated with a second domain specific task using a second loss function and the pseudo label associated with the sconed domain specific task.
In a related field of endeavor Hu teaches fine-tuning a second machine learning model to generate text by applying a sequence prediction task to a domain-specific document (Hu: conditional text generation by predicting a next token over a sequence prediction series of objectives (Hu: Abstract; § 2; eqn 1: downstream conditional text generation by an autoregressive model; “a pre-trained autoregressive language model… can be a generic multi-task learner…” adapted to “downstream conditional text generation tasks, such as summarization, machine reading comprehension, and natural language to SQL,” for modelling of conditional text, sequence prediction, etc. loss over article, document, etc. tokens; that is a domain specific document is provided as input and text is generated as output by a next token sequence prediction task over to the domain specific document);
updating first and second low-rank weight matrices of the second machine learning model associated with first and second domain-specific tasks using first and second loss functions and the pseudo labels associated with the first and second tasks, respectively (Hu: § 1, 4.1, 7.1, 7.2, F, G; Fig 1; eqn 1, 2: “pre-trained model can be shared and used to build many small modules for different tasks” such as by freezing “the shared model,” to thereby “efficiently switch tasks by replacing the matrices A and B’, that is first and second matrices trained, refined, updated, etc. using specific loss function with respect to input output training pairs and subsequently replaced and trained, refined, updated, using a subsequent specific loss function with respect to subsequent input output training pairs; (please see additionally at least the provided Deberta documentation “pretrained language models well known to be trained on task specific labels,” (Deberta: § 1)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the Pen system, method, etc. and the teachings of Hu to thereby tune or fine tune a model, subsequent model, etc. using a multi stage training pipeline comprising determining pseudo labels such as with respect to a confidence score and confidence calibration, thresholding, etc. as taught or suggested by Pen and to adapt the loss based fine tuning thereof by adapting the pipeline to utilize the low rank matrices taught or suggested by Hu to thereby train a reduced set of parameters as taught or suggested by Hu with respect to a reified subset of pseudo labelled data with improved calibration based on the application of specific loss calculations for the first and second Hu matrices as taught or suggested by Pen for at least the purpose of reducing computational costs of the Pen taught fine tuning by compacting the data sizes using low rank matrices to generate domain specific outputs using fewer parameters, smaller structures, etc.; one of ordinary skill in the art would have expected only predictable results therefrom.
Pen in view of Hu does not explicitly discuss or suggest utilizing the taught first and second models to generate text in the manner claimed such as by using a pipeline such as that suggested by Pen in view of Hu.
In a related field of endeavor Zha teaches a system, method, etc. using a first machine learning model to generate pseudo label training data such as from a provided human labelled seed (Zha: § 1: starting from human written pairs the system expands training data to fine tune a model; in this way the system “supplements the limited set of training data (e.g., manual input-output pairs 104) by pseudo labeling outputs”); to thereby fine-tune a second machine learning model based on the first model generated output pairs, pseudo labels, etc. to generate text by applying a sequence prediction task (Zha: § 1, 2.3: based on the above “Alpaca fine-tunes the entire 7B parameters in LLaMA, producing an exceptional model,” such as by autoregressively supervising over fine tuning by generating pseudo labels, pairs, etc. by a first model and maximizing over the sequences thereof); such as by fine-tuning using first and second low rank weight matrices (Zha: § 2.3; Fig 2: system utilizes LORA to fine tune). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to fine tune a model, such as in the manner disclosed by Zha by autoregressively generating pseudo labels for use in a per-task low rank matrix adaptation for conditional sequence prediction as taught or suggested by Pen in view of Hu such as by utilizing the bridging pipeline taught or suggested by Zha and for at least the purpose of
Regarding claim 2
Pen in view Hu in view of Zha teaches or suggests:
The method of claim 1, further comprising: generating, by the first machine learning model, a first training data set of a first size, wherein the first training data set comprises a plurality of pseudo labels paired with domain-specific documents (Pen: Col 1:27-1:40, 3:14-3:18, 4:6-4:17: a small strongly labelled domain dataset comprising documents and entity labeling of textual instances therein is augmented by generated new input label pairs); (Zha: § 1: : machine expanded first training set comprises machine expanded training data, labels, etc. paired with domains specific documents to allow the system to self-instruct domain based training); and
fine-tuning the second machine learning model using the first training data set of the first size (Pen: Col 3:14-3:18, 3:45-4:35; Fig 2: system operates to fine tune based on a set of strongly labelled data, additional pseudo labelled data, etc.); (Zha: § 1: such as by self-supervising training over a domain). The claim is considered obvious over Pen as modified by Hu, and Zha as addressed in the base claim as it would have been obvious to apply the further teaching of Pen, Hu, and/or Zha to the modified device of Pen, Hu, and Zha; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 3
Pen in view of Hu in view of Zha teaches or suggests:
The method of claim 2, wherein the second machine learning model is pretrained on a second training data set of a second size, and the first size of the first training data set is less than the second size of the second training data set (Pen: Col 3:14-3:18, 3:45-4:35; Fig 2: system operates to fine tune based on a set of strongly labelled data, additional pseudo labelled data, etc.); (Zha: § 1: such as by training a language model on a corpus of trillions of tokens and subsequently fine tuning on 52 thousand instruction output pairs). The claim is considered obvious over Pen as modified by Hu, and Zha as addressed in the base claim as it would have been obvious to apply the further teaching of Pen, Hu, and/or Zha to the modified device of Pen, Hu, and Zha; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 4
Pen in view of Hu in view of Zha teaches or suggests:
The method of claim 1, further comprising: accessing the second machine learning model pretrained on domain-neutral data (Pen: 5:45-5:50: a general model pretrained on general data for entity recognition); (Hu: Abstract, § 4.1: system adapts a first, general domain data model, llm, etc. to a second set of tasks or domains) wherein the second machine learning model comprises a plurality of pretrained weights in a pretrained weight matrix (Hu: § 4, 4.1: such as a frozen matrix of weights adaptable to train, fine tune, etc. a second, subsequent, etc. model based on an adaptive relationship between first and second low rank matrices). The claim is considered obvious over Pen as modified by Hu, and Zha as addressed in the base claim as it would have been obvious to apply the further teaching of Pen, Hu, and/or Zha to the modified device of Pen, Hu, and Zha; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 5
Pen in view of Hu in view of Zha teaches or suggests:
The method of claim 1, wherein the first machine learning model comprises a large language model and wherein the second machine learning model comprises a large language model (Pen: 5:45-5:50; Fig 2: a general model pretrained on general data for entity recognition and adapted to create a subsequent fine-tuned domain model); (Hu: Abstract: system adapts a first, general domain data model, llm, etc. to a second set of tasks or domains); (Zha: § 1: models comprise large language models fine-tuned domain specific version thereof). The claim is considered obvious over Pen as modified by Hu, and Zha as addressed in the base claim as it would have been obvious to apply the further teaching of Pen, Hu, and/or Zha to the modified device of Pen, Hu, and Zha; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 6
Pen in view of Hu in view of Zha teaches or suggests:
The method of claim 1, wherein the domain-specific training document is an unstructured document. Examiner considers the claimed subject matter obvious to try. It is well known that datasets, documents may be structured or unstructured and the cited art generally orbits the problem of adapting a domain model trained on a dataset of labelled and pseudo labeled documents upon a second domain of distinct and unlabeled documents. There exist a plurality of means for accomplishing such a task including using structured documents, using unstructured documents and/or a combination thereof these means comprise a finite number of predictable potential solutions which one of ordinary skill in the art could have pursued with a reasonable expectation of success. The claim is considered obvious over Pen as modified by Hu, and Zha as addressed in the base claim as it would have been obvious to apply the further teaching of Pen, Hu, and/or Zha to the modified device of Pen, Hu, and Zha; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 7
Pen in view of Hu in view of Zha teaches or suggests:
The method of claim 1, wherein fine-tuning the second machine learning model comprises defining the first low-rank weight matrix and the second low-rank weight matrix, and the method further comprises: storing the defined first low-rank weight matrix and the defined second low-rank weight matrix associated with the first domain-specific task; storing the first low-rank weight matrix and the second low-rank weight matrix associated with the second domain-specific task. The claim is considered obvious over Pen as modified by Hu, and Zha as addressed in the base claim as it would have been obvious to apply the further teaching of Pen, Hu, and/or Zha to the modified device of Pen, Hu, and Zha; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 8
Pen in view of Hu in view of Zha teaches or suggests:
The method of claim 7, further comprising: inputting, to the second machine learning model, such as a fine tuned model, comprising a plurality of pretrained weights in a pretrained weight matrix, a document to obtain text of the content type from the document, wherein the second machine learning model further comprises an adaptation component comprising the defined first low-rank weight matrix and the defined second low-rank weight matrix (Pen: Col 1:27-1:40, 3:14-3:18, 3:45-4:35, 9:35-9:49; Fig 2: such as the model to be fine-tuned which receives additional documents to adaptively label); (Hu: § 4: such as the frozen weights of the W matrix using the first, second low rank matrices); (Zha: § 1, 2.3: system fine tunes by self-instruction using generated pairs, labels, etc.). The claim is considered obvious over Pen as modified by Hu, and Zha as addressed in the base claim as it would have been obvious to apply the further teaching of Pen, Hu, and/or Zha to the modified device of Pen, Hu, and Zha; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claims 9, 16—the claims are considered to recite substantially similar subject matter to that of claim 1 and are similarly rejected.
Regarding claims 10, 17—the claims are considered to recite substantially similar subject matter to that of claim 2 and are similarly rejected.
Regarding claims 11, 18—the claims are considered to recite substantially similar subject matter to that of claim 3 and are similarly rejected.
Regarding claims 12, 20—the claims are considered to recite substantially similar subject matter to that of claim 5 and are similarly rejected.
Regarding claim 13—the claim is considered to recite substantially similar subject matter to that of claim 6 and is similarly rejected.
Regarding claim 14—the claim is considered to recite substantially similar subject matter to that of claim 7 and is similarly rejected.
Regarding claims 15, 19—the claims are considered to recite substantially similar subject matter to that of claim 8 and are similarly rejected.
Response to Arguments
Applicant’s arguments and claim amendments, see Remarks and Claims, filed 3/12/26, with respect to the rejection(s) of claim(s) 1-20 under 345 USC 103 over Pena, Kalluri, Sandler, and Hu have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Pena, Hu, and Zhang.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Don’t Stop Pretraining: additional system for fine-tuning of pretrained model to plurality of target tasks.
20250021865: adapting a second machine learning model to generate text with respect to domain specific tasks based on pseudo labels generated by a first model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL C MCCORD whose telephone number is (571)270-3701. The examiner can normally be reached 730-630 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CAROLYN EDWARDS can be reached at (571) 270-7136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL C MCCORD/ Primary Examiner, Art Unit 2692
Prosecution Insights