Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the application filed 07/11/2023. Claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted 07/11/2023 has been considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 1: The claim recites “A system comprising:”; therefore, it is directed to the statutory category of a machine.
Step 2A Prong 1: The claim recites, inter alia:
ranks a plurality of finetuned machine learning models based on performance of the plurality of finetuned machine learning models over one or more representative datasets: These limitations recite a mentally performable process with the aid of pen and paper of using observation and judgement to rank a plurality of finetuned machine learning models based on performance of the plurality of finetuned machine learning models over one or more representative datasets.
finetunes a pretrained machine learning model and one or more candidate models selected based on the ranking of the plurality of finetuned machine learning models on one or more target datasets: These limitations recite mathematical calculations similar to an act of calculating using mathematical methods to determine a variable or number per MPEP 2106.04(a)(2)(I)(C).
compares performance of the one or more candidate models to a defined performance metric: These limitations recite a mentally performable process with the aid of pen and paper of using observation and judgement to compare performance of the one or more candidate models to a defined performance metric.
and selects a base model from the pretrained machine learning model and the one or more candidate models based on the performance of the one or more candidate models over the one or more target datasets: These limitations recite a mentally performable process with the aid of pen and paper of using observation, judgement, and evaluation to and select a base model from the pretrained machine learning model and the one or more candidate models based on the performance of the one or more candidate models over the one or more target datasets.
Thus, the claim recites a judicial exception.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
a memory that stores computer executable components; a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise: These additional elements are recited at a high level of generality and amount to invoking computers or other machinery merely as a tool to apply the underlying judicial exception. See MPEP § 2106.05(f).
a ranking component that: These additional elements are recited at a high level of generality and amount to invoking computers or other machinery merely as a tool to apply the underlying judicial exception. See MPEP § 2106.05(f).
and a comparison component that: These additional elements are recited at a high level of generality and amount to invoking computers or other machinery merely as a tool to apply the underlying judicial exception. See MPEP § 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include invoking generic computer components to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 2
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of claim 1 as well as, inter alia:
parameter efficiently finetunes the plurality of finetuned machine learning models on the one or more representative datasets: These limitations recite mathematical calculations similar to an act of calculating using mathematical methods to determine a variable or number per MPEP 2106.04(a)(2)(I)(C).
and compares the performance of the plurality finetuned machine learning models: These limitations recite a mentally performable process with the aid of pen and paper of using observation and judgement to compare the performance of the plurality finetuned machine learning models.
Thus, the claim recites a judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a
practical application and is not considered to be significantly more. As such, the claim is patent
ineligible.
Claim 3
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the parameter efficiently finetuning comprises a linear probe finetuning: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. the parameter efficiently finetuning, to a particular technological environment or field of use, e.g. comprises a linear probe finetuning. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 4
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
adds a new finetuned machine learning model to the plurality of finetuned machine learning models: These additional elements amount to insignificant extra-solution activity in the form of selecting a particular data source or type of data to be manipulated MPEP § 2106.05(g).
and updates the ranking of the plurality of finetuned machine learning models with the new finetuned machine learning model: These additional elements amount to insignificant extra-solution activity in the form of selecting a particular data source or type of data to be manipulated MPEP § 2106.05(g).
Step 2B: The additional elements from Step 2A Prong 2 include insignificant extra-solution activity of data gathering recited by “adds a new finetuned machine learning model to the plurality of finetuned machine learning models” which are well-understood routine and conventional activities similar to receiving or transmitting data over a network per MPEP 2106.05(d)(II), and “and updates the ranking of the plurality of finetuned machine learning models with the new finetuned machine learning model” which are well-understood routine and conventional activities similar to arranging a hierarchy of groups, sorting information, eliminating less restrictive pricing information and determining the price per MPEP 2106.05(d)(II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 5
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the one or more representative datasets comprises different classification labels from the one or more target datasets: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. the one or more representative datasets, to a particular technological environment or field of use, e.g. comprises different classification labels from the one or more target datasets. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 6
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein one or more finetuned machine learning models of the plurality of finetuned machine learning models are previously finetuned on a plurality of datasets: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. one or more finetuned machine learning models of the plurality of finetuned machine learning models, to a particular technological environment or field of use, e.g. are previously finetuned on a plurality of datasets. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claim 7
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas as the judicial exception of claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein one or more finetuned machine learning models of the plurality of finetuned machine learning models are previously finetuned on a plurality of finetuning methods: These additional elements are recited at a high level of generality and merely indicate a field of use or technological environment in which to apply a judicial exception, e.g. one or more finetuned machine learning models of the plurality of finetuned machine learning models, to a particular technological environment or field of use, e.g. are previously finetuned on a plurality of finetuning methods. See MPEP 2106.05(h). Elements that use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to indicate a field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP § 2106.05.
Claims 8 & 10-14
Step 1: These claims are directed to “A computer-implemented method comprising:”; therefore, it is directed the statutory category of a process.
Step 2A Prong 1: Claims 8 & 10-14 recite the same judicial exception as Claims 1-7, respectively.
Step 2A Prong 2: The judicial exception recited in these claims are not integrated into a practical application. The analysis at this step for Claims 8 & 10-14 mirrors that of Claims 1-7, respectively.
Step 2B: The additional elements from Step 2A Prong 2 do not contain significantly more than the judicial exception for these claims. The analysis at this step for Claims 8 & 10-14 mirrors that of Claims 1-7, respectively.
Claim 9
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the abstract ideas of claim 8 as well as, inter alia:
finetuning, by the system, the base model further on the one or more target datasets: These limitations recite mathematical calculations similar to an act of calculating using mathematical methods to determine a variable or number per MPEP 2106.04(a)(2)(I)(C).
Thus, the claim recites a judicial exception.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a
practical application and is not considered to be significantly more. As such, the claim is patent
ineligible.
Claims 15-20
Step 1: This claim recites "A computer program product, comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:"; therefore, it is directed to the statutory category of an article of manufacture.
Step 2A Prong 1: Claims 15-20 recite the same judicial exception as Claims 1-7, respectively.
Step 2A Prong 2: The judicial exception recited in these claims are not integrated into a practical application. The only difference between Claims 15-20 and Claims 1-7, is that Claims 15-20 are directed to "A computer program product, comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a computer program product, comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). With that exception, the analysis at this step for Claims 15-20 mirrors that of Claims 1-7, respectively.
Step 2B: The additional elements from Step 2A Prong 2 do not contain significantly more than the judicial exception for these claims. The only difference between Claims 15-20 and Claims 1-7, is that Claims 15-20 are directed to "A computer program product, comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a computer program product, comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to, cannot amount to significantly more than the judicial exception. See MPEP 2106.05(f). With that exception, the analysis at this step for Claims 15-20 mirrors that of Claims 1-7, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-6, 8, 10-11, 13-17, & 19 are rejected under 35 U.S.C. 103 as being unpatentable over Poth et al. ("What to Pre-Train on? Efficient Intermediate Task Selection", Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, ACL) (Year: 2021), hereafter Poth, in view of Deshpande et al. ("A linearized framework and a new benchmark for model selection for fine-tuning", arXiv:2102.00084v1 [cs.CV] 29 Jan 2021, 14 pages), hereafter Deshpande.
Deshpande was cited in the IDS submitted 07/11/2023.
Regarding independent claim 1, Poth teaches a system comprising:
a memory that stores computer executable components; a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise ([App. B] discusses implementing the system using PyTorch, which necessarily uses a memory to store executable components, and a processor to execute said components);
a ranking component that ranks a plurality of finetuned machine learning models based on performance of the plurality of finetuned machine learning models over one or more representative datasets ([Sec. 4.3] discusses the system ranks the finetuned models based on their respective performances over a dataset and target task);
and a comparison component that finetunes a pretrained machine learning model and one or more candidate models selected based on the ranking of the plurality of finetuned machine learning models on one or more target datasets, compares performance of the one or more candidate models ([Sec. 1, 3.1-4, & Table 1] discusses selecting the top ranked machine learning models, then subsequently fine-tuning the selected models on a target dataset; [Sec. 3.3 & 5.2] discusses comparing the performance of the candidate models against each other).
Poth does not explicitly teach a system that compares performance of the one or more candidate models to a defined performance metric, and selects a base model from the pretrained machine learning model and the one or more candidate models based on the performance of the one or more candidate models over the one or more target datasets.
However, in a similar field of endeavor, Deshpante teaches a system for selection of finetuned models, wherein the performance of candidate models is compared to a defined performance metric and further, a base model is selected from these candidates based on the performance ([Sec. 3.2-4.3 & 5] discusses using LFC scores as a selection gate in which candidate scores are compared against a threshold to determine whether the candidate can be used and further, a base model is selected from these candidate models).
Because Poth teaches a memory that stores computer executable components; a processor that executes the computer executable components stored in the memory, a ranking component that ranks a plurality of finetuned machine learning models based on performance of the plurality of finetuned machine learning models over one or more representative datasets, a comparison component that finetunes a pretrained machine learning model and one or more candidate models selected based on the ranking of the plurality of finetuned machine learning models on one or more target datasets, and compares performance of the one or more candidate models; and Deshpante teaches comparing performance of the one or more candidate models to a defined performance metric, and selecting a base model from the pretrained machine learning model and the one or more candidate models based on the performance of the one or more candidate models over the one or more target datasets, accordingly, it would have been obvious to one of ordinary skill in the art to incorporate comparing performance of the one or more candidate models to a defined performance metric, and selecting a base model from the pretrained machine learning model and the one or more candidate models based on the performance of the one or more candidate models over the one or more target datasets as taught by Deshpante, into Poth’s system, with a reasonable expectation of success, to teach a memory that stores computer executable components; a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise: a ranking component that ranks a plurality of finetuned machine learning models based on performance of the plurality of finetuned machine learning models over one or more representative datasets; and a comparison component that finetunes a pretrained machine learning model and one or more candidate models selected based on the ranking of the plurality of finetuned machine learning models on one or more target datasets, compares performance of the one or more candidate models to a defined performance metric, and selects a base model from the pretrained machine learning model and the one or more candidate models based on the performance of the one or more candidate models over the one or more target datasets. This combination would have been motivated by the desire to select optimal models to fine-tune in few selections, while increasing selection accuracy (Deshpante [Abstract]).
Regarding dependent claim 2, the combination of Poth and Deshpante teaches the claimed invention as claimed in claim 1, including:
wherein the ranking component further: parameter efficiently finetunes the plurality of finetuned machine learning models on the one or more representative datasets (Poth [Abstract & Sec. 1] discusses using parameter efficient adapter settings for the process of finetuning the machine learning models on the target datasets);
and compares the performance of the plurality finetuned machine learning models (Poth [Sec. 3.3, Table 1, & Fig. 2] discusses comparing the results of the performance of the plurality of finetuned models).
Regarding dependent claim 3, the combination of Poth and Deshpante teaches the claimed invention as claimed in claim 2, including wherein the parameter efficiently finetuning comprises a linear probe finetuning (Poth [Sec. 4.3 & 5.1] discusses training and finetuning using linear regression as proxy models, and the process is a form of linear probe finetuning).
Regarding dependent claim 5, the combination of Poth and Deshpante teaches the claimed invention as claimed in claim 2, including wherein the one or more representative datasets comprises different classification labels from the one or more target datasets (Poth [Sec. 3.1 & 6] discusses a task repository, constituting a representative dataset, comprising multiple different tasks and classification labels).
Regarding dependent claim 6, the combination of Poth and Deshpante teaches the claimed invention as claimed in claim 1, including wherein one or more finetuned machine learning models of the plurality of finetuned machine learning models are previously finetuned on a plurality of datasets (Deshpande [Abstract & Sec. 1] discusses one or more of the finetuned machine learning models are multi-domain models; thus, they are previously finetuned on another domain or dataset; Deshpande [Sec. 4.1] discusses these multi-domain models were previously trained on at least 1 of 8 large source datasets).
Regarding claims 8, 10-11, & 13-14, they are method claims that are substantially the same as the system of claims 1-3 & 5-6, respectively. Therefore, claims 8, 10-11, & 13-14 are rejected for the same reasons as claims 1-3 & 5-6, respectively.
Regarding claims 15-17, & 19, they are computer-readable storage medium claims that are substantially the same as the system of claims 1-3 & 6, respectively. Therefore, claims 15-17, & 19 are rejected for the same reasons as claims 1-3 & 6, respectively.
Claims 4, 7, 12, 18, & 20 are rejected under 35 U.S.C. 103 as being unpatentable over Poth et al. ("What to Pre-Train on? Efficient Intermediate Task Selection", Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, ACL) (Year: 2021), hereafter Poth, in view of Deshpande et al. ("A linearized framework and a new benchmark for model selection for fine-tuning”, arXiv:2102.00084v1 [cs.CV] 29 Jan 2021, 14 pages), hereafter Deshpande, as applied in claims 1-2, 8, & 15, and further in view of Pfeiffer et al. ("AdapterHub: A Framework for Adapting Transformers", Proceedings of the 2020 EMNLP, ACL) (Year: 2020), hereafter Pfeiffer.
Regarding dependent claim 4, the combination of Poth and Deshpande teaches the claimed invention as claimed in claim 2, including ranking a plurality of finetuned machine learning models based on performance of the plurality of finetuned machine learning models over one or more representative datasets (Poth [Sec. 4.3] discusses the system ranks the finetuned models based on their respective performances over a dataset and target task).
The combination of Poth and Deshpande does not explicitly teach the ranking component further adds a new finetuned machine learning model to the plurality of finetuned machine learning models and updates the ranking of the plurality of finetuned machine learning models with the new finetuned machine learning model.
However, in a similar field of endeavor, Pfeiffer teaches a system for adding additional finetuned models to a pool of models and regenerating the list of models ([Sec. 3-3.4] discusses a new finetuned model is added to the pool of finetuned models, and the list is dynamically updated).
Because the combination of Poth and Deshpande teaches ranking a plurality of finetuned machine learning models based on performance of the plurality of finetuned machine learning models over one or more representative datasets; and Pfeiffer teaches adding additional finetuned models to a pool of models and regenerating the list of models, accordingly, it would have been obvious to one of ordinary skill in the art to incorporate adding additional finetuned models to a pool of models and regenerating the list of models as taught by Pfeiffer into the combination of Poth and Deshpande’s system, with a reasonable expectation of success, to teach the ranking component further adds a new finetuned machine learning model to the plurality of finetuned machine learning models and updates the ranking of the plurality of finetuned machine learning models with the new finetuned machine learning model. This combination would have been motivated by the desire to improve efficiency and provide access to more pretrained models (Pfeiffer [Sec. 3-4]).
Regarding dependent claim 7, the combination of Poth and Deshpande teaches the claimed invention as claimed in claim 1, including wherein one or more finetuned machine learning models of the plurality of finetuned machine learning models are previously finetuned (Deshpande [Abstract & Sec. 1] discusses one or more of the finetuned machine learning models are multi-domain models; thus, they are previously finetuned on another domain or dataset; Deshpande [Sec. 4.1] discusses these multi-domain models were previously trained on at least 1 of 8 large source datasets).
The combination of Poth and Deshpande does not explicitly teach wherein one or more finetuned machine learning models of the plurality of finetuned machine learning models are previously finetuned on a plurality of finetuning methods.
However, in a similar field of endeavor, Pfeiffer teaches a system for incorporating machine learning models which have been previously finetuned on a plurality of finetuning methods ([Sec. 2 & Table 1] discusses the finetuned machine learning models are previously trained on separate finetuning methods, including Full fine-tuning, Pfeifer, and Houlsby methods).
Because the combination of Poth and Deshpande teaches one or more finetuned machine learning models of the plurality of finetuned machine learning models are previously finetuned; and Pfeiffer teaches incorporating machine learning models which have been previously finetuned on a plurality of finetuning methods, accordingly, it would have been obvious to one of ordinary skill in the art to incorporate machine learning models which have been previously finetuned on a plurality of finetuning methods as taught by Pfeiffer into the combination of Poth and Deshpande’s system, with a reasonable expectation of success, to teach wherein one or more finetuned machine learning models of the plurality of finetuned machine learning models are previously finetuned on a plurality of finetuning methods. This combination would have been motivated by the desire to enable scalable and easy access to sharing task-specific models, particularly in low resource scenarios (Pfeiffer [Abstract]).
Regarding claim 12, claim 12 is a method claim that is substantially the same as the system of claim 4. Therefore, claim 12 is rejected for the same reasons as claim 4.
Regarding claims 18 & 20, they are computer-readable storage medium claims that are substantially the same as the system of claims 4 & 7, respectively. Therefore, claims 18 & 20 are rejected for the same reasons as claims 4 & 7, respectively.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Poth et al. ("What to Pre-Train on? Efficient Intermediate Task Selection", Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, ACL) (Year: 2021), hereafter Poth, in view of Deshpande et al. ("A linearized framework and a new benchmark for model selection for fine-tuning”, arXiv:2102.00084v1 [cs.CV] 29 Jan 2021, 14 pages), hereafter Deshpande, as applied in claim 8, and further in view of Phang et al. ("Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks."arXiv:1811.01088v2 [cs.CL] 27 Feb 2019, 12 pages), hereafter Phang.
Phang was cited in the IDS submitted 07/11/2023.
Regarding dependent claim 9, the combination of Poth and Deshpande teaches the claimed invention as claimed in claim 8, including finetuning, by the system, a pretrained machine learning model and the one or more candidate models on one or more target datasets (Poth [Sec. 1, 3.1-4, & Table 1] discusses selecting the top ranked machine learning models, then subsequently fine-tuning the selected models on a target dataset; [Sec. 3.3 & 5.2] discusses comparing the performance of the candidate models against each other).
The combination of Poth and Deshpande does not explicitly teach finetuning, by the system, the base model further on the one or more target datasets.
However, in a similar field of endeavor, Phang teaches supplemental training for a base model on the one or more target datasets ([Sec. 1, 3, & 5] discusses further fine-tuning the model on the target task dataset, after having already finetuned the model previously).
Because the combination of Poth and Deshpande teaches finetuning a pretrained machine learning model and the one or more candidate models on one or more target datasets; and Phang teaches finetuning the base model further on the one or more target datasets, accordingly, it would have been obvious to one of ordinary skill in the art to incorporate finetuning the base model further on the one or more target datasets as taught by Phang into the combination of Poth and Deshpande’s system, with a reasonable expectation of success, to teach finetuning, by the system, the base model further on the one or more target datasets. This combination would have been motivated by the desire to obtain additional performance improvements (Phang [Abstract]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Vu et al. ("Exploring and Predicting Transferability across NLP Tasks", Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, ACL) (Year: 2020) ([Abstract] In this paper, we conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems (text classification, question answering, and sequence labeling). Our results show that transfer learning is more beneficial than previously thought, especially when target task data is scarce, and can improve performance even with low-data source tasks that differ substantially from the target task (e.g., part-of-speech tagging transfers well to the DROP QA dataset). We also develop task embeddings that can be used to predict the most transferable source tasks for a given target task, and we validate their effectiveness in experiments controlled for source and target data size. Overall, our experiments reveal that factors such as data size, task and domain similarity, and task complexity all play a role in determining transferability).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RILEY S ACOSTA whose telephone number is (571)272-8714. The examiner can normally be reached Monday-Thursday 6am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer N Welch can be reached at (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RILEY S ACOSTA/Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights