DETAILED ACTION
This correspondence is responsive to the application filed on August 22, 2023. Claims 1-25 are pending in the case, with claims 1, 10, 19-20, and 23 in independent form.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Summary of Detailed Action
Claim 24 is objected to regarding informalities.
I. Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
II. Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
III. Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
IV. Claims 2 and 11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
V. Claims 3 and 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
VI. Claims 20-25 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
VII. Claims 20 and 25 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
Claims 1-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1-2, 4-5, 10-11, 13-14, 19-21 and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Bansal et al., in view of Clement et al., and Liu et al. “Automating Feature Subspace Exploration via Multi-Agent Reinforcement Learning,” 2019.
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Bansal in view of Clement and Liu, and further in view of Kishimoto et al.
Claim Objections
Claim 24 is objected to because of the following informalities:
Claim 24, lines 2-3: The period in the middle of the claim must be deleted, as a claim can only have one period at the very end of the claim.
Amend as follows: “a set of training data and the tuning type for the identified pipeline, and one or more hyperparameters as part of the inner search.”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
I. Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 10 and 19 recite a limitation for generating a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type. It is not clear what the elements are or are not. It is further unclear whether “generating a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type” means that the pipelines include elements selected from an agent, a foundation model and a tuning type or that each pipeline has all of the listed elements. Thus, claims 1, 10 and 19 are unclear and indefinite. For examination purposes, claims 1, 10 and 19 are interpreted broadly as generating a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type, such that the plurality of pipelines include an agent, foundation model and tuning type, but each pipeline does not necessarily include all three elements of an agent, a foundation model and a tuning type. In other words, each pipeline of the plurality of pipelines may include only one of an agent, a foundation model element or a tuning type. Applicant may cancel claims 1, 10 and 19 or amend claims 1, 10 and 19 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 2-9 depend directly or indirectly from claim 1 and are rejected for the same reasons discussed above with respect to claim 1.
Claims 11-18 depend directly or indirectly from claim 10 and are rejected for the same reasons discussed above with respect to claim 10.
II. Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 10 and 19 recite in the preamble a method, system, or computer program product for tunning a model. It is not clear what model is tuned because the body of the claims do not mention or refer to the model. The claims recite a foundation model element and tuning elements, but not tuning the model. It is not clear what model is being tuned or how the model is being tuned. Thus, claims 1, 10 and 19 are unclear and indefinite. Applicant may cancel claims 1, 10 and 19 or amend claims 1, 10 and 19 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 2-9 depend directly or indirectly from claim 1 and are rejected for the same reasons discussed above with respect to claim 1.
Claims 11-18 depend directly or indirectly from claim 10 and are rejected for the same reasons discussed above with respect to claim 10.
III. Claims 1-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 1, 10 and 19 recite a limitation for generating a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type. It is not clear how the different “elements” work together in the generated pipelines because a pipeline is generally understood to refer to a well-defined sequence of operations or models chained together. It is not clear how the agent and the foundation model are supposed to interact, or which models is/are tuned. Thus, claims 1, 10 and 19 are unclear and indefinite. Applicant may cancel claims 1, 10 and 19 or amend claims 1, 10 and 19 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 2-9 depend directly or indirectly from claim 1 and are rejected for the same reasons discussed above with respect to claim 1.
Claims 11-18 depend directly or indirectly from claim 10 and are rejected for the same reasons discussed above with respect to claim 10.
IV. Claims 2 and 11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Dependent claims 2 and 11 recite a limitation for wherein the elements of at least one pipeline of the plurality of pipelines further includes a reward model. It is not clear what function the reward model performs or what the reward model is rewarding. It is further unclear how the reward model interacts or works together with the different pipeline “elements” including the agent and the foundation model and the tuning type. Thus, claims 2 and 11 are unclear and indefinite. Applicant may cancel claims 2 and 11 or amend claims 2 and 11 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
V. Claims 3 and 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Dependent claims 3 and 12 recite a limitation for wherein the agent of at least one pipeline of the plurality of pipelines is “a pass-through agent” that “corresponds with supervised tuning” of the foundation model. A “pass-through agent” is not common terminology in the field and it is not clear what a pass-through agent is or is not. It is not clear what a pass-through agent does or does not do. It is yet further unclear what pass-through agent that corresponds with supervised training means, much less how a pass-through agent would correspond with an action of supervised training of the foundation model. Thus, claims 3 and 12 are unclear and indefinite. Applicant may cancel claims 3 and 12 or amend claims 3 and 12 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
VI. Claims 20-25 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 20 and 23 recite a limitation for performing an outer search of “a plurality of pipelines” according to a performance metric defined by an input task, “each having elements that include at least an agent, a foundation model, and a tuning type, and with at least one of the plurality of pipelines additionally having a reward model element.” It is not clear how the different “elements” work together in the pipelines because a pipeline is generally understood to refer to a well-defined sequence of operations or models chained together. It is not clear how the agent and the foundation model are supposed to interact, or which models is/are tuned. It is further unclear how the additional reward model and agent and foundation model are supposed to interact, or which models is/are tuned. Thus, claims 20 and 23 are unclear and indefinite. Applicant may cancel claims 20 and 23 or amend claims 20 and 23 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 21-22 depend directly or indirectly from claim 20 and are rejected for the same reasons discussed above with respect to claim 20.
Claims 24-25 depend directly or indirectly from claim 23 and are rejected for the same reasons discussed above with respect to claim 23.
VII. Claims 20 and 25 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Independent claims 20 and 23 recite in the preamble a method or system, for tunning a model. It is not clear what model is tuned because the body of the claims do not mention or refer to the model. The claims recite a foundation model element and tuning elements, but not tuning the model. It is not clear what model is being tuned or how the model is being tuned. Thus, claims 20 and 23 are unclear and indefinite. Applicant may cancel claims 20 and 23 or amend claims 20 and 23 to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The examiner notes that claims 21-22 depend from claim 20 and clarify which model is tuned. Similarly, claims 24-25 depend from claim 23 and clarify which model is tuned.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) subject matter at a general, high-level of a first method for tuning a model, comprising: generating a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type; setting hyperparameters of elements of the plurality of pipelines in accordance with a task; tuning elements of the plurality of pipelines in accordance with the task; and performing the task using a highest-performance pipeline of the plurality of pipelines, which are which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, the recitation of performing the task using a highest-performance pipeline of the plurality of pipelines are also mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III). This judicial exception is not integrated into a practical application and the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additionally, the claim(s) recite(s) subject matter at a general, high-level of a second method for tuning a model, comprising: performing an outer search of a plurality of pipelines according to a performance metric defined by a task, each having elements that include at least an agent, a foundation model, and a tuning type, and with at least one of the plurality of pipelines additionally having a reward model element, over a space with dimensions defined by the elements of the plurality of pipelines; for each pipeline identified by the outer search, performing an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the input task; and performing the task using a highest-performing tuned pipeline of the identified pipelines according to the performance metric, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III). Additionally, the these recitations are also mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). This judicial exception is not integrated into a practical application and the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 1-25 recite one of the four statutory categories of patent able subject matter and belong to the statutory class(es) of a process (method claims 1, 20 ), a machine (system/apparatus claims 10, 23), and an article of manufacture (non-transitory computer readable media claims 19).
Claim 1 recites a method, thus a process and one of the four statutory categories of patentable subject matter. However, claim 1 further recites for tuning a model, comprising: generating a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type; setting hyperparameters of elements of the plurality of pipelines in accordance with a task; tuning elements of the plurality of pipelines in accordance with the task; and performing the task using a highest-performance pipeline of the plurality of pipelines, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, the recitation of performing the task using a highest-performance pipeline of the plurality of pipelines are also mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
computer-implemented (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
input (An additional element of extra-solution activity that courts have identified is well understood, routine and conventional activity for receiving or transmitting data over a network, e.g., using the internet to gather data. See also, MPEP 2106.05(d)(II), MPEP 2106.05(g), 2019 Guidance, 84 FR 50 at 55, 2019 Guidance, 84 FR 50, footnote 31.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more, and transmitting data over a network is well-understood, routine and conventional (MPEP 2106.05(d), and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Claim 2, dependent on claim 1, recites only additional abstract ideas for wherein the elements of at least one pipeline of the plurality of pipelines further includes a reward model, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, this recited subject matter is also mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
Claim 3, dependent on claim 1, does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein the agent of at least one pipeline of the plurality of pipelines is a pass-through agent that corresponds with supervised tuning of the foundation model (This additional element amounts to merely the words to “apply it” (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer. MPEP 2106.05(f).) Also, this additional element amounts to no more than generally linking the use of the judicial exception to a particular technologic environment or field of use - The application or use of the judicial exception in this manner does not meaningfully limit the claim by going beyond generally linking the use of the judicial exception to a particular technological environment. MPEP 2106.05(h)).
Claim 4, dependent on claim 1, recites only additional abstract ideas for wherein generating the plurality of pipelines includes performing a search over a space, with dimensions of the space defined by the elements of the plurality of pipelines, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III). Additionally, the recitation of a space, with dimensions of the space defined by the elements of the plurality of pipelines, is also mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
Claim 5, dependent on claim 4, recites only additional abstract ideas for wherein performing the search includes varying the elements of the pipelines in accordance with a performance metric of the input task, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
Claim 6, dependent on claim 4, does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein performing the search includes performing a limited discrepancy search over a tree, where the tree includes a set of levels that correspond to respective elements of the plurality of pipelines (This additional element amounts to merely the words to “apply it” (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer. MPEP 2106.05(f).) Also, this additional element amounts to no more than generally linking the use of the judicial exception to a particular technologic environment or field of use - The application or use of the judicial exception in this manner does not meaningfully limit the claim by going beyond generally linking the use of the judicial exception to a particular technological environment. MPEP 2106.05(h)).
Claim 7, dependent on claim 1, recites only additional abstract ideas for wherein generating the plurality of pipelines includes selecting, for each of the plurality of pipelines, a tuning type from a group that includes at least prefix tuning, fine tuning, and fractional tuning, which are mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
Claim 8, dependent on claim 1, does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein generating the plurality of pipelines includes selecting, for each of the plurality of pipelines, an agent from a group that includes at least advantage actor-critic (A2C), proximal policy optimization (PPO), trust region policy optimization (TRPO), and a pass-through agent (This additional element amounts to merely the words to “apply it” (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer. MPEP 2106.05(f).) Also, this additional element amounts to no more than generally linking the use of the judicial exception to a particular technologic environment or field of use - The application or use of the judicial exception in this manner does not meaningfully limit the claim by going beyond generally linking the use of the judicial exception to a particular technological environment. MPEP 2106.05(h)).
Claim 9, dependent on claim 1, does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein generating the plurality of pipelines includes selecting, for each of the plurality of pipelines, a foundation model from a group that includes at least a text-to-text transfer transformer (T5) model, a generative pre-trained transformer (GPT) model, a BigScience Large Open-science Open-access Multilingual Language (BLOOM) model, and a fine-tuned language net (FLAN) model (This additional element amounts to merely the words to “apply it” (or an equivalent) or are mere instructions to implement an abstract idea or other exception on a computer. MPEP 2106.05(f).) Also, this additional element amounts to no more than generally linking the use of the judicial exception to a particular technologic environment or field of use - The application or use of the judicial exception in this manner does not meaningfully limit the claim by going beyond generally linking the use of the judicial exception to a particular technological environment. MPEP 2106.05(h)).
Claim 10 recites a system, thus a machine and one of the four statutory categories of patentable subject matter. However, claim 10 further recites for tuning a model, generate a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type; set hyperparameters of elements of the plurality of pipelines in accordance with a task; tune elements of the plurality of pipelines in accordance with the task; and perform the task using a highest-performance pipeline of the plurality of pipelines, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, the recitation of perform the task using a highest-performance pipeline of the plurality of pipelines are also mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
system, comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
input (An additional element of extra-solution activity that courts have identified is well understood, routine and conventional activity for receiving or transmitting data over a network, e.g., using the internet to gather data. See also, MPEP 2106.05(d)(II), MPEP 2106.05(g), 2019 Guidance, 84 FR 50 at 55, 2019 Guidance, 84 FR 50, footnote 31.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more, and transmitting data over a network is well-understood, routine and conventional (MPEP 2106.05(d), and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Dependent claims 11-18 are comparably rejected as set forth above with respect to dependent claims 2-9.
Claim 19 recites a computer program product, thus an article of manufacture and one of the four statutory categories of patentable subject matter. However, claim 19 further recites for tuning a model, to generate a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type; set hyperparameters of elements of the plurality of pipelines in accordance with a task; tune elements of the plurality of pipelines in accordance with the task; and perform the task using a highest-performance pipeline of the plurality of pipelines, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, the recitation of perform the task using a highest-performance pipeline of the plurality of pipelines are also mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being readable by a hardware processor to cause the hardware processor to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
input (An additional element of extra-solution activity that courts have identified is well understood, routine and conventional activity for receiving or transmitting data over a network, e.g., using the internet to gather data. See also, MPEP 2106.05(d)(II), MPEP 2106.05(g), 2019 Guidance, 84 FR 50 at 55, 2019 Guidance, 84 FR 50, footnote 31.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more, and transmitting data over a network is well-understood, routine and conventional (MPEP 2106.05(d), and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Claim 20 recites a method, thus a process and one of the four statutory categories of patentable subject matter. However, claim 20 further recites for tuning a model, comprising: performing an outer search of a plurality of pipelines according to a performance metric defined by a task, each having elements that include at least an agent, a foundation model, and a tuning type, and with at least one of the plurality of pipelines additionally having a reward model element, over a space with dimensions defined by the elements of the plurality of pipelines; for each pipeline identified by the outer search, performing an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the task; and performing the task using a highest-performing tuned pipeline of the identified pipelines according to the performance metric, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, the recitations of claim 20 for performing an outer search of a plurality of pipelines according to a performance metric defined by a task, each having elements that include at least an agent, a foundation model, and a tuning type, and with at least one of the plurality of pipelines additionally having a reward model element, over a space with dimensions defined by the elements of the plurality of pipelines; for each pipeline identified by the outer search, performing an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the task; and performing the task using a highest-performing tuned pipeline of the identified pipelines according to the performance metric are also mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
computer-implemented (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
input (An additional element of extra-solution activity that courts have identified is well understood, routine and conventional activity for receiving or transmitting data over a network, e.g., using the internet to gather data. See also, MPEP 2106.05(d)(II), MPEP 2106.05(g), 2019 Guidance, 84 FR 50 at 55, 2019 Guidance, 84 FR 50, footnote 31.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more, and transmitting data over a network is well-understood, routine and conventional (MPEP 2106.05(d), and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Claim 21, dependent on claim 20, recites only additional abstract ideas for wherein the inner search comprises tuning the foundation model for the identified pipeline according to a set of training data, the tuning type for the identified pipeline, and according to one or more hyperparameters, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, the recitation of
Claim 22, dependent on claim 20, recites only additional abstract ideas for wherein the inner search comprises tuning the agent for the identified pipeline according to a set of training data and a reward model that guides the agent’s behavior, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
Claim 23 recites a system, thus a machine and one of the four statutory categories of patentable subject matter. However, claim 23 further recites for tuning a model, perform an outer search of a plurality of pipelines according to a performance metric defined by a task, each having elements that include at least an agent, a foundation model, and a tuning type, and with at least one of the plurality of pipelines additionally having a reward model element, over a space with dimensions defined by the elements of the plurality of pipelines; for each pipeline identified by the outer search, perform an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the task; and perform the task using a highest-performing tuned pipeline of the identified pipelines according to the performance metric, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I). Additionally, the recitations of claim 23 for tuning a model, perform an outer search of a plurality of pipelines according to a performance metric defined by a task, each having elements that include at least an agent, a foundation model, and a tuning type, and with at least one of the plurality of pipelines additionally having a reward model element, over a space with dimensions defined by the elements of the plurality of pipelines; for each pipeline identified by the outer search, perform an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the task; and perform the task using a highest-performing tuned pipeline of the identified pipelines according to the performance metric are also mental processes or concepts that can be performed in the human mind, including observation, evaluation, judgment or opinion, or by a human using pen and paper. MPEP 210604(a)(2)(III).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
A system, comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
input (An additional element of extra-solution activity that courts have identified is well understood, routine and conventional activity for receiving or transmitting data over a network, e.g., using the internet to gather data. See also, MPEP 2106.05(d)(II), MPEP 2106.05(g), 2019 Guidance, 84 FR 50 at 55, 2019 Guidance, 84 FR 50, footnote 31.).
Thus, the claim is directed to the abstract idea.
Further, the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself, because implementation on a computer (MPEP 2106.05(f)) cannot provide significantly more, and transmitting data over a network is well-understood, routine and conventional (MPEP 2106.05(d), and the combination of additional elements does not provide an inventive concept. Thus, the claim is ineligible.
Claim 24, dependent on claim 23, recites additional abstract ideas to tune the foundation model for the identified pipeline according to a set of training data. and the tuning type for the identified pipeline, and one or more hyperparameters as part of the inner search, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein computer program further causes the hardware processor to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
Claim 25, dependent on claim 23, recites additional abstract ideas to tune the agent for the identified pipeline according to a set of training data and a reward model that guides the agent’s behavior as part of the inner search, which are mathematical concepts including mathematical relationships, mathematical formulas or equations, and mathematical calculations. MPEP 210604(a)(2)(I).
The claim does not include any additional elements which integrate the abstract idea into a practical application since the additional elements consist of:
wherein the computer program further causes the hardware processor to (an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See also, MPEP 2106.05(f), MPEP 2106.04(d), 2019 Guidance, 84 FR 50 at 55, footnote 30.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-5, 10-11, 13-14, 19-21 and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Bansal et al. (Pub. No. US 2021/0097444 A1, published April 1, 2021) hereinafter Bansal in view of Clement et al. (Pub. No. US 2022/0398462 A1, published December 15, 2022) hereinafter Clement, and Liu et al. “Automating Feature Subspace Exploration via Multi-Agent Reinforcement Learning,” 2019, hereinafter Liu.
Regarding claim 1, Banal teaches:
A computer-implemented method for tuning a model (i.e., In some embodiments, the AMPGS (A computer-implemented method) thus provides a “white box” approach by showing users the incremental steps or jobs that were executed to arrive at an ultimate ML model produced for inference purposes, which may also enable users to modify and repeat the process to iteratively fine tune ML pipelines (for tuning a model (tuning a pipeline ML model)) to their specific needs. Bansal, Fig 1, 8, 13, para 18, 19.), comprising:
generating a plurality of pipelines, with elements that include at least an agent, a foundation model, and a tuning type;
Bansal teaches that, In some embodiments, users may utilize both the pipeline recommendation system 112 (to generate candidate ML pipelines to explore) as well as the pipeline optimizer system 116 (to explore and evaluate the candidate ML pipelines (generating a plurality of pipelines, with elements). Bansal, Figs 1-10, para 45, 50, 18-19. In some cases, an ensemble model can provide improved accuracy, e.g., via applying an ensemble approach that combines several base models (with elements that include at least a ) to produce one optimal predictive model, … or other approach known to those of skill in the art. Bansal, Figs 1-10, para 39, 85. The pipeline recommender system 112 can also recommend the hardware (compute instance type and count), identifiers of which hyperparameters to tune (and optionally their effective search space) ((with elements that include at least a tuning type (hyperparameters tuning type)), and the degree of parallelization that the pipeline optimizer system 116 should use when exploring the ML pipelines. Bansal, Figs 1-10, para 45, 50.
Thus, Bansal teaches generating a plurality of pipelines, with elements that include at least a base model and a tuning type. Bansal does not specifically disclose elements including an agent and a foundation model.
However, Clement teaches in the field related to automated tuning and deployment of pre-trained deep learning models. Clement, abstract, para 1-4. Clement, which is analogous to the claimed invention because Clement is directed to fine-tuning pre-trained deep learning models, teaches that, A cloud platform is disclosed that provides web services to enable the automated tuning, deployment, and execution of pre-trained deep learning models customized for software engineering tasks. The cloud platform offers several deep learning models, having been trained on large unsupervised datasets of natural language text and source code, for use by users (e.g., developers, customers, clients, researchers, etc.), to fine-tune for a particular downstream related task. The reuse of the pre-trained models with developed weights and biases for source code is a good starting point to develop different models for various software engineering tasks faster and with less computational cost and resources. The cloud platform builds a fine-tuning infrastructure (pipeline) to fine-tune a pre-trained deep learning model (pipeline elements including a foundation model), a deployment infrastructure that deploys the fine-tuned deep learning model, and a model endpoint to facilitate execution of the model, without requiring the developer to specify the infrastructure configurations. Clement, Fig. 1, para 20, 21-24.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of generating a plurality of pipelines and model tuning of Bansal using the pipeline elements including a foundation model element of Clement, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise. Clement, para 2-5. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks.
Thus, Bansal in view of Clement teaches generating a plurality of pipelines, with elements that include at least a foundation model and a tuning type. Bansal in view of Clement does not specifically disclose pipeline element includes an agent.
However, Liu teaches in the field relate to automated exploration and multi-agent reinforcement learning. Liu, which is analogous to the claimed invention because Liu is directed to searching strategy and pipeline reinforcement agents with reward schemes, teaches that, We assign an agent to each feature, the actions of these feature agents are to select or deselect their corresponding features, and the state of environment is characteristics of the selected feature subspace. Liu, page 208, left column, paragraph 2, lines 3-6. We apply our method to different predictors in order to investigate whether our explored feature subset are consistently stable and can consistently outperform other baseline methods on various predictors. Liu, page 213, left column, paragraph 2, lines 2-5. Table 1 and Figure 1 teaches each of the selected and unselected feature/feature sets as separate machine learning pipelines coupled to respective reinforcement learning agents, as the separate features/feature sets form pipelines with the predictors listed in Table 1.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of generating a plurality of pipelines and model tuning of Bansal using the pipeline elements including a foundation model element of Clement and the pipeline elements including an agent of Liu, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise and to provide a more reasonable reward scheme by improved coordination between different features. Clement, para 2-5. Liu, page 207, abstract, lines 18-20. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks and improving generated pipelines.
setting hyperparameters of elements of the plurality of pipelines in accordance with an input task (i.e., As discussed above, Bansal in view of Clement and Liu teaches generating a plurality of pipelines, with elements that include at least an agent, a foundation model and a tuning type. Bansal teaches that, [0067] The interactive code exploration user interface 800 may also include a section 815 to display results of the pipelines being run, and finally, a code section 820 to define different combinations of machine learning models and pipelines, each including values for a name, an ML algorithm to use, a set of hyperparameters (setting hyperparameters of elements of the plurality of pipelines in accordance with an input task (input task target column, para 18, input task target column for prediction output, para 32)), an identifier of a storage location storing a particular set of input values generated by one of the feature processing pipelines, and the like. Bansal, Figs 1-10, para 67, 18-19, 32, 50, 44.);
tuning elements of the plurality of pipelines in accordance with the input task (i.e., As discussed above, Bansal in view of Clement and Liu teaches generating a plurality of pipelines, with elements that include at least an agent, a foundation model and a tuning type. Bansal teaches that, The pipeline recommender system 112, in some embodiments, is responsible for determining the set of ML pipelines to explore and be optimized by the pipeline optimizer system 116. The pipeline recommender system 112, in some embodiments, can analyze the user's provided dataset and infer one or more of the probabilistic schema of the data set, target leakage, feature importance, the type of ML problem (e.g., classification, regression, etc.) based on the user-identified target column (input task, see also para 18, 32), etc. The pipeline recommender system 112 can use the dataset provided by the user and knowledge learned from metadata collected to recommend a promising and diverse set of feature processing pipelines to apply to the customer dataset along with the code for the feature processing model. The pipeline recommender system 112 can also recommend the hardware (compute instance type and count), identifiers of which hyperparameters to tune (and optionally their effective search space) (tuning elements of the plurality of pipelines in accordance with the input task), and the degree of parallelization that the pipeline optimizer system 116 should use when exploring the ML pipelines. Thereafter, the pipeline optimizer system 116 can use the pipelines recommended by pipeline recommender system 112 to start an optimization job, which typically involves running multiple training jobs to identify the most performant ones. As the optimization job progresses, the pipeline optimizer system 116 can discard the low-performing models and can tune the hyperparameters of the most performant ones. Bansal, Figs 1-10, para 44, 50, 56, 67, 18-19, 32.
As one example, the pipeline recommender system 112 may recommend up to ten pipelines to explore, such as (1) “apply one-hot encoding and principal component analysis (as the feature preprocessors/transforms) followed by use of the ‘XGBOOST’ algorithm with hyperparameter tuning” (tuning elements of the plurality of pipelines in accordance with the input task (input task, para 18, 32, 44)), (2) “apply one-hot encoding and principal component analysis (as the feature preprocessors/transforms) followed by use of a ‘linear learner’ algorithm with hyperparameter tuning” (tuning elements of the plurality of pipelines in accordance with the input task (input task, para 18, 32, 44), (3) “apply principal component analysis (as the feature preprocessor/transform) followed by use of the ‘XGBOOST’ algorithm with hyperparameter tuning” (tuning elements of the plurality of pipelines in accordance with the input task (input task, para 18, 32, 44), and the like. Bansal, para 50, 44, 56, 67, 18-19, 32. Such hyperparameter tuning systems and techniques are known to those of skill in the art and can be utilized to work to find better and better pipelines. Figs 1-10, Bansal, para 56, 50, 44, 56, 67, 18-19, 32.); and
performing the input task using a highest-performance pipeline of the plurality of pipelines (i.e., As discussed above, Bansal in view of Clement and Liu teaches generating a plurality of pipelines, with elements that include at least an agent, a foundation model and a tuning type. Bansal teaches that, An automated ML pipeline generation system allows users to easily construct optimized ML pipelines by providing a dataset, identifying a target column in the dataset, and providing an exploration budget. Multiple candidate ML pipelines can be identified and evaluated through an exploration process, and a best ML pipeline can be provided to the requesting user or deployed for production inference (performing the input task (input task, target column prediction output, para 18, 32, 44) using a highest-performance (best evaluated, highest performing) pipeline of the plurality of pipelines). Bansal, abstract, para 19, 18, 32, 44. For example, users in some embodiments may provide a tabular dataset and identify a target column in the dataset to predict, and the AMPGS system then automatically explores ML pipeline solutions with different combinations of data preprocessors, algorithms, and/or algorithm parameter settings to find a “best” model. In some embodiments, users may then directly deploy this best model (in terms of a ML pipeline) to a production environment (e.g., with just one click) (performing the input task (input task, target column prediction output, para 18, 32, 44) using a highest-performance (best evaluated, highest performing) pipeline of the plurality of pipelines) or iterate on the recommended solution(s) to further improve the model quality. Bansal, abstract, Figs 1-10, para 19, 18, 32, 44, 50, 56.).
Regarding claim 2, which depends from claim 1 and recites:
wherein the elements of at least one pipeline of the plurality of pipelines further includes a reward model.
Bansal in view of Clement and Liu teaches the computer-implemented method of claim 1 from which claim 2 depends, including the elements of at least one pipeline of the plurality of pipelines. Bansal does not specifically disclose elements of at least one pipeline further includes a reward model
However, Liu teaches in the field relate to automated exploration and multi-agent reinforcement learning. Liu, which is analogous to the claimed invention because Liu is directed to searching strategy and pipeline reinforcement agents with reward schemes, teaches that, We assign an agent to each feature, the actions of these feature agents are to select or deselect their corresponding features, and the state of environment is characteristics of the selected feature subspace. Liu, page 208, left column, paragraph 2, lines 3-6. We apply our method to different predictors in order to investigate whether our explored feature subset are consistently stable and can consistently outperform other baseline methods on various predictors. Liu, page 213, left column, paragraph 2, lines 2-5. Liu teaches that We develop a strategy to allocate the overall reward to each feature agent. Liu, Figures 1-2, last paragraph right column of page 208 to second paragraph left column of page 209. Table 1 and Figures 1 and 2 teaches each of the selected and unselected feature/feature sets as separate machine learning pipelines coupled to respective reinforcement learning agents and reward assignment models, as the separate features/feature sets form pipelines with the predictors listed in Table 1.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of generating a plurality of pipelines and model tuning of Bansal using the pipeline elements including a foundation model element of Clement and the pipeline elements including an agent and reward model of Liu, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise and to provide a more reasonable reward scheme by improved coordination between different features. Clement, para 2-5. Liu, page 207, abstract, lines 18-20. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks and improving generated pipelines.
Regarding claim 4, which depends from claim 1 and recites:
wherein generating the plurality of pipelines includes performing a search over a space, with dimensions of the space defined by the elements of the plurality of pipelines.
Bansal in view of Clement and Liu teaches the computer-implemented method of claim 1 from which claim 4 depends, including generating the plurality of pipelines and the elements of the plurality of pipelines. Bansal teaches that, The pipeline recommender system 112, in some embodiments, is responsible for determining the set of ML pipelines to explore and be optimized by the pipeline optimizer system 116. … The pipeline recommender system 112 can also recommend the hardware (compute instance type and count), identifiers of which hyperparameters to tune (and optionally their effective search space), and the degree of parallelization that the pipeline optimizer system 116 should use when exploring the ML pipelines (generating the plurality of pipelines includes performing a search over a space, with dimensions of the space defined by the elements of the plurality of pipelines (effective search space defined by the elements of the plurality of pipelines)). Bansal, Figs 1-10, para 44, 45, 18-19.
As similarly discussed above, Bansal teaches the plurality of pipelines, with elements that include at least a base model and a tuning type. Bansal does not specifically disclose elements including an agent and a foundation model. However, as similarly discussed above, Clement teaches the foundation model element and Liu teaches an agent element.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of generating a plurality of pipelines and model tuning of Bansal using the pipeline elements including a foundation model element of Clement and the pipeline elements including an agent of Liu, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise and to provide a more reasonable reward scheme by improved coordination between different features. Clement, para 2-5. Liu, page 207, abstract, lines 18-20. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks and improving generated pipelines.
Regarding claim 5, which depends from claim 4 and recites:
wherein performing the search includes varying the elements of the pipelines in accordance with a performance metric of the input task.
Bansal in view of Clement and Liu teaches the computer-implemented method of claim 4 from which claim 5 depends, including performing the search, the elements of the pipelines, and the input task. Bansal teaches that, [0026] As described herein, in some embodiments the AMPGS 102 makes the ML model building process easier and faster. Instead of requiring users 109 to decide which ML algorithm to use, the AMPGS 102 can automatically select multiple different ML algorithms from a list of high performing algorithms it natively supports and evaluate some or all of them. The AMPGS 102 can also automatically evaluate different hyperparameter settings for those algorithms in an effort to increase the resulting quality of ML model found (wherein performing the search (search, exploration, evaluation, building process, para 44, 45, 18-19) includes varying the elements (varying models, hyperparameter elements) of the pipelines in accordance with a performance metric (quality performance metric) of the input task (target prediction input task, para 18, 19, 36, 44)). In some embodiments, users 109 do not need to be concerned with data cleaning or preprocessing either, as the AMPGS 102 can automatically apply different types of data preprocessors on the data before passing it through the ML algorithms to train ML models. The AMPGS 102 in some embodiments also makes details and artifacts of the ML pipelines it has evaluated, such as the corresponding source code, fully accessible to users, allowing advanced users (such as data scientists) to quickly run baselines and iterate on the results to further improve model quality. Bansal, Figs 1-10, para 26, 37, 40, 44-45, 48.
As similarly discussed above, Bansal teaches the plurality of pipelines, with elements that include at least a base model and a tuning type. Bansal does not specifically disclose elements including an agent and a foundation model. However, as similarly discussed above, Clement teaches the foundation model element and Liu teaches an agent element.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of generating a plurality of pipelines and model tuning of Bansal using the pipeline elements including a foundation model element of Clement and the pipeline elements including an agent of Liu, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise and to provide a more reasonable reward scheme by improved coordination between different features. Clement, para 2-5. Liu, page 207, abstract, lines 18-20. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks and improving generated pipelines.
Claims 10-11 and 13-14 recite systems that parallel the computer-implemented methods of claims 1-2 and 4-5. Therefore, the analysis discussed above with respect to claims 1-2 and 4-5 also applies to claims 10-11 and 13-4, respectively. Accordingly, claims 10-11 and 13-14 are rejected based on substantially the same rationale as set forth above with respect to claims 1-2 and 4, respectively. More specifically regarding A system, comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to (i.e., Bansal, Fig. 13, para 144-149).
Claim 19 recites a computer program product that parallels the computer-implemented method of claim 1. Therefore, the analysis discussed above with respect to claim 1 also applies to claim 19. Accordingly, claim 19 is rejected based on substantially the same rationale as set forth above with respect to claim 1. More specifically regarding A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being readable by a hardware processor to cause the hardware processor to (i.e., Bansal, Fig. 13, para 150, 153-156).
Regarding claim 20, Bansal teaches:
A computer-implemented method for tuning a model (i.e., In some embodiments, the AMPGS (A computer-implemented method) thus provides a “white box” approach by showing users the incremental steps or jobs that were executed to arrive at an ultimate ML model produced for inference purposes, which may also enable users to modify and repeat the process to iteratively fine tune ML pipelines (for tuning a model (tuning a pipeline ML model)) to their specific needs. Bansal, Fig 1, 8, 13, para 18, 19.), comprising:
performing an outer search of a plurality of pipelines according to a performance metric defined by an input task, each having elements that include at least an agent, a foundation model, and a tuning type, and with at least one of the plurality of pipelines additionally having a reward model element, over a space with dimensions defined by the elements of the plurality of pipelines;
Bansal teaches that, The pipeline recommender system 112, in some embodiments, is responsible for determining the set of ML pipelines to explore and be optimized by the pipeline optimizer system 116 (performing an outer search of a plurality of pipelines (outer search exploration of ML pipelines)). The pipeline recommender system 112, in some embodiments, can analyze the user's provided dataset and infer one or more of the probabilistic schema of the data set, target leakage, feature importance, the type of ML problem (e.g., classification, regression, etc.) based on the user-identified target column, etc. (performing an outer search of a plurality of pipelines according to a performance metric defined by an input task (most performative metric (see also performance objective metric, para 36-37,44) defined by an input task, target column prediction output, para 18, 19, 32, 44)) The pipeline recommender system 112 can use the dataset provided by the user and knowledge learned from metadata collected to recommend a promising and diverse set of feature processing pipelines to apply to the customer dataset along with the code for the feature processing model. The pipeline recommender system 112 can also recommend the hardware (compute instance type and count), identifiers of which hyperparameters to tune (and optionally their effective search space), and the degree of parallelization that the pipeline optimizer system 116 should use when exploring the ML pipelines (performing an outer search of a plurality of pipelines according to a performance metric defined by an input task (most performative metric (see also performance objective metric, para 36-37,44) defined by an input task, target column prediction output, para 18, 19, 32, 44), each having elements that include at least , over a space with dimensions defined by the elements of the plurality of pipelines). Thereafter, the pipeline optimizer system 116 can use the pipelines recommended by pipeline recommender system 112 to start an optimization job, which typically involves running multiple training jobs to identify the most performant ones. As the optimization job progresses, the pipeline optimizer system 116 can discard the low-performing models and can tune the hyperparameters of the most performant ones. Bansal, Figs 1-10, para 44-45, 50, 56, 67, 36-37, 18-19, 32.
In some embodiments, users may utilize both the pipeline recommendation system 112 (to generate candidate ML pipelines to explore) as well as the pipeline optimizer system 116 (to explore and evaluate the candidate ML pipelines (performing an outer search of a plurality of pipelines according to a performance metric defined by an input task (most performative metric (see also performance objective metric, para 36-37, 44) defined by an input task, target column prediction output, para 18, 32, 44), each having elements that include at least . Bansal, para 45, 50, 18-19. In some cases, an ensemble model can provide improved accuracy, e.g., via applying an ensemble approach that combines several base models (with elements that include at least a ) to produce one optimal predictive model, … or other approach known to those of skill in the art. Bansal, Figs 1-10, para 39, 85, 44-45, 50, 56, 67, 36-37, 18-19, 32.
Thus, Bansal teaches performing an outer search of a plurality of pipelines according to a performance metric defined by an input task, each having elements that include at least a base model and a tuning type, over a space with dimensions defined by the elements of the plurality of pipelines. Bansal does not specifically disclose elements including an agent and a foundation model and with at least one of the plurality of pipelines additionally having a reward model element.
However, Clement teaches in the field related to automated tuning and deployment of pre-trained deep learning models. Clement, abstract, para 1-4. Clement, which is analogous to the claimed invention because Clement is directed to fine-tuning pre-trained deep learning models, teaches that, A cloud platform is disclosed that provides web services to enable the automated tuning, deployment, and execution of pre-trained deep learning models customized for software engineering tasks. The cloud platform offers several deep learning models, having been trained on large unsupervised datasets of natural language text and source code, for use by users (e.g., developers, customers, clients, researchers, etc.), to fine-tune for a particular downstream related task. The reuse of the pre-trained models with developed weights and biases for source code is a good starting point to develop different models for various software engineering tasks faster and with less computational cost and resources. The cloud platform builds a fine-tuning infrastructure (pipeline) to fine-tune a pre-trained deep learning model (pipeline elements including a foundation model), a deployment infrastructure that deploys the fine-tuned deep learning model, and a model endpoint to facilitate execution of the model, without requiring the developer to specify the infrastructure configurations. Clement, Fig. 1, para 20, 21-24.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of model tuning and performing an outer search of a plurality of pipelines of Bansal using the pipeline elements including a foundation model element of Clement, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise. Clement, para 2-5. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks.
Thus, Bansal in view of Clement teaches Thus, Bansal teaches performing an outer search of a plurality of pipelines according to a performance metric defined by an input task, each having elements that include at least a foundation model and a tuning type, over a space with dimensions defined by the elements of the plurality of pipelines. Bansal in view of Clement does not specifically disclose pipeline element includes an agent and with at least one of the plurality of pipelines additionally having a reward model element.
However, Liu teaches in the field relate to automated exploration and multi-agent reinforcement learning. Liu, which is analogous to the claimed invention because Liu is directed to searching strategy and pipeline reinforcement agents with reward schemes, teaches that, We assign an agent to each feature, the actions of these feature agents are to select or deselect their corresponding features, and the state of environment is characteristics of the selected feature subspace. Liu, page 208, left column, paragraph 2, lines 3-6. We apply our method to different predictors in order to investigate whether our explored feature subset are consistently stable and can consistently outperform other baseline methods on various predictors. Liu, page 213, left column, paragraph 2, lines 2-5. Liu teaches that We develop a strategy to allocate the overall reward to each feature agent. Liu, Figures 1-2, last paragraph right column of page 208 to second paragraph left column of page 209. Table 1 and Figures 1 and 2 teaches each of the selected and unselected feature/feature sets as separate machine learning pipelines coupled to respective reinforcement learning agents and reward assignment models, as the separate features/feature sets form pipelines with the predictors listed in Table 1.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of model tuning and performing an outer search of a plurality of pipelines of Bansal using the pipeline elements including a foundation model element of Clement and the pipeline elements including an agent and reward model of Liu, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise and to provide a more reasonable reward scheme by improved coordination between different features. Clement, para 2-5. Liu, page 207, abstract, lines 18-20. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks and improving pipelines.
for each pipeline identified by the outer search, performing an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the input task (i.e., As discussed above, Bansal in view of Clement and Liu teaches pipelines recommended and identified by the outer search, each with elements that include at least an agent, a foundation model and a tuning type and at least one additionally having a reward model. Bansal teaches that, In some embodiments, users are thus enabled to quickly build ML models—e.g., classification and regression models—without any substantial ML knowledge. For example, users in some embodiments may provide a tabular dataset and identify a target column in the dataset to predict, and the AMPGS system then automatically explores ML pipeline solutions with different combinations of data preprocessors, algorithms, and/or algorithm parameter settings to find a “best” model (for each pipeline identified by the outer search (for each recommended identified pipeline identified by the outer search exploration, see para 44, 45), performing an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the input task (inner search for parameters corresponding to pipeline element algorithm of the recommended and identified pipeline in accordance with finding the “best” model, most performative performance metric to optimize the identified pipeline for the input target column prediction output task, see para 18, 19, 32, 44)). In some embodiments, users may then directly deploy this best model (in terms of a ML pipeline) to a production environment (e.g., with just one click) or iterate on the recommended solution(s) to further improve the model quality. Bansal, Figs 1-10, para 19, 18, 44-45, 50, 56.
The pipeline recommender system 112, in some embodiments, is responsible for determining the set of ML pipelines to explore and be optimized (for each pipeline identified by the outer search (for each recommended identified pipeline identified by the outer search exploration), performing an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric to optimize the identified pipeline for the input task (inner search for parameters corresponding to pipeline elements of the recommended and identified pipeline in accordance with finding the “best” model, most performative performance metric to optimize for the input task, target column prediction output, see para 18, 19, 32, 44))) by the pipeline optimizer system 116. The pipeline recommender system 112, in some embodiments, can analyze the user's provided dataset and infer one or more of the probabilistic schema of the data set, target leakage, feature importance, the type of ML problem (e.g., classification, regression, etc.) based on the user-identified target column, etc. The pipeline recommender system 112 can use the dataset provided by the user and knowledge learned from metadata collected to recommend a promising and diverse set of feature processing pipelines to apply to the customer dataset along with the code for the feature processing model. The pipeline recommender system 112 can also recommend the hardware (compute instance type and count), identifiers of which hyperparameters to tune (and optionally their effective search space), and the degree of parallelization that the pipeline optimizer system 116 should use when exploring the ML pipelines. Thereafter, the pipeline optimizer system 116 can use the pipelines recommended by pipeline recommender system 112 to start an optimization job, which typically involves running multiple training jobs to identify the most performant ones (for each pipeline identified (identified recommended pipelines) by the outer search, performing an inner search for parameters corresponding to the elements of the identified pipeline in accordance with the performance metric (most performative metric, see also performance objective metric, para 36-37,44, explore, search and identify algorithm parameter settings to find the “best” model pipeline element, para 19) to optimize the identified pipeline for the input task). As the optimization job progresses, the pipeline optimizer system 116 can discard the low-performing models and can tune the hyperparameters of the most performant ones. Bansal, Figs 1-10, para 44, 45-46, 19, 18, 50, 56.); and
performing the input task using a highest-performing tuned pipeline of the identified pipelines according to the performance metric (i.e., In some embodiments, users are thus enabled to quickly build ML models—e.g., classification and regression models—without any substantial ML knowledge. For example, users in some embodiments may provide a tabular dataset and identify a target column in the dataset to predict (input task), and the AMPGS system then automatically explores ML pipeline solutions with different combinations of data preprocessors, algorithms, and/or algorithm parameter settings to find a “best” model. In some embodiments, users may then directly deploy this best model (in terms of a ML pipeline) to a production environment (performing the input task using a highest-performing tuned pipeline of the identified pipelines according to the performance metric (deploy best highest performing tuned and optimized pipeline model in production environment to perform the input task)) (e.g., with just one click) or iterate on the recommended solution(s) to further improve the model quality. Bansal, Figs 1-10, para 19, 18-20,46. For example, the operations of the AMPGS 102 (and the pipeline recommender system 112 and/or pipeline optimizer system 116) can be implemented as shown with regard to FIG. 5, which is a diagram illustrating an exemplary set of processing jobs for automated machine learning pipeline exploration and deployment according to some embodiments. Bansal, Figs 1-10, para 46, 18-20.).
Regarding claim 21, which depends from claim 20 and recites:
wherein the inner search comprises tuning the foundation model for the identified pipeline according to a set of training data, the tuning type for the identified pipeline, and according to one or more hyperparameters.
Bansal in view of Clement and Liu teaches the computer-implemented method of claim 20 from which claim 21 depends, including the inner search, the foundation model for the identified pipeline, the tuning type for the identified pipeline. Bansal teaches that, The user interface 200 also allows provides a user interface element 215 the user to identify a dataset to be used for model training purposes, such as by selecting a file or storage location (e.g., from a set of storage objects associated with the user's account within a storage service), providing a resource identifier (e.g., a Uniform Resource Locator (URL)), directly uploading a dataset, etc. Bansal, Figs 1-10, para 31. The pipeline recommender system 112, in some embodiments, is responsible for determining the set of ML pipelines to explore and be optimized by the pipeline optimizer system 116. The pipeline recommender system 112, in some embodiments, can analyze the user's provided dataset (according to a set of training data, see also para 31) and infer one or more of the probabilistic schema of the data set, target leakage, feature importance, the type of ML problem (e.g., classification, regression, etc.) based on the user-identified target column, etc. The pipeline recommender system 112 can use the dataset provided by the user (according to a set of training data, see also para 31) ,and knowledge learned from metadata collected to recommend a promising and diverse set of feature processing pipelines to apply to the customer dataset along with the code for the feature processing model. The pipeline recommender system 112 can also recommend the hardware (compute instance type and count), identifiers of which hyperparameters to tune (inner search comprises tuning the ) (and optionally their effective search space), and the degree of parallelization that the pipeline optimizer system 116 should use when exploring the ML pipelines. Thereafter, the pipeline optimizer system 116 can use the pipelines recommended by pipeline recommender system 112 to start an optimization job, which typically involves running multiple training jobs to identify the most performant ones. As the optimization job progresses, the pipeline optimizer system 116 can discard the low-performing models and can tune the hyperparameters of the most performant ones (inner search comprises tuning the ). Bansal, Figs 1-10, para 44, 50, 56. As one example, the pipeline recommender system 112 may recommend up to ten pipelines to explore, such as (1) “apply one-hot encoding and principal component analysis (as the feature preprocessors/transforms) followed by use of the ‘XGBOOST’ algorithm with hyperparameter tuning”, (2) “apply one-hot encoding and principal component analysis (as the feature preprocessors/transforms) followed by use of a ‘linear learner’ algorithm with hyperparameter tuning”, (3) “apply principal component analysis (as the feature preprocessor/transform) followed by use of the ‘XGBOOST’ algorithm with hyperparameter tuning”, and the like. Bansal, Figs 1-10, para 50, 44, 56.
As similarly discussed above, Bansal teaches the model but does not specifically disclose the foundation model. However, as discussed above, Clement teaches the foundation model pipeline element. Clement, Fig. 1, para 20, 21-224.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of model tuning and performing an outer search of a plurality of pipelines of Bansal using the pipeline elements including a foundation model element of Clement, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise. Clement, para 2-5. This would have provided the advantages of easier and faster pipeline model tuning for desired tasks.
Claims 23-24 recite systems that parallel the computer-implemented methods of claim 20-21. Therefore, the analysis discussed above with respect to claims 20-21 also applies to claims 23-24. Accordingly, claims 23-24 are rejected based on substantially the same rationale as set forth above with respect to claims 20-21. More specifically regarding A system, comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to (i.e., Bansal, Fig. 13, para 144-149).
Claim(s) 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Bansal in view of Clement and Liu as applied to claims 4 and 10 above, and further in view of Kishimoto et al. (Pub. No. US 2021/0326736 A1, published October 21, 2021) hereinafter Kishimoto.
Regarding claim 6, which depends from claim 4 and recites:
wherein performing the search includes performing a limited discrepancy search over a tree, where the tree includes a set of levels that correspond to respective elements of the plurality of pipelines.
Bansal in view of Clement and Liu teaches the computer-implemented method of claim 4 from which claim 6 depends, including performing the search and elements of the plurality of pipelines. Bansal in view of Clement and Liu does not specifically disclose performing a limited discrepancy search over a tree, where the tree includes a set of levels that correspond to respective elements of the plurality of pipelines.
However, Kishimoto teaches in the field related to automated generation of a machine learning pipeline, and more specifically, to automated generation of a machine learning pipeline based on a pipeline grammar. Kishimoto, para 1. Kishimoto, which is analogous to the claimed invention because Kishimoto is directed to generation of machine learning pipelines, teaches that, 0053] In the above example, as described in detail below, machine learning pipeline generation system 102 can further facilitate via processor 106 (e.g., a classical processor, a quantum processor, etc.): evaluating one or more machine learning pipeline structure candidates in a pipeline space defined by the pipeline grammar to generate the machine learning pipeline structure based on the pipeline grammar; performing a limited discrepancy search to select the one or more machine learning modules; performing a first limited discrepancy search based on a first search parameter value and one or more second limited discrepancy searches based on one or more second search parameter values to select the one or more machine learning modules (performing the search includes performing a limited discrepancy search over a tree (over a DAG tree), where the tree includes a set of levels that correspond to respective elements (machine learning modules) of the plurality of pipelines); performing at least one of incremental training or incremental evaluation of an instantiated machine learning pipeline candidate using defined quantities of data samples in a dataset to select the one or more machine learning modules; and/or tuning one or more hyperparameters of one or more machine learning module candidates to select the one or more machine learning modules. In the examples above, the machine learning pipeline structure can comprise a directed acyclic graph (DAG) structured machine learning pipeline structure. Kishimoto, para 53, 54 72, 74-75, 93, 101-102.
It would have been obvious to one of ordinary skill in the art to implement the computer-implemented method of generating a plurality of pipelines and model tuning of Bansal using the pipeline elements including a foundation model element of Clement and the pipeline elements including an agent of Liu and the performing the search includes performing a limited discrepancy search over a tree, where the tree includes a set of levels that correspond to respective elements of the plurality of pipelines of Krishimoto, with a reasonable expectation of success, in order to use a pre-trained model as a starting point for a related task and reduce the training time and cost in developing a deep learning model, and to provide the automatic construction of the tuning, deployment, and execution infrastructures for those users having limited machine learning expertise and to provide a more reasonable reward scheme by improved coordination between different features and in order to provide for evaluating one or more machine learning pipeline structure candidates in a pipeline space and select ones that achieve a defined objective. Kishimoto, Abstract, para 3-5, 53, 72. Clement, para 2-5. Liu, page 207, abstract, lines 18-20. This would have provided the advantages of easier and faster pipeline generation, evaluation, selection and pipeline model tuning for desired tasks and improving generated pipelines.
Claim 15 recites a system that parallels the computer-implemented method of claim 6. Therefore, the analysis discussed above with respect to claim 6 also applies to claim 15. Accordingly, claim 15 is rejected based on substantially the same rationale as set forth above with respect to claim 6.
Allowable Subject Matter
Claims 3, 7-9, 12, 16-18, 22 and 25 would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and if the indefiniteness rejections are overcome and if the abstract idea rejections are overcome.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US-20250053784-A1, US-20240330589-A1, US-20240330589-A1, US-12619915-B2, US-11551151-B2, US-20220207349-A1, US-20220398460-A1, US-20220180207-A1, US-20030220906-A1.
Isabelly Rocha et al., PipeTune Pipeline Parallelism of Hyper and System Parameters Tuning for Deep Learning Clusters. In 21st International Middleware Conference (Middleware ’20), December 7–11, 2020, Delft, Netherlands. ACM, New York, NY, USA, 16 pages, https://doi.org/10.1145/3423211.3425692.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BARBARA LEVEL whose telephone number is (303)297-4748. The examiner can normally be reached Monday through Friday 8:00 AM - 5:00 PM MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BARBARA M LEVEL/ Examiner, Art Unit 2142
Prosecution Insights