DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and claims filed 10/18/2023. Claims 1-20 are pending and have been examined. Claims 1-20 are rejected.
Information Disclosure Statement
Acknowledgment is made of the information disclosure statement filed 10/18/2023, which complies with 37 CFR 1.97. As such, the information disclosure statement has been placed in the application file and the information referred to therein has been considered by the examiner.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “602” in Fig. 6 has been used to designate both “Obtain graph of GTM” and “Transform graph of HM into graph of GTM.” Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
Reference character 603 shown in Figure 6 is not found in the detailed description (see, e.g., paragraph 116 describing FIG. 6).
Appropriate correction is required.
Claim Objections
Claim 14 is objected to because of the following informalities: In line 4 of claim 14, the recitation of “the second set of training data set” is grammatically correct and appears to be missing the words “optimization problems of the” between “of” and “training dataset.” Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis below of the claims’ subject matter eligibility follows the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”) and the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, 89 Fed. Reg. 58128-58138 (July 17, 2024) (“2024 AI SME Update”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Regarding independent claim 1, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: claim 1 is directed to a computer-implemented method, corresponding to a process, which is one of the four statutory categories.
Step 2 Prong 1: The claim recites “transforming the graph of the HM into the graph of the GTM through a series of transformation steps, a total number of the transformation steps to transform the graph of the HM into the graph of the GTM being a measure of the accuracy of the HM.”
This limitation, under the broadest reasonable interpretation (BRI), covers a mathematical concept (see e.g., paragraph 20 of specification, “ The method also comprises calculating a score representing the accuracy of the LLM, the score being a function of all the total numbers of transformation steps for transforming all the HM graphs associated with plurality of HMs into all the respective GTM graphs associated with the plurality of GTMs.”). Such a method of transforming a graph with transforming steps being a measure of accuracy using a function of the total number of transformation steps can be done with pen and paper, further providing evidence that the claimed transformation of the graph of the HM is for the purpose of reaching a measure of accuracy of the HM is itself a mathematical concept.
Accordingly, claim 1 recites an abstract idea.
Step 2 Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements: “A computer-implemented method for evaluating an accuracy of a hypothesis model (HM) against a ground truth model (GTM), the method comprising: <the above-noted step>, which are recited at a high level of generality as mere instructions to implement an abstract idea on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f). This limitation is recited at a high level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. The evaluating is recited at a high-level of generality with no details such that the limitation amounts to no more than mere instructions to apply the exception using a generic computer component (i.e., the “computer” used to implement the “computer-implemented method for evaluating”) (See MPEP 2106.05(f)).
Regarding the “hypothesis model (HM)” and “ground truth model (GTM)”, no details of the models or their training are recited and the models are recited at a high level of generality and can be constructed by hand with pen and paper. The claimed models, under the BRI, in light of the specification, could be constructed and modified by hand with pen and paper based on a reasonable amount of observed data (i.e., ground truth data). The models are recited at a high level of generality and therefore are being interpreted as performing an abstract idea on a generic computer.
The claim also recites the additional elements:
obtaining a graph of the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices;
obtaining a graph of the GTM, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices”
The obtaining graphs of the generically-recited HM and GTM model limitations are insignificant extra-solution activities that are not integrated into the claim as a whole and do not add a meaningful limitation to the above-noted abstract idea specified in this claim. That is, “obtaining a graph of the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices;” and “obtaining a graph of the GTM, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices” amount to mere data gathering as they recite obtaining data that defines the obtained graphs. (See MPEP 2106.05(g)).
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are insignificant extra-solution activities or mere instructions to apply an exception. Insignificant extra-solution activities and mere instructions to apply an exception cannot provide an inventive concept.
Moreover, receiving, communicating, and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, the recitations of “obtaining a graph of the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices;” and “obtaining a graph of the GTM, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
This claim is not patent eligible.
Regarding claim 2, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step1: Claim 2 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites “wherein transforming the graph of the HM into the graph of the GTM includes automatically transforming the graph of the HM into the graph of the GTM using a graph edit distance computing algorithm.”
Using a graph edit distance computing algorithm (e.g., deleting, inserting or substituting nodes and edges to transform/modify “the graph of the HM into the graph of the GTM” is a mathematical concept. This can be done by hand with pen and paper, as suggested by the discussion of this step in paragraph 24 of the specification, further providing evidence that the claimed transformation using a graph edit distance computing algorithm is a mathematical concept.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 3, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step1: Claim 3 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: Claim 3 does not recite any abstract ideas but is dependent on claim 1. See the analysis of Claim 1 above.
Step 2A Prong 2: The claim recites “further comprising obtaining the HM by inputting a math word problem into a large language model (LLM) to generate the HM.” This limitation describes data gathering with the variables as the data. The “obtaining” limitation is adding insignificant extra-solution activity (amounts to necessary data gathering) to the judicial exception, as discussed in MPEP § 2106.05(g). The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional element of this claim as a combination does not add anything further than the individual elements.
Receiving, communicating, and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, recitations of “further comprising obtaining the HM by inputting a math word problem into a large language model (LLM) to generate the HM” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, and storing information in memory, as discussed in MPEP § 2106.05(d).
Furthermore, “obtaining1 the HM” is also understood to be well-understood, routine, and conventional (see, e.g., U.S. Publication No. 20180083833 (Zoll, Michael), paragraph 5: “conventional machine learning is typically applied to generate models for certain select components based upon certain select data signals for that component.”).
The claim is not patent eligible.
Therefore, recitations of “further comprising obtaining the HM by inputting a math word problem into a large language model (LLM) to generate the HM” are the well-understood, routine, conventional (WURC) activity of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Regarding claim 4, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 4 is directed to a method as depending from claim 3 and further depending from claim 1, thus the analysis for patent eligibilities of those claims are incorporated herein.
Step 2A Prong 1: Claim 4 does not recite any abstract ideas but is dependent on claims 1 and 3. See the analysis of Claims 1 and 3 above.
Step 2A Prong 2: The claim recites “wherein inputting the math word problem into the LLM includes inputting a math word optimization problem into the LLM.”
This limitation amounts to mere data gathering.
The claim recites “wherein inputting the math word problem into the LLM includes inputting a math word optimization problem into the LLM.” The ”inputting” limitation is adding insignificant extra-solution activity (amounts to necessary data gathering) to the judicial exception, as discussed in MPEP § 2106.05(g). The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
The “wherein inputting the math word problem into the LLM includes inputting a math word optimization problem into the LLM” can be characterized as well understood routine and conventional. The LLM is mentioned a number of times in the specification and is referred to as a generic language model and thus, “the element is widely prevalent or in common use in the relevant industry” [see, i.e., MPEP 2106.05(d)I2]. Further, the inputting data/math word problem into the LLM is a routine and conventional activity commonly associated with an LLM.
This claim is not patent eligible.
Regarding claim 5, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 5 is directed to a method as depending from claim 3 and further depending from claim 1, thus the analysis for patent eligibilities of those claims are incorporated herein.
Step 2A Prong 1: The claim recites “transforming the at least one HM function and the one or more HM constraints equation into a general linear programming model of the math word problem to obtain the HM.”
This limitation of “transforming the at least one HM function and the one or more HM constraints equation into a general linear programming model of the math word problem to obtain the HM” does nothing to alter the fundamental nature of the claim with a mathematical concept. The above limitation covers concepts encompassing a mathematical concept (mathematical formula/equations – putting a function and equation into a model to obtain a result [e.g., see paragraph 96 of specification, “The value of the edges between the vertices may be determined in accordance with the constraints equation, the general form of which i
PNG
media_image1.png
221
438
media_image1.png
Greyscale
”]). Those inputs are put into the model [e.g., “Linear programming may refer to a mathematical technique used to find the best outcome in a mathematical model with linear relationships”]. Such transformation of at least one function and equation can be done by hand with pen and paper, and is itself a math concept pertaining to mathematical formula/equation.
Dependent claim 5, when analyzed as a whole, is not patent eligible under 35 U.S.C. 101 because the additional recited limitation fails to establish that the claim is not directed to an abstract idea. Thus, this limitation does nothing to alter the analysis of claim 3 and claim 1.
Step 2A Prong 2: The claim recites “wherein generating the HM comprises: obtaining, from the LLM, the HM variables;
a lower bound for each respective HM variable of the HM variables; and
an upper bound for each respective HM variable of the HM variables;
at least one HM function depending on the HM variables, the at least one function defining an output to be optimized in accordance with the math word problem; and
These limitations amount to mere data gathering.
The limitation of “wherein generating the HM comprises: obtaining, from the LLM, the HM variables” describes data gathering whereby variables/data are being obtained/gathered from the LLM. Such data gathering can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The limitations “a lower bound for each respective HM variable of the HM variables; and an upper bound for each respective HM variable of the HM variables;” merely state the need to have a lower and upper bound to generate the HM. Such nominal or tangential addition to the claim can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The limitation “at least one HM function depending on the HM variables, the at least one function defining an output to be optimized in accordance with the math word problem; and” merely state the need for a function defining an output. Such nominal or tangential addition to the claim can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 6, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 6 is directed to a method as depending from claim 5 and further depending from claim 3 and then from claim 1, thus the analysis for patent eligibilities of those claims are incorporated herein.
Step 2A Prong 1: The claim recites “a lower bound and an upper bound for each respective GTM variable of the GTM variables;”
The limitation of “a lower bound and an upper bound for each respective GTM variable of the GTM variables;” added by this claim covers a mathematical concept (“ The first vertex 502 may correspond to the first constraint: x.sub.1+x.sub.2<=50 of the prediction 306 (l.sub.1.sup.s=−∞,u.sub.1.sup.s=50). The attributes, [−∞, 50].sup.T, of the first constraint vertex 502 may include a lower bound of −∞ and an upper bound of 50. The second vertex 504 may correspond to the second constraint: x.sub.2<=2x.sub.1, which may be rewritten as −2x.sub.1+x.sub.2<=0 (l.sub.2.sup.s=−∞, u.sub.2.sup.s=0). For the second constraint vertex 504, its attributes, [−∞, 0].sup.T, include a lower bound of −∞ and an upper bound of 0. In the present embodiment, HM graph is based on the HM, which based on the output of a LLM to which a LPWP has been provided.”, as suggested by the discussion of this step in paragraph 93 of the applicant’s specification, further providing evidence that the claimed GTM model association is for the purpose of associating with lower and upper bounds of respective GTM variables is itself a mathematical concept.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application. In particular, the claim recites the additional element of “wherein the GTM model has associated thereto: a lower bound and an upper bound for each respective GTM variable of the GTM variables;” which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – “a computing device” performing generic computer function) , which does not integrate a judicial exception into a practical application.. See MPEP 2106.05(f).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
Also, as discussed above with respect to integration of the abstract idea into a practical application, mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 7, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 7 is directed to a method as depending from claim 6 and further depending from claim 5, then from claim 3, and then from claim 1, thus the analysis for patent eligibilities of those claims is incorporated herein.
Step 2A Prong 1: Claim 7 does not recite any abstract ideas but is dependent on claims 1, 3, 5 and 6. See the analysis of Claims 1, 3, 5, and 6 above.
Step 2A Prong 2: The claim recites “wherein: obtaining the graph of the HM includes forming a HM attributed bipartite graph by generating: the HM constraints vertices in accordance with: the HM variables; and
the one or more HM constraints equation;
the HM variables vertices in accordance with: the HM variables;
the lower bound and the upper bound for each respective HM variable of the HM variables; and
the at least one HM function; and
the HM edges connecting the HM constraints vertices to the HM variables vertices in accordance with the one or more HM constraints equation, wherein: the HM constraints vertices form a first HM set of vertices, the HM variables vertices form a second HM set of vertices, the first HM set of vertices and the second HM set of vertices are disjoint, and
the one or more GTM constraints equation;
the GTM variables vertices in accordance with: the GTM variables;
the lower bound and the upper bound for each respective GTM variable of the GTM variables; and
the at least one GTM function; and
the GTM edges connecting the GTM constraints vertices to the GTM variables vertices in accordance with the one or more GTM constraints equation, wherein: the GTM constraints vertices form a first GTM set of vertices, the GTM variables vertices form a second GTM set of vertices, and the first GTM set of vertices and the second GTM set of vertices are disjoint.”
These limitations amount to mere data gathering.
The claim recites “the HM variables; and” This limitation describes data gathering with the variables as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the one or more HM constraints equation;” This limitation describes data gathering with the one or more constraints equation(s) as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the HM variables vertices in accordance with: the HM variables” This limitation describes data gathering with the one or more constraints equation(s) as the data. Obtaining such variables can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the lower bound and the upper bound for each respective HM variable of the HM variables; and” This limitation describes data gathering with the lower and upper bound of HM variables as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the at least one HM function” This limitation describes data gathering with the at least one HM function as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the HM edges connecting the HM constraints vertices to the HM variables vertices in accordance with the one or more HM constraints equation, wherein: the HM constraints vertices form a first HM set of vertices, the HM variables vertices form a second HM set of vertices, the first HM set of vertices and the second HM set of vertices are disjoint” This limitation describes data gathering with the edges connecting to the HM constraints vertices to the HM variable vertices as the data. obtaining such variables can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the one or more GTM constraints equation;” This limitation describes data gathering with the one or more constraints equation(s) as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the GTM variables vertices in accordance with: the GTM variables;” This limitation describes data gathering with the GTM variable vertices as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the lower bound and the upper bound for each respective GTM variable of the GTM variables; and” This limitation describes data gathering with the lower and upper bounds of the GTM variables as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the at least one GTM function;” This limitation describes data gathering with the GTM function as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “the GTM edges connecting the GTM constraints vertices to the GTM variables vertices in accordance with the one or more GTM constraints equation, wherein: the GTM constraints vertices form a first GTM set of vertices, the GTM variables vertices form a second GTM set of vertices, and the first GTM set of vertices and the second GTM set of vertices are disjoint.” This limitation describes data gathering with edges, connecting the vertices to the variable’s vertices as the data. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
The “obtaining the graph of the GTM includes forming a GTM attributed bipartite graph by generating: the GTM constraints vertices in accordance with: the GTM variables;” and can be characterized as well understood routine and conventional. According to MPEP 2106.05(d) Subsection II, "The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)". Therefore, the recitations of “wherein: obtaining the graph of the HM includes …” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
A mere act to apply an exception using a generic act of receiving and transmitting cannot provide an inventive concept.
Regarding claim 8, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 8 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites “the series of transformation steps includes at least one of: a substitution of a HM constraints value with a different HM constraints value;”
a substitution of a HM variables value with a different HM variables value;
an addition of a HM constraints vertex;
an addition of a HM variables vertex;
a deletion of a HM constraints vertex;
a deletion of a HM variables vertex;
an addition of an edge connecting an HM constraints vertex to an HM variables vertex;
a deletion of a second edge connecting a second HM constraints vertex to a second HM variables vertex; and
a substitution of a weight of any edge with a different weight.”
The substitution of a HM constraints value with a different HM constraints value limitation, the substitution of a HM variables value with a different HM variables value limitation and a substitution of a weight of any edge with a different weight limitation added by this claim covers mathematical concepts (each one as one of a series of transformation steps to transform the HM into a reference model). Such transformations using substitutions are by themselves a mathematical concept.
The addition of a HM constraints vertex limitation, the addition of a HM variables vertex limitation, and the addition of an edge connecting an HM constraints vertex limitation added by this claim covers a mathematical concept (each one as one of a series of transformation steps to transform the HM into a reference model). Such transformation using addition is itself a mathematical concept.
The deletion of a HM constraints vertex limitation, the deletion of a HM variables vertex limitation, and the deletion of a second edge connecting a second HM constraints vertex limitation added by this claim covers a mathematical concept (each one as one of a series of transformation steps to transform the HM into a reference model). Such transformation using deletion is itself a mathematical concept.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 9, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 9 is directed to a method as depending from claim 3 and further depending from claim 1, thus the analysis for patent eligibilities of those claims are incorporated herein.
Step 2A Prong 1: The claim recites “modifying the LLM in accordance with the accuracy score; and
D) transforming the graph of the further HM into the graph of the further GTM through a series of further transformation steps, a total number of the further transformation steps to transform the graph of the further HM into the graph of the further GTM being a measure of the accuracy of the further HM;
F) modifying the LLM in accordance with the accuracy score;
The limitation of “modifying the LLM in accordance with the accuracy score;” added by this claim covers a mathematical concept (modifying an algorithm). Such modification is based off of an accuracy score uses iteratively performed actions A) through G) on a LLM that can be done by hand a with pen and paper, as suggested by the discussion of this step, in paragraph 12 of the specification, further providing evidence that the claimed modification of a LLM based off of an accuracy score is itself a mathematical concept.
The limitation of “D) transforming the graph of the further HM into the graph of the further GTM through a series of further transformation steps, a total number of the further transformation steps to transform the graph of the further HM into the graph of the further GTM being a measure of the accuracy of the further HM;” added by this claim covers a mathematical concept (transforming the graph). Such transformation through a series of transformation steps can be done by hand with pen and paper, as suggested by the discussion of this step inside paragraph 12-14 of the specification, further providing evidence that the claimed transformation steps is for the purpose of transforming the graph is itself a mathematical concept.
The limitation of “F) modifying the LLM in accordance with the accuracy score;” added by this claim covers a mathematical concept (modifying the LLM). Such modifying of the LLM can be done by hand with pen and paper, as suggested by the discussion of this step in the specification in paragraph 12, concerning steps A) through G) iteratively performed, further providing evidence that modifying the LLM in accordance with the accuracy score is itself a mathematical concept.
Step 2A Prong 2: The claim recites “obtaining an accuracy score in accordance with the total number of transformation steps, the accuracy score indicating the ability of the LLM to generate accurately the HM;
B) obtaining a graph of the further HM;
C) obtaining a graph of the further GTM;
E) obtaining a further accuracy score in accordance with the total number of the
further transformation steps, the further accuracy score indicating the ability of the
LLM to generate accurately the further HM;
These limitations amount to mere data gathering.
G) determining if the stop criteria is met.
This limitation merely repeats the above-noted mathematical calculations/computations
The limitation of “obtaining an accuracy score in accordance with the total number of transformation steps, the accuracy score indicating the ability of the LLM to generate accurately the HM;” describes data gathering as obtaining/gathering an accuracy score. Such data gathering can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The limitations “B) obtaining a graph of the further HM;” and “C) obtaining a graph of the further GTM;” describe data gathering as obtaining/gathering a graph. Such data gathering can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The limitation of “E) obtaining a further accuracy score in accordance with the total number of the further transformation steps, the further accuracy score indicating the ability of the LLM to generate accurately the further HM;” describes data gathering as obtaining/gathering a further accuracy score. Such data gathering can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The limitation “G) determining if the stop criteria is met” merely repeats the above-noted mathematical calculations/computations. Such repetition of calculations can be characterized as insignificant extra solution activity. See MPEP 2106.05(g).
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional element of this claim as a combination does not add anything further than the individual elements.
The “iteratively performing actions A) through G) until a stop criteria is met” can be characterized as insignificant extra solution activity that is well understood routine and conventional. See MPEP 2106.05(g) and MPEP 2106.05(d)(II) example (ii) provides that performing repetitive calculations has been understood by the courts to be well-understood, routine and conventional.
ii. Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values); Bancorp Services v. Sun Life, 687 F.3d 1266, 1278, 103 USPQ2d 1425, 1433 (Fed. Cir. 2012). Therefore, recitations of “iteratively performing actions A) through G) until stop criteria is met” is a well-understood, routine, conventional (WURC) activity of performing repetitive calculations
As an ordered whole, the claim is directed to a method of performing mathematical calculations, data gathering (i.e., insignificant extra solution activities) and performing repetitive calculations (WURC). Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Regarding claim 10, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 10 is directed to a method as depending from claim 5, further depending from intervening claim 3 and base claim 1. Thus, the analysis for patent eligibilities of these claims are incorporated herein.
Step 2A Prong 1: Claim 10 does not recite any abstract ideas but is dependent on claims 1, 3 and 5. See the analysis of Claims 1, 3 and 5 above.
Step 2A Prong 2: The claim recites “obtaining a first graph representation comprises: obtaining a first set of graph representations of a first set of optimization models generated by the machine learning model based on a first set of optimization problems, each graph representation of the first set of graph representations corresponding to an optimization model of the first set of optimization models based on the corresponding optimization problem of the first set of optimization problems;
obtaining a second graph representation comprises: obtaining a second set of graph representations of a second set of optimization models serving as ground truth for the first set of optimization problems, each graph representation of the second set of graph representations corresponding to an optimization model of the second set of optimization models based on the corresponding optimization problem of the first set of optimization problems; and
obtaining a minimum number of edit operations comprises: obtaining a set of edit operations to transform the first set of graph representations to the second set of graph representations.”
These limitations amount to mere data gathering.
The claim recites “obtaining a first graph representation comprises: obtaining a first set of graph representations of a first set of optimization models generated by the machine learning model based on a first set of optimization problems, each graph representation of the first set of graph representations corresponding to an optimization model of the first set of optimization models based on the corresponding optimization problem of the first set of optimization problems;” This limitation describes data gathering with obtaining, meaning gathering a first graph representation. obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “obtaining a second graph representation comprises: obtaining a second set of graph representations of a second set of optimization models serving as ground truth for the first set of optimization problems, each graph representation of the second set of graph representations corresponding to an optimization model of the second set of optimization models based on the corresponding optimization problem of the first set of optimization problems” This limitation describes data gathering with obtaining, meaning gathering a second graph representation. obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “obtaining a minimum number of edit operations comprises: obtaining a set of edit operations to transform the first set of graph representations to the second set of graph representations.” This limitation describes data gathering with obtaining, meaning gathering a set of edit operations. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exceptions are not integrated into a practical application.
This claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
The “obtaining a first graph representation comprises: obtaining a first set of graph representations of a first set of optimization models generated by the machine learning model based on a first set of optimization problems, each graph representation of the first set of graph representations corresponding to an optimization model of the first set of optimization models based on the corresponding optimization problem of the first set of optimization problems; obtaining a second graph representation comprises: obtaining a second set of graph representations of a second set of optimization models serving as ground truth for the first set of optimization problems, each graph representation of the second set of graph representations corresponding to an optimization model of the second set of optimization models based on the corresponding optimization problem of the first set of optimization problems; and obtaining a minimum number of edit operations comprises: obtaining a set of edit operations to transform the first set of graph representations to the second set of graph representations” can be characterized as well understood routine and conventional.
According to MPEP 2106.05(d) Subsection II, "The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)". Therefore, the recitations of “obtaining …” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
This claim is not patent eligible.
Regarding claim 11, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 11 is directed to a method as depending from claim 10, further depending from claim 5, intervening claim 3, and base claim 1. Thus, the analysis for patent eligibilities of these claims is incorporated herein.
Step 2A Prong 1: Claim 11 does not recite any abstract ideas but is dependent on claims 1, 3, 5 and 10. See the analysis of Claims 1, 3, 5 and 10 above.
Step 2A Prong 2: The claim recites “obtaining, for said each optimization problem, a sequence of one or more edit operations of the set of edit operations to transform a corresponding graph of the first set of graph representations to a corresponding graph representation of the second set of graph representations.”
This limitation amounts to mere data gathering.
The claim recites “obtaining, for said each optimization problem, a sequence of one or more edit operations of the set of edit operations to transform a corresponding graph of the first set of graph representations to a corresponding graph representation of the second set of graph representations.” This limitation describes data gathering with obtaining, meaning gathering a sequence of one or more edit operations. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exception is not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B:
The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
The “obtaining, for said each optimization problem, a sequence of one or more edit operations of the set of edit operations to transform a corresponding graph of the first set of graph representations to a corresponding graph representation of the second set of graph representations” can be characterized as well understood routine and conventional.
According to MPEP 2106.05(d) Subsection II, "The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)". Therefore, the recitation of “obtaining …” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
This claim is not patent eligible.
Regarding claim 12, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 12 is directed to a method as depending from claim 11, further depending from claim 10, intervening claims 5 and 3, and base claim 1. Thus, the analysis for patent eligibilities of these claims is incorporated herein.
Step 2A Prong 1: The claim recites “a mean of GEDs based on the set of GEDs.”
The limitation of “a mean of GEDs based on the set of GEDs.” added by this claim covers a mathematical concept [a mean of a GED (graph edit distance)]. Calculating such a mean/average can be done by hand with pen and paper and is a mathematical concept pertaining to a type of calculation.
Step 2A Prong 2: The claim recites “obtaining, a set of graph edit distances (GEDs) corresponding to the set of edit operations, each GED corresponding to said each optimization problem and indicating a measure of the corresponding sequence of the one or more edit operations;
obtaining one or more of: a ratio of exact match based on the set of GEDs, the ratio of exact match indicating a proportion of the first set of optimization models having a corresponding GED of the set of GEDs indicating an equivalent match; and”
These limitations amount to mere data gathering.
The claim recites “obtaining, a set of graph edit distances (GEDs) corresponding to the set of edit operations, each GED corresponding to said each optimization problem and indicating a measure of the corresponding sequence of the one or more edit operations;” This limitation describes data gathering with obtaining, meaning gathering a set of GEDs. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “obtaining one or more of: a ratio of exact match based on the set of GEDs, the ratio of exact match indicating a proportion of the first set of optimization models having a corresponding GED of the set of GEDs indicating an equivalent match” This limitation describes data gathering with obtaining, meaning gathering a ratio of exact match. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exception is not integrated into a practical application.
This claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 13, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 13 is directed to a method as depending from claim 10, further depending from claim 5, intervening claim 3, and base claim 1. Thus, the analysis for patent eligibilities of these claims is incorporated herein.
Step 2A Prong 1: The claim recites “adjusting one or more weights of the machine learning model based on the set of edit operations, the machine learning model comprising a plurality of nodes connected with one another via a plurality of connections, the machine learning model further comprising the one or more weights corresponding to the plurality of connections.”
The limitation of “adjusting one or more weights of the machine learning model based on the set of edit operations, the machine learning model comprising a plurality of nodes connected with one another via a plurality of connections, the machine learning model further comprising the one or more weights corresponding to the plurality of connections” as drafted, under its BRI, in view of the specification (see, e.g., paragraph 19), covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion based on the set of edit operations). The above limitation in the context of this claim encompasses adjusting weights of the machine learning model with respect to the plurality of nodes connected with one another (corresponding to a mental process which can be done mentally or by pen and paper).
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 14, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 14 is directed to a method as depending from claim 10, further depending from claim 5, intervening claim 3, and base claim 1. Thus, the analysis for patent eligibilities of these claims is incorporated herein.
Step 2A Prong 1: The claim recites “generating a training dataset based on the set of edit operations, the training dataset comprising a second set of optimization problems; and
training the machine learning model using the second set of training dataset.”
The limitation of “generating a training dataset based on the set of edit operations, the training dataset comprising a second set of optimization problems” added by this claim covers a mathematical concept (generating a training dataset including optimization problems). Such generating a training dataset can be done by hand with pen and paper, based off of edit operations and the optimization problems are a mathematical concept pertaining to a formula or equation based on a calculation.
The limitation of “training the machine learning model using the second set of training dataset2” added by this claim covers a mathematical concept (training a generically-recited machine learning model using the second set of optimization problems of the training dataset). Such optimization problems can be solved by hand with pen and paper- a mathematical concept pertaining to a formula or equation based on calculations.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding independent claim 15, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Independent claim 15 is directed to a method, corresponding to a process, which is one of the four statutory categories.
Step 2A Prong 1: The claim recites “transforming the graph of the HM into the graph of the GTM through a sequence of transformation steps having a total number of transformation steps;
calculating a score representing the accuracy of the LLM, the score being a function of all
the total numbers of transformation steps for transforming all the HM graphs associated with plurality of HMs into all the respective GTM graphs associated with the plurality of GTMs.”
The limitations of “transforming the graph of the HM into the graph of the GTM through a sequence of transformation steps having a total number of transformation steps; and calculating a score representing the accuracy of the LLM, the score being a function of all the total numbers of transformation steps for transforming all the HM graphs associated with plurality of HMs into all the respective GTM graphs associated with the plurality of GTMs which cover mathematical concepts (transforming a graph may involve the insertion or deletion or even rotation and substitution of nodes, [see, e.g., figure 5B], and calculating an accuracy score of the LLM depends on deleting, inserting and substituting nodes and edges). Such transforming the graph of the HM and calculating accuracy score can be done by hand with pen and paper, as suggested by the discussion of this step in paragraph 93 of the specification, further providing evidence that the claimed transformation of the graph for the purpose of node substitution and insertion and edge insertion is a mathematical concept (i.e., mathematical relationships).
Step 2A Prong 2: The claim recites the additional limitations: “obtaining, from the LLM, a plurality of HMs corresponding to a plurality of LPWPs, each LPWP of the plurality of LPWPs having associated thereto a respective ground truth model (GTM), all the respective GTMs forming a plurality of GTMs, each HM of the plurality of HMs having associated thereto a respective GTM of the plurality of GTMs;
for each HM of the plurality of HMs: obtaining a graph of the HM; and
obtaining a graph of the GTM associated with the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices”
These limitations amount to mere data gathering.
The claim also recites “obtaining, from the LLM, a plurality of HMs corresponding to a plurality of LPWPs, each LPWP of the plurality of LPWPs having associated thereto a respective ground truth model (GTM), all the respective GTMs forming a plurality of GTMs, each HM of the plurality of HMs having associated thereto a respective GTM of the plurality of GTMs” This limitation describes data gathering associated with obtaining a plurality of HMs. Such obtaining can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim additionally recites “for each HM of the plurality of HMs: obtaining a graph of the HM; and” This limitation describes data gathering with obtaining, meaning gathering a graph of the HM. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “obtaining a graph of the GTM associated with the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices; and” This limitation describes data gathering with obtaining, meaning gathering data representing a graph of the GTM associated with the HM. Such obtaining can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exception is not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are insignificant extra-solution activities or mere instructions to apply an exception. Insignificant extra-solution activities and mere instructions to apply an exception cannot provide an inventive concept.
Moreover, receiving, communicating, and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, the recitations of “obtaining a graph of the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices;” and “obtaining a graph of the GTM, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices” are the well-understood, routine, conventional activities of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
This claim is not patent eligible.
Regarding independent claim 16, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Independent claim 16 is directed to a method., corresponding to a process, which is one of the four statutory categories.
Step 2A Prong 1: The limitation of “obtaining a first graph representation of a hypothesis optimization problem model generated by a machine learning model based on an optimization problem;” Under its BRI, in view of the specification, covers a mathematical concept. For example, applicant’s specification discloses “the machine learning model may be a LLM and the optimization problem may be one of: a LPWP; a mixed integer linear programming problem; a quadratic programming problem; and a quadratically constrained quadratic programming problem” [see e.g., paragraph 22]). Such an optimization problem can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation based on a calculation.
Step 2A Prong 2: The claim recites the additional limitations: “obtaining a second graph representation of a reference optimization problem model based on the optimization problem; and
obtaining a minimum number of edit operations to transform the first graph representation into the second graph representation, the minimum number of operations indicating a performance of the machine learning model.”
These limitations amount to mere data gathering.
The second graph representation limitation describes data gathering with obtaining, meaning gathering a second graph representation. The ”obtaining” limitation is adding insignificant extra-solution activity (amounts to necessary data gathering) to the judicial exception, as discussed in MPEP § 2106.05(g). The minimum number of edit operations limitation describes data gathering with obtaining, meaning gathering a minimum number of edit operations. The “obtaining” limitation is adding insignificant extra-solution activity (amounts to necessary data gathering) to the judicial exception, as discussed in MPEP § 2106.05(g). The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 17, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 17 is directed to a method as depending from claim 16. Thus, the analysis for patent eligibility of claim 16 is incorporated herein.
Step 2A Prong 1: The claim recites “the machine learning model is a large language model;
the optimization problem is one of: a linear programming word problem;
a mixed integer linear programming problem;
a quadratic programming problem; and
a quadratically constrained quadratic programming problem;
the first graph representation is a first attributed bipartite graph;
the second graph representation is a second attributed bipartite graph; and
the minimum number of operations is a function of a graph edit distance between the first attributed bipartite graph and the second attributed bipartite graph.”
The limitation of “the machine learning model is a large language model;” added by this claim covers a mathematical concept. That is, “In some embodiments of the fourth aspect, the machine learning model may be a LLM and the optimization problem may be one of: a LPWP; a mixed integer linear programming problem; a quadratic programming problem; and a quadratically constrained quadratic programming problem [see e.g., paragraph 22]). Such machine learning model can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation based on a calculation.
The limitation of “the optimization problem is one of: a linear programming word problem;” added by this claim covers a mathematical concept. That is, “Evaluating prediction models, particularly those generated by machine learning models like LLMs for optimization problems such as linear programming word problems (LPWPs), presents a unique set of challenges… Addressing these challenges in evaluating prediction models for optimization problems is important for enhancing model reliability and applicability in real-world scenarios… Therefore, improvements in obtaining a mathematical form of an optimization problem with LLMs are desirable [see e.g., paragraph 3-4]). Such improvement in obtaining a mathematical form can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation.
The limitation of “a mixed integer linear programming problem;” added by this claim covers a mathematical concept. That is, “the machine learning model may be a LLM and the optimization problem may be one of: a LPWP; a mixed integer linear programming problem; a quadratic programming problem; and a quadratically constrained quadratic programming problem.” [see e.g., paragraph 22]). Such an optimization problem can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation.
The limitation of “a quadratic programming problem; and “added by this claim covers a mathematical concept. That is, “the machine learning model may be a LLM and the optimization problem may be one of: a LPWP; a mixed integer linear programming problem; a quadratic programming problem; and a quadratically constrained quadratic programming problem.” [see e.g., paragraph 22]). Such optimization problem can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation.
The limitation of “a quadratically constrained quadratic programming problem;” added by this claim covers a mathematical concept. That is, “the machine learning model may be a LLM and the optimization problem may be one of: a LPWP; a mixed integer linear programming problem; a quadratic programming problem; and a quadratically constrained quadratic programming problem.” [see e.g., paragraph 22]). Such optimization problem can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation.
The limitation of “the first graph representation is a first attributed bipartite graph;” That is, “Each graph representation of the first set of graph representations may correspond to an optimization model of the first set of optimization models based on the corresponding optimization problem of the first set of optimization problems.” [see e.g., paragraph 13]). Such graph representation corresponding to an optimization problem can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation.
The limitation of “the second graph representation is a second attributed bipartite graph; and” That is, “Each graph representation of the second set of graph representations may correspond to an optimization model of the second set of optimization models based on the corresponding optimization problem of the first set of optimization problems.” [see e.g., paragraph 13]). Such graph representation corresponding to an optimization problem can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a formula or equation.
The limitation of “the minimum number of operations is a function of a graph edit distance between the first attributed bipartite graph and the second attributed bipartite graph.” That is, “The method further comprises obtaining a minimum number of edit operations to transform the first graph representation into the second graph representation. The minimum number of operations indicates a performance of the machine learning model.” [see e.g., paragraph 21]). Such minimum number of edit operations indicating machine learning model performance can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a mathematical relationship and formula.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 18, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 18 is directed to a method as depending from claim 16. Thus, the analysis for patent eligibility of claim 16 is incorporated herein.
Step 2A Prong 1: Claim 18 does not recite any abstract ideas but is dependent on claim 16. See the analysis of claim 16 above.
Step 2A Prong 2: The claim recites “obtaining a first general form of linear programming corresponding to the hypothesis optimization problem model; and
obtaining a second general form of linear programming corresponding to the reference optimization problem model.”
These limitations amount to mere data gathering.
The claim recites “obtaining a first general form of linear programming corresponding to the hypothesis optimization problem model; and” This limitation describes data gathering with obtaining, meaning gathering a first general form of linear programming corresponding to a model. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The claim recites “obtaining a second general form of linear programming corresponding to the reference optimization problem model.” This limitation describes data gathering with obtaining, meaning gathering a second general form of linear programming corresponding to a model. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 19, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 19 is directed to a method as depending from claim 16. Thus, the analysis for patent eligibility of claim 16 is incorporated herein.
Step 2A Prong 1: The claim recites “wherein: the minimum number of edit operations is a minimum sequence of edit operations to transform the first attributed bipartite to the second attributed bipartite graph;
the minimum sequence of edit operations: relate to one or more of a vertex of the first set of vertices, a vertex of the second set of vertices, an edge of the set of edges; and
comprise one or more of: an insertion operation, a deletion operation, and a substitution operation.”
The limitation of “wherein: the minimum number of edit operations is a minimum sequence of edit operations to transform the first attributed bipartite to the second attributed bipartite graph;” added by this claim covers a mathematical concept - calculation. That is, “The minimum number of operations may be a function of a graph edit distance between the first attributed bipartite graph and the second attributed bipartite graph.” [see e.g., paragraph 22]). Such function of a graph can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a calculation.
The limitation of “the minimum sequence of edit operations: relate to one or more of a vertex of the first set of vertices, a vertex of the second set of vertices, an edge of the set of edges; and” added by this claim covers a mathematical concept. That is, “For example, variables vertex 512 having attributes [0, ∞, 2].sup.T may be substituted with variables vertex 522 having attributes [5, ∞, 2].sup.T, where variables vertex 522 is equivalent to variables vertex 572 of the reference graph 560.” [see e.g., paragraph 24]). Such operations of relating one or more nodes/vertices can be done by hand with pen and paper, and is itself a mathematical concept pertaining to a mathematical relationship.
The limitation of “comprise one or more of: an insertion operation, a deletion operation, and a substitution operation.” That is, “Determining the graph edit path (a minimum sequence of edit operations) from the hypothesis graph 500 to the reference graph 560 may involve one or more edit operations as illustrated in FIG. 5B. In an embodiment, graph 500 may be edited via one or more edit operations, to obtain graph 520.” [see e.g., paragraph 105]). Such operations can be done by hand with pen and paper
, and is itself a mathematical concept pertaining to a mathematical calculation.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 20, this claim is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 20 is directed to a method as depending from claim 16. Thus, the analysis for patent eligibility of claim 16 is incorporated herein.
Step 2A Prong 1: Claim 20 does not recite any abstract ideas but is dependent on claim 16. See the analysis of claim 16 above.
Step 2A Prong 2: The claim recites “obtaining a total cost for the minimum sequence of edit operations based on a cost value assigned for each edit operation of the minimum sequence of edit operations, the total cost indicative of the performance of the machine learning model.”
These limitations amount to mere data gathering.
The claim recites “obtaining a total cost for the minimum sequence of edit operations based on a cost value assigned for each edit operation of the minimum sequence of edit operations, the total cost indicative of the performance of the machine learning model.” This limitation describes data gathering with obtaining, meaning gathering a total cost for a sequence of operations. Obtaining such variables/data can be characterized as insignificant extra-solution activity. See MPEP 2106.05(g).
The judicial exception is not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 are rejected under 35 U.S.C. 103 as being unpatentable over non-patent literature Abu-Aisheh, Zeina, et al. (“An Exact Graph Edit Distance Algorithm for Solving Pattern Recognition Problems” (06/2015), hereinafter “Abu”), in view of Geiger, Davi et al. (U.S. Publication No. 20060050962, hereinafter “Geiger”) and further in view of Blackwell, John et al. (U.S. Publication No. 20220067233, hereinafter “Blackwell”) and additionally further in view of Gutierrez, Steve (U.S. Publication No. 20100135535, hereinafter “Gutierrez”).
Regarding claim 1, Abu discloses the invention as claimed including transforming the graph of the HM into the graph of the GTM through a series of transformation steps, a total number of the transformation steps to transform the graph of the HM into the graph of the GTM being a measure of the accuracy of the HM (see, Abu, paragraph before Figure 1, “An example of an edit path between two graphs g1 and g2 is shown in Figure 1, the following operations have been applied in order to transform g1 into g2 : three edge deletions, one node deletion, one node insertion, one edge insertions and three node substitutions,” [i.e., transforming graph g1 into g2 through a series of steps (i.1., edge deletion, node deletion, node insertion, edge insertion and node substitution) is to determine accuracy – the less steps there are of transformations the more accurate g1 is to g2 because that means g1 is more similar to g2 as opposed to needing more transformation steps for g1 to be g2).
Although Abu substantially discloses the claimed invention, Abu does not explicitly disclose the graph of the HM.
Nevertheless, in the same field, analogous art Geiger teaches the graph of the HM (see, Geiger, e.g., paragraph 142, “The decision of whether to place the edge/segment into the graph and/or the scoring thereof can be based on the information coming from various hypothesis-filtering models” [i.e., graph is based on hypothesis model/HM]).
Abu and Geiger are analogous because they are both directed to graph matching (see, Abu, paragraph before Figure 1 and Geiger, paragraph 142).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu to incorporate the teachings of Geiger to provide placing the edge/segment into the graph and/or the scoring thereof can be based on the information coming from various hypothesis-filtering models. In particular, if the string corresponding to a hypothesis that has been propagated to the node N(t-i, m') which forms a legal prefix of the lexicon if m.sub.j, which is concatenated to thereto, then the new prefix is identified to be legal in the dictionary and the edge/segment is permissible and/or the score can be higher (see Geiger, e.g., paragraph 142).
Although Abu in view of Geiger substantially discloses the claimed invention, Abu in view of Geiger does not explicitly teach A computer-implemented method for evaluating an accuracy of a hypothesis model (HM) against a ground truth model (GTM), the method comprising … the graph of the GTM and A.
Nevertheless, in the same field, analogous art Blackwell teaches A computer-implemented method for evaluating an accuracy of a hypothesis model (HM) against a ground truth model (GTM), the method comprising: both the graph of the GTM and (see, Blackwell, e.g., paragraph 45, “An optimizing operation 204 optimizes accuracy of component attributes of components in the environment by combining the physical model 108 of the environment and the process model 110 of the environment using information from the relational model 112… During graph matching (or other methods of combining the models), the parameters may then be stored in the semantic model 118 as vertex or edge attributes, as appropriate” [i.e., a method evaluating accuracy related to graph matching of two models/graphs] and paragraph 46, “During the graph matching, the process model 110 may be used as ground truth where the vertices and edges of the process model 110 are used as a checklist for or to otherwise verify initial graph matching steps” [i.e., graph is based on ground truth model] and.
Abu, Geiger and Blackwell are analogous because they are each directed to graph matching (see, Abu, paragraph before Figure 1; Gieger, paragraph 142; Blackwell, paragraph 46). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu in view of Geiger to incorporate the teachings of Blackwell to provide having the process model be used as ground truth where the vertices and edges of the process model are used as a checklist for or to otherwise verify initial graph matching steps. In particular, the process model/(ground truth) to provide additional information about the object, or request that the vertex be removed from the semantic model (see Blackwell, e.g., paragraph 46).
Although Abu in view of Geiger and Blackwell substantially teach the claimed invention, Abu in view of Geiger and Blackwell do not appear to explicitly teach obtaining a graph of the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices;
obtaining a graph of the GTM, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices.
Nevertheless, in the same field, analogous art Gutierrez teaches obtaining a graph of the HM, the graph of the HM containing HM constraints vertices associated with HM constraints, HM variables vertices associated with HM variables, and HM edges connecting the HM constraints vertices to the HM variables vertices (see, Gutierrez, e.g., paragraph 8, “scanning the first topology and the second topology, deriving from the first topology a first topology map relating vertices, edges and faces of the first model and from the second topology a second topology map relating vertices, edges and faces of the second model” [i.e., first topology map/graph is affiliated with vertices, edges, and faces of the hypothesis model/second model]);
obtaining a graph of the GTM, the graph of the GTM containing GTM constraints vertices associated with GTM constraints, GTM variables vertices associated with GTM variables, and GTM edges connecting the GTM constraints vertices to the GTM variables vertices (see, Gutierrez, e.g., paragraph 8, “scanning the first topology and the second topology, deriving from the first topology a first topology map relating vertices, edges and faces of the first model and from the second topology a second topology map relating vertices, edges and faces of the second model” [i.e., second topology map/graph is affiliated with vertices, edges, and faces of the ground truth model/second model]).
Abu, Geiger, Blackwell and Gutierrez are analogous because they are each directed to graph matching (see, Abu, paragraph before Figure 1; see, Geiger, paragraph 142; see Blackwell, paragraph 46; see Gutierrez, paragraph 8). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu in view of Geiger and Blackwell to incorporate the teachings of Gutierrez to provide scanning the first topology and the second topology, deriving from the first topology a first topology map relating vertices, edges and faces of the first model and from the second topology a second topology map relating vertices, edges and faces of the second model. In particular, using those mapped vertices to identify all possible mappings between edges of the first and edges of the second model, using those mapped edges to identify all possible mappings between faces of the first and faces of the second model, and using those mapped elements to linearly compare the first and the second model for geometrical sameness. (see Gutierrez, e.g., paragraph 8).
Regarding claim 2, as discussed above, Abu in view of Geiger Blackwell and Gutierrez teaches the method of claim 1.
Abu further discloses wherein transforming the graph of the HM into the graph of the GTM includes automatically transforming the graph of the HM into the graph of the GTM using a graph edit distance computing algorithm using a graph edit distance computing algorithm (see, Abu, first paragraph of 2.1, “Graph edit distance (GED) is a graph matching approach whose concept was first reported in (Sanfeliu and Fu, 1983). The basic idea of GED is to find the best set of transformations that can transform graph g1 into graph g2 by means of edit operations on graph g1. The allowed operations are inserting, deleting and/or substituting vertices and their corresponding edges” [i.e., Transformation of g1 (HM) into g2 (GTM) is through a set of transformations considered as graph matching which is called graph edit distance or GED]).
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Abu, Geiger, Blackwell and Gutierrez as applied to claim 1 above and further in view of non-patent literature Zhang, MengXue et al. (“Interpretable Math Word Problem Solution Generation Via Step-by-step Planning”, cited in applicant’s IDS filed 12/22/2023, hereinafter “Zhang”).
Regarding claim 3, as discussed above, Abu in view of Geiger, Blackwell, and Gutierrez teaches the method of claim 1.
Although Abu in view of Geiger, Blackwell, and Gutierrez substantially teaches the claimed invention, Abu in view of Geiger, Blackwell, and Gutierrez do not explicitly teach further comprising obtaining the HM by inputting a math word problem into a large language model (LLM) to generate the HM.
Nevertheless, in the same field, analogous art Zhang teaches further comprising obtaining the HM by inputting a math word problem into a large language model (LLM) to generate the HM (see, Zhang, e.g., Table 4, “Demonstrations of generated solutions comparing planning-LM and chain-of-thought. Question 1 shows the intermediate step of chain-of-thought has wrong reasoning but still reaches the final answer. Question 2 shows that planning-LM results in a better reasoning strategy since the calculation process is simple and more concrete…” [i.e., The above demonstrates the inputting of a math word problem (1.) and another optimized version of the math word problem (4.) in the language model/LLM]).
Abu, Geiger, Blackwell, Gutierrez and Zhang are analogous because they are each directed to data processing method involving a word problem, LLM, HM, GTM and a transformation to determine an accuracy score (see, Abu, paragraph before Figure 1; see, Geiger, paragraph 142; see Blackwell, paragraph 46; see Gutierrez, paragraph 8; see, Zhang, Table 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu in view of Geiger, Blackwell and Gutierrez to incorporate the teachings of Zhang to provide demonstrating generated solutions comparing planning-LM and chain of thought. Question 1 shows the intermediate step of chain-of-thought has wrong reasoning but still reaches the final answer. Question 2 shows that planning-LM results in a better reasoning strategy since the calculation process is simple and more concrete… In particular, to see that planning-LM/LLM can generate multiple solutions if it predicts a different math operation in the next step (an optimization) compared to the ground truth solution. (see Zhang, e.g., Section 3.6 Quantitative Analysis).
Regarding claim 4, as discussed above, Abu in view of Geiger, Blackwell and Gutierrez teaches the method of claim 3.
Although Abu in view of Geiger, Blackwell, and Gutierrez substantially teaches the claimed invention, Abu in view of Geiger, Blackwell, and Gutierrez do not explicitly teach wherein inputting the math word problem into the LLM includes inputting a math word optimization problem into the LLM.
Nevertheless, in the same field, analogous art Zhang teaches wherein inputting the math word problem into the LLM includes inputting a math word optimization problem into the LLM (see, Zhang, e.g., Table 4, “Demonstrations of generated solutions comparing planning-LM and chain-of-thought. Question 1 shows the intermediate step of chain-of-thought has wrong reasoning but still reaches the final answer. Question 2 shows that planning-LM results in a better reasoning strategy since the calculation process is simple and more concrete…” [i.e., The above demonstrates the inputting of a math word problem (1.) and another optimized version of the math word problem (4.) in the language model/LLM]).
Abu, Geiger, Blackwell, Gutierrez and Zhang are analogous because they are each directed to data processing method involving a word problem, LLM, HM, GTM and a transformation to determine an accuracy score (see, Abu, paragraph before Figure 1; see, Geiger, paragraph 142; see Blackwell, paragraph 46; see Gutierrez, paragraph 8; see, Zhang, Table 4). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu in view of Geiger, Blackwell and Gutierrez to incorporate the teachings of Zhang to provide demonstrating generated solutions comparing planning-LM and chain of thought. Question 1 shows the intermediate step of chain-of-thought has wrong reasoning but still reaches the final answer. Question 2 shows that planning-LM results in a better reasoning strategy since the calculation process is simple and more concrete… In particular, to see that planning-LM/LLM can generate multiple solutions if it predicts a different math operation in the next step (an optimization) compared to the ground truth solution. (see Zhang, e.g., Section 3.6 Quantitative Analysis).
Claims 16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over non-patent literature Abu in view of Sy, Bon et al. (U.S. Publication No. 20020111780, hereinafter “Sy”) and further in view of Ragavan, Harish et al. (U.S. Publication No. 20200349452, hereinafter “Ragavan”).
Regarding claim 16, Abu discloses the invention as claimed including A method comprising: obtaining a minimum number of edit operations to transform the first graph representation into the second graph representation, the minimum number of operations indicating a performance of the machine learning model (see, Abu, Abstract, “Graph edit distance is an error tolerant matching technique emerged as a powerful and flexible graph matching paradigm that can be used to address different tasks in pattern recognition, machine learning and data mining; it represents the minimum-cost sequence of basic edit operations to transform one graph into another by means of insertion, deletion and substitution of vertices and/or edges” [i.e., Minimum amount of edit operations to transform one graph into another indicating maximum accuracy]).
Although Abu substantially discloses the claimed invention, Abu does not explicitly disclose obtaining a second graph representation of a reference optimization problem model based on the optimization problem.
Nevertheless, in the same field, analogous art Sy teaches obtaining a second graph representation of a reference optimization problem model based on the optimization problem (see, Sy, e.g., paragraph 9, “two methods that are constantly being used, and are robust for solving many linear optimization problems” [i.e., data/problem which can be optimized data/problem/linear optimization problem]).
Abu and Sy are analogous because they are both directed to data processing and involving the type of input into the kind of model to produce a graphical result (see, Abu, Abstract see, Sy, paragraph 9). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu to incorporate the teachings of Geiger to provide having two methods that are constantly being used, and are robust for solving many linear optimization problems. In particular, on primal-dual formulation for the interior point method with different variants of search methods for solving non-linear optimization problems. (see Sy, e.g., paragraph 9).
Although Abu in view of Sy substantially teaches the claimed invention, Abu in view of Sy does not explicitly teach obtaining a first graph representation of a hypothesis optimization problem model generated by a machine learning model based on an optimization problem;
Nevertheless, in the same field, analogous art Ragavan teaches obtaining a first graph representation of a hypothesis optimization problem model generated by a machine learning model based on an optimization problem (see, Ragavan, e.g., paragraph 25, “networked computing system for analyzing information stored in a big data repository based on a plurality of predictive models and generating one or more predictive graphs based on the analysis in accordance with one or more example embodiments” [i.e., based on a predictive model/machine learning model that generates a graph]).
Abu, Sy and Ragavan are analogous because they are each directed to data processing and involve the type of input into the kind of model to produce a graphical result (see, Abu, Abstract see, Sy, paragraph 9; Ragavan, paragraph 25). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu in view of Sy to incorporate the teachings of Ragavan to provide networking computing system analyze information stored in a big data repository based on a plurality of predictive models and generating one or more predictive graphs based on the analysis in accordance with one or more example embodiments. In particular, the big data predictive graph computing system may include a data accumulation service, which may be a service implemented on one or more servers. (see Ragavan, e.g., paragraph 25).
Regarding claim 20, as discussed above, Abu in view of Sy, and Ragavan teaches the method of claim 16. Abu further discloses obtaining a total cost for the minimum sequence of edit operations based on a cost value assigned for each edit operation of the minimum sequence of edit operations, the total cost indicative of the performance of the machine learning model (see, Abu, below Definition 2, “Where c denotes the cost function measuring the strength c(e;) of an edit operation e; and y(g1 ,g2) denotes the set of edit paths transforming g1 into g2. A standard set of edit operations is given by insertions, deletions and substitutions of both vertices and edges. We denote the substitution of two vertices u and v by (u --+ v), the deletion of node u by (u --+ £) and the insertion of node v by(£--+ v)” [i.e., A sequence of edit operations/edit paths transformation where the edit operation is measure of strength based off of the cost function]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Abu, Sy and Ragavan as applied to claim 16 above and further in view of Hoffman, Jordan et al. (U.S. Publication No. 20230315532, hereinafter “Hoffmann”), Okuyama et al. (U.S. Publication No. 20220164412, hereinafter “Okuyama”) and Mironica, Ionut et al. (U.S. Publication No. 20210342972, hereinafter “Mironica”) and additionally further in view of Willits, Andra et al. (U.S. Publication No. 20110289081, hereinafter “Willits”).
Regarding claim 17, as discussed above, Abu in view of Sy and Ragavan teaches the method of claim 16.
Although Abu in view of Sy and Ragavan substantially teaches the claimed invention, Abu in view of Sy and Ragavan do not explicitly teach wherein: the machine learning model is a large language model.
Nevertheless, in the same field, analogous art Hoffmann teaches wherein: the machine learning model is a large language model. (see, Hoffmann, e.g., paragraph 82, “Large machine learning models such as large language models (e.g., machine learning models including neural networks that perform language modeling tasks as described above)” [i.e., Machine learning model such as an LLM]).
Abu, Sy, Ragavan and Hoffmann are analogous because they are each directed to determining accuracy of graph transformations generated from word problem inputs into a model (see, Abu, Abstract; see, Sy, paragraph 9; see, Ragavan, paragraph 25 and Hoffmann, paragraph 82). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu, Sy, and Ragavan to incorporate the teachings of Hoffmann to provide “Large machine learning models such as large language models.” In particular, have demonstrated impressive performance on many machine learning tasks (e.g., language modeling tasks) using a variety of training and evaluation protocols including zero-shot, few-shot, and fine-tuning. (see Hoffmann, e.g., paragraph 82).
However, Abu in view of Sy, Ragavan and Hoffmann do not appear to explicitly teach the optimization problem is one of: a linear programming word problem; a mixed integer linear programming problem; a quadratic programming problem; and a quadratically constrained quadratic programming problem;
Nevertheless, in the same field, analogous art Okuyama teaches the optimization problem is one of: a linear programming word problem; a mixed integer linear programming problem; a quadratic programming problem; and a quadratically constrained quadratic programming problem. (see, Okuyama, e.g., paragraph 95, “The model transformation unit 611 transforms the problem data 601 into the quadratic programming format problem data 602, which is the format of the quadratic programming problem” [i.e., The model uses a quadratic programming problem]).
Abu, Sy, Ragavan, Hoffmann and Okuyama are analogous because they are each directed to determining accuracy of graph transformations generated from word problem inputs into a model/LLM (see, Abu, Abstract; see, Sy, paragraph 9; see Ragavan, paragraph 25, Hoffmann paragraph 82 and Okuyama paragraph 95). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu, Sy, Ragavan and Hoffmann to incorporate the teachings of Okuyama to provide the model transformation unit which transforms the problem data into the quadratic programming format problem data, which is the format of the quadratic programming problem. In particular, a function of deriving Equation 11 from Equation 1 to be implemented in the model transformation unit as software or hardware. (see, Okuyama, e.g., paragraph 95).
However, Abu in view of Sy, Ragavan, Hoffmann and Okuyama do not appear to explicitly teach the first graph representation is a first attributed bipartite graph;
the second graph representation is a second attributed bipartite graph.
Nevertheless, in the same field, analogous art Mironica teaches the first graph representation is a first attributed bipartite graph (see, Mironica, e.g., paragraph 63, “On a first side of the bipartite graph, a vertex is created for each target template location. On a second side of the bipartite graph, a vertex is created for each salient shape” [i.e., first and second bipartite graphs are differentiated]);
the second graph representation is a second attributed bipartite graph (see, Mironica, e.g., paragraph 63, “On a first side of the bipartite graph, a vertex is created for each target template location. On a second side of the bipartite graph, a vertex is created for each salient shape” [i.e., first and second bipartite graphs are differentiated]).
Abu, Sy, Ragavan, Hoffmann, Okuyama and Mironica are analogous because they are each directed to determining accuracy of graph transformations on bipartite graphs generated from word problem inputs into a model (see, Abu, Abstract; see, Sy, paragraph 9; see Ragavan, paragraph 25, Hoffmann see paragraph 82, Okuyama see paragraph 95 and Mironica; see paragraph 63). One of ordinary skill in the art would be motivated to combine Mironica with Abu, Sy and Ragavan. Doing so would have allowed Abu, Sy and Ragavan to use Mironica’s example of on a first side of the bipartite graph, a vertex is created for each target template location. On a second side of the bipartite graph, a vertex is created for each salient shape. In particular, use the shape matching system which minimizes the objective function by finding a minimal bipartite matching on the complete bipartite graph. (see, Mironica, e.g., paragraph 63)
However, Abu in view of Sy and Ragavan, Hoffmann, Okuyama, and Mironica do not appear to explicitly teach if and the minimum number of operations is a function of a graph edit distance between the first attributed bipartite graph and the second attributed bipartite graph.
Nevertheless, in the same field, analogous art Willits teaches and the minimum number of operations is a function of a graph edit distance between the first attributed bipartite graph and the second attributed bipartite graph (see, Willits, e.g., paragraph 85, “The semantic score may be calculated in block S716 using mathematical models including Bipartite Graphs and Edit Distance calculations that determine how similar the definitions of the words in the query and the words in the response title are” [i.e., graph edit distances attributed to more than one bipartite graph]).
Abu in view of Sy and Ragavan, Hoffmann, Okuyama, Mironica, and Willits are analogous because they are each directed to determining the accuracy from minimum number of graph transformation steps using GED on bipartite graphs generated from word problem inputs into a model (see, Abu, Abstract; see, Sy, paragraph 9; see Ragavan, paragraph 25, Hoffmann see paragraph 82, Okuyama see paragraph 95 and Mironica; see paragraph 63 and Willits, paragraph 85). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu, Sy, Ragavan, Hoffmann, Okuyama and Mironica to incorporate the teachings of Willits to provide a semantic score that may be calculated in block S716 using mathematical model including Bipartite Graphs and Edit Distance calculations that determine how similar the definitions of the words in the query and the words in the response title are. In particular, two lists, one containing a definition and relations for each word in the query and one containing a definition and relations for each word in the response title. For each word in the query, each element in the response title is compared. This may be done by calculating the similarity between the two definition strings using edit distance (see, Willits, e.g., paragraph 88 and 89).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Abu, Sy and Ragavan as applied to claim 16 above and further in view of Moreau, Samuel et al. (U.S. Publication No. 20210392399, hereinafter “Moreau”).
Regarding claim 18, as discussed above, Abu in view of Sy and Ragavan teaches the method of claim 16.
Although Abu in view of Sy and Ragavan substantially teaches the claimed invention, Abu in view of Sy and Ragavan do not explicitly teach obtaining a first general form of linear programming corresponding to the hypothesis optimization problem model; and
obtaining a second general form of linear programming corresponding to the reference optimization problem model.
Nevertheless, in the same field, analogous art Moreau teaches obtaining a first general form of linear programming corresponding to the hypothesis optimization problem model. (see, Moreau, e.g., paragraph 37, “FIG. 1 illustrates a conceptual model for the blending of linear programming, non-linear programming and managed content in a single user interface or iTV application in accordance with an embodiment of the present invention” [i.e., linear programming corresponds to conceptual model/hypothesis optimization problem model]); and
and obtaining a second general form of linear programming corresponding to the reference optimization problem model (see, Moreau, e.g., paragraph 37, “FIG. 1 illustrates a conceptual model for the blending of linear programming, non-linear programming and managed content in a single user interface or iTV application in accordance with an embodiment of the present invention” [i.e., linear programming corresponds to conceptual model or a type of model such as reference optimization problem model]).
Abu, Sy, Ragavan and Moreau are analogous because they are each directed to determining accuracy of graph transformations on bipartite graphs generated from word problem inputs into a model (see, Abu, Abstract; see, Sy, paragraph 9; see Ragavan, paragraph 25 and Moreau paragraph 37). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu in view of Sy and Ragavan to incorporate the teachings of Moreau to provide a conceptual model for the blending of linear programming, non-linear programming and managed content in a single user interface or iTV application in accordance with an embodiment of the present invention. In particular, model 10 is meant to illustrate the universe (or a portion thereof) of television content offerings within an iTV service. This universe of content has been divided into content categories (News, Sports, Movies, etc.) 12a-12p and each content category may have content elements that are within on-demand, on now or managed content subcategories. (see, Moreau, e.g., paragraph 37).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Abu, Sy and Ragavan as applied to claim 16 above and further in view of Estevez-Schwarz, Diana et al. (U.S. Publication No. 20030084421, hereinafter “Estevez”).
Regarding claim 19, as discussed above, Abu, Sy and Ragavan teaches the method of claim 16.
Although Abu in view of Sy and Ragavan substantially teaches the claimed invention, Abu in view of Sy and Ragavan do not explicitly teach the minimum number of edit operations is a minimum sequence of edit operations to transform the first attributed bipartite to the second attributed bipartite graph;
the minimum sequence of edit operations: relate to one or more of a vertex of the first set of vertices, a vertex of the second set of vertices, an edge of the set of edges
and comprise one or more of: an insertion operation, a deletion operation, and a substitution operation.”
Nevertheless, in the same field, analogous art Estevez teaches the minimum number of edit operations is a minimum sequence of edit operations to transform the first attributed bipartite to the second attributed bipartite graph. (see, Estevez, e.g., paragraph 30, “For each unmatched node of the first bipartite graph corresponding to a modifiable equation, a node is inserted (in each case) into the visibility range of the second bipartite graph which contains the corresponding node from the first bipartite graph.” [i.e., transformation of first bipartite graph into the second bipartite graph”]).
the minimum sequence of edit operations: relate to one or more of a vertex of the first set of vertices, a vertex of the second set of vertices, an edge of the set of edges. (see, Estevez, e.g., paragraph 15, “A bipartite graph is characterized by a large number of nodes and edges. The nodes of the bipartite graph are subdivided into two partitions. Each edge connects precisely one node in the first partition to precisely one further node in the second partition” [i.e., edit operations based on bipartite graph relates to a large number/one or more vertices/nodes and edges]).
and comprise one or more of: an insertion operation, a deletion operation, and a substitution operation. (see, Estevez, e.g., paragraph 31 and paragraph 37, “A node is inserted (in each case) into the second bipartite graph for each unmatched node in the first bipartite graph which corresponds to an unknown” and “In the next step, the edges of matched, modifiable equation nodes from the first bipartite graph are selected and deleted from the first bipartite graph” [i.e., edit operations comprises of deletion and insertion operations]).
Abu, Sy, Ragavan and Estevez are analogous because they are each directed to determining accuracy of graph transformations on bipartite graphs generated from word problem inputs into a model (see, Abu, Abstract; see, Sy, paragraph 9; see Ragavan, paragraph 25 and Estevez 30). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Abu in view of Sy and Ragavan to incorporate the teachings of Estevez to provide each unmatched node of the first bipartite graph corresponding to a modifiable equation, a node is inserted (in each case) into the visibility range of the second bipartite graph which contains the corresponding node from the first bipartite graph. In particular, nodes belonging to the first partition of the second bipartite graph and are designated selector quantity nodes according to one embodiment. (see, Estevez, e.g., paragraph 30). Likewise, It would also have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide a bipartite graph that is characterized by a large number of nodes and edges. The nodes of the bipartite graph are subdivided into two partitions. Each edge connects precisely one node in the first partition to precisely one further node in the second partition. In particular, it states that an unknown represented by the second node is contained in the equation represented by the second node. (see, Estevez, e.g., paragraph 15). Additionally, It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have a node inserted (in each case) into the second bipartite graph for each unmatched node in the first bipartite graph which corresponds to an unknown” and “In the next step, the edges of matched, modifiable equation nodes from the first bipartite graph are selected and deleted from the first bipartite graph. In particular, nodes belonging to the second partition of the second bipartite graph and, according to one embodiment, are designated break quantity nodes and the first bipartite graph reduced in this way has an unmatched modifiable equation node and an additional unmatched unknown node. (see, Estevez, e.g., paragraph 31 and 38).
Conclusion
The prior art made of record, listed on form PTO-892, and not relied upon, is considered pertinent to applicant's disclosure.
The references listed on form PTO-892 are all generally related to techniques, methods and systems for selecting sets of data used in models and neural networks, such as transforming graphs and models.
For example, Travalini; Michael et al. (U.S. Publication No. 20230259821, hereinafter “Travalini”) discloses “Moreover, the problem model may use at least one large language model (LLM) (e.g., BERT, GPT, Dialogflow, or the like) to classify the data within the one or more inputs, to generate a vectorization structure that may be used to progress through the node graph” [i.e., uses an LLM to input data/input/math word problem and generate a node graph for a model] (see e.g., paragraph 129).
Also, for example, Klein, Dana et al. (U.S. Publication No. 20150025846, hereinafter “Klein”) discloses “For example, the robustness score may be related to a variance or any other statistical measure of the plurality of values of the at least one accuracy score” [i.e., a further accuracy score] (see e.g., paragraph 44).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN PENG whose telephone number is (571)270-0897. The examiner can normally be reached Monday - Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEVEN PENG/Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
1 Under the BRI, in view of the specification, further comprising obtaining the HM by inputting a math word problem into a large language model (LLM) to generate the HM” has been interpreted as creating the HM by inputting a math word problem into a large language model (LLM)”
2 As noted above in the objection to this claim, “using the second set of training dataset” should recite “using the second set of optimization problems of the training dataset”
Prosecution Insights