DETAILED ACTION
Claims 1-6, 8-14, and 16-22 have been examined.
Claim Objections
Claims 1-6, 8-14, and 16-22 are objected to because they are dependent on rejected claims Appropriate correction is required.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The invention, as taught in Claims 1-6, 8-14, and 16-22, is directed to “mental steps” and “mathematical steps” without significantly more.
The claims recite:
• identifying a first feature (i.e., a mental step)
• determining one or more first trained ML models that are using the first feature (the ML models are already trained) (i.e., a mental step)
• generating metrics to determine an accuracy of the first trained ML model, the metrics corresponding to each first trained ML model and comprising one or
more of an area under a receiver operating characteristic curve, precision or
recall (i.e., a mathematical step)
Claim 1
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “1. A method of detecting data drift associated with machine learning (ML) models, the method comprising…” Therefore, it is a “method” (or “process”), which is a statutory category of invention. Therefore, the answer to the inquiry is: “YES.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 1 that recite abstract ideas?
YES. The following limitations in Claim 1 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• identifying a first feature (i.e., a mental step)
• determining one or more first trained ML models that are using the first feature (the ML models are already trained) (i.e., a mental step)
• generating metrics to determine an accuracy of the first trained ML model, the metrics corresponding to each first trained ML model and comprising one or
more of an area under a receiver operating characteristic curve, precision or
recall (i.e., a mathematical step)
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) An “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data”
(2) A “storing” by a “feature store”/“the feature store comprises an offline store and an online store”
(3) A “generating” of “an alert”
A “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
This “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
A “storing” by a “feature store”/“the feature store comprises an offline store and an online store” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
This “storing” by a “feature store”/“the feature store comprises an offline store and an online store” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
A “generating” of “an alert” (i.e., data display) is a broad term which is described at a high level. M.P.E.P. § 2106.05 (h) recites in part:
Examples of limitations that the courts have described as merely indicating a field of use or technological environment in which to apply a judicial exception include:
***
vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016);
This “generating” of “an alert” (i.e., data display) limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) An “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data”
(2) A “storing” by a “feature store”/“the feature store comprises an offline store and an online store”
(3) A “generating” of “an alert”
A “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “storing” by a “feature store”/“the feature store comprises an offline store and an online store” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “generating” of “an alert” (i.e., data display) is a broad term which is described at a high level. M.P.E.P. § 2106.05 (h) recites in part:
Examples of limitations that the courts have described as merely indicating a field of use or technological environment in which to apply a judicial exception include:
***
vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016);
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 1 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 2
Claim 2 recites:
2. (Original) The method of claim 1, further comprising:
for a second feature, determining new feature values ingested by the offline store of the feature store;
determining an occurrence of a second data drift between the new feature values and previous corresponding feature values; and
in response to the determining, labeling the second feature and preventing the second feature from being used to train a second ML model.
Applicant’s Claim 2 merely teaches the mental steps of determinations, labeling, and “preventing” (which is elimination of selected data). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 2 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 3
Claim 3 recites:
3. (Original) The method of claim 2, wherein the determining the second data drift comprises determining mean, median and mode between the new feature values and previous corresponding feature values.
Applicant’s Claim 3 merely teaches the mental step (or math step) of “determining” (or, calculating) “mean, median and mode between the new feature values and previous corresponding feature values”. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 3 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 4
Claim 4 recites:
4. The method of claim 1, wherein invoking the first trained ML model comprises accessing a representational state transfer application programming interface (REST API) server with an inference request.
Applicant’s Claim 4 merely teaches an inference request (i.e., a mental step). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 4 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 5
Claim 5 recites:
5. (Original) The method of claim 2, further comprising converting data from one or more data sources into the second feature.
Applicant’s Claim 5 merely teaches the mental step of data conversion. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 5 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 6
Claim 6 recites:
6. (Original) The method of claim 2, wherein the labeling the second feature is implemented by the feature store.
Applicant’s Claim 6 merely teaches a mental step to use a data collection to make labels. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 6 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 7
Claim 7 recites:
7. The method of claim 1, wherein the metrics comprise determining if an area under an ROC curve is below a threshold.
Applicant’s Claim 7 merely teaches a mathematical calculation (i.e., integration). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 7 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 8
Claim 8 recites:
8. (Original) The method of claim 2, the preventing the second feature from being used to train the second ML model comprising a gate between the offline store and the second ML model.
Applicant’s Claim 8 merely teaches the mental step of preventing data from being included in a larger data set. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 8 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 9
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “9. A computer readable medium having instructions stored thereon that, when executed by one or more processors, cause the processors to detect data drift associated with machine learning (ML) models, the detecting comprising…” Therefore, it is not a “non-transitory computer-readable medium” (or “product of manufacture”), which is not a statutory category of invention. Therefore, the answer to the inquiry is: “NO.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 9 that recite abstract ideas?
YES. The following limitations in Claim 9 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• identifying a first feature (i.e., a mental step)
• determining one or more first trained ML models that are using the first feature (the ML models are already trained) (i.e., a mental step)
• generating metrics to determine an accuracy of the first trained ML model,
the metrics corresponding to each first trained ML model and comprising one or
more of an area under a receiver operating characteristic curve, precision or
recall (i.e., a mathematical step)
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) An “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data”
(2) A “storing” by a “feature store”/“the feature store comprises an offline store and an online store”
(3) A “generating” of “an alert”
A “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
This “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
A “storing” by a “feature store”/“the feature store comprises an offline store and an online store” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
This “storing” by a “feature store”/“the feature store comprises an offline store and an online store” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
A “generating” of “an alert” (i.e., data display) is a broad term which is described at a high level. M.P.E.P. § 2106.05 (h) recites in part:
Examples of limitations that the courts have described as merely indicating a field of use or technological environment in which to apply a judicial exception include:
***
vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016);
This “generating” of “an alert” (i.e., data display) limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) An “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data”
(2) A “storing” by a “feature store”/“the feature store comprises an offline store and an online store”
(3) A “generating” of “an alert”
A “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “storing” by a “feature store”/“the feature store comprises an offline store and an online store” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “generating” of “an alert” (i.e., data display) is a broad term which is described at a high level. M.P.E.P. § 2106.05 (h) recites in part:
Examples of limitations that the courts have described as merely indicating a field of use or technological environment in which to apply a judicial exception include:
***
vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016);
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 9 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 10
Claim 10 recites:
10. (Original) The computer readable medium of claim 9, the detecting further comprising:
for a second feature, determining new feature values ingested by the offline store of the feature store;
determining an occurrence of a second data drift between the new feature values and previous corresponding feature values; and
in response to the determining, labeling the second feature and preventing the second feature from being used to train a second ML model.
Applicant’s Claim 10 merely teaches the mental steps of determinations, labeling, and “preventing” (which is elimination of selected data). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 10 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 11
Claim 11 recites:
11. (Original) The computer readable medium of claim 10, wherein the determining the second data drift comprises determining mean, median and mode between the new feature values and previous corresponding feature values.
Applicant’s Claim 11 merely teaches the mental step (or math step) of “determining” (or, calculating) “mean, median and mode between the new feature values and previous corresponding feature values”. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 11 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 12
Claim 12 recites:
12. The computer readable medium of claim 9, wherein invoking the first trained ML model comprises accessing a representational state transfer application programing interface (REST API) server with an inference request.
Applicant’s Claim 12 merely teaches an inference request (i.e., a mental step). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 12 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 13
Claim 13 recites:
13. (Original) The computer readable medium of claim 10, the detecting further comprising converting data from one or more data sources into the second feature.
Applicant’s Claim 13 merely teaches the mental step of data conversion. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 13 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 14
Claim 14 recites:
14. (Original) The computer readable medium of claim 10, wherein the labeling the second feature is implemented by the feature store.
Applicant’s Claim 14 merely teaches a mental step to use a data collection to make labels. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 14 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 15
Claim 15 recites:
15. The computer readable medium of claim 9, wherein the metrics comprise determining if an area under an ROC curve is below a threshold.
Applicant’s Claim 15 merely teaches a mathematical calculation (i.e., integration). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 15 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 16
Claim 16 recites:
16. (Original) The computer readable medium of claim 10, the preventing the second feature from being used to train the second ML model comprising a gate between the offline store and the second ML model.
Applicant’s Claim 16 merely teaches the mental step of preventing data from being included in a larger data set. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 16 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 17
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “17. A cloud infrastructure comprising…,” which is not one of the four required patentable subject matter categories. That is, it is not a “process,” nor is it a “computer readable medium” (i.e., “product of manufacture”), nor is it a “composition of matter,” nor is it an “apparatus”.
Therefore, it fails to recite a statutory category of invention. Therefore, the answer to the inquiry is: “NO.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 17 that recite abstract ideas?
YES. The following limitations in Claim 17 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• identifying a first feature (i.e., a mental step)
• determining one or more first trained ML models that are using the first feature (the ML models are already trained) (i.e., a mental step)
• generating metrics to determine an accuracy of the first trained ML model,
the metrics corresponding to each first trained ML model and comprising one or
more of an area under a receiver operating characteristic curve, precision or
recall (i.e., a mathematical step)
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) An “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data”
(2) A “storing” by a “feature store”/“the feature store comprises an offline store and an online store”
(3) A “generating” of “an alert”
A “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
This “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
A “storing” by a “feature store”/“the feature store comprises an offline store and an online store” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
This “storing” by a “feature store”/“the feature store comprises an offline store and an online store” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
A “generating” of “an alert” (i.e., data display) is a broad term which is described at a high level. M.P.E.P. § 2106.05 (h) recites in part:
Examples of limitations that the courts have described as merely indicating a field of use or technological environment in which to apply a judicial exception include:
***
vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016);
This “generating” of “an alert” (i.e., data display) limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) An “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data”
(2) A “storing” by a “feature store”/“the feature store comprises an offline store and an online store”
(3) A “generating” of “an alert”
A “invoking” (i.e., execution on a processor) of “the first trained ML model using synthetic data or validation data” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “storing” by a “feature store”/“the feature store comprises an offline store and an online store” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “generating” of “an alert” (i.e., data display) is a broad term which is described at a high level. M.P.E.P. § 2106.05 (h) recites in part:
Examples of limitations that the courts have described as merely indicating a field of use or technological environment in which to apply a judicial exception include:
***
vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016);
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 17 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 18
Claim 18 recites:
18. (Original) The cloud infrastructure of claim 17, the detecting further comprising:
for a second feature, determining new feature values ingested by the offline store of the feature store;
determining an occurrence of a second data drift between the new feature values and previous corresponding feature values; and
in response to the determining, labeling the second feature and preventing the second feature from being used to train a second ML model.
Applicant’s Claim 18 merely teaches the mental steps of determinations, labeling, and “preventing” (which is elimination of selected data). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 18 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 19
Claim 19 recites:
19. (Original) The cloud infrastructure of claim 18, wherein the determining the second data drift comprises determining mean, median and mode between the new feature values and previous corresponding feature values.
Applicant’s Claim 19 merely teaches the mental step (or math step) of “determining” (or, calculating) “mean, median and mode between the new feature values and previous corresponding feature values”. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 19 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 20
Claim 20 recites:
20. The cloud infrastructure of claim 17, wherein invoking the first trained ML model comprises accessing a representational state transfer application programming interface (REST API) server with an inference request.
Applicant’s Claim 20 merely teaches an inference request (i.e., a mental step). It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 20 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 21
Claim 21 recites:
21. (New) The cloud infrastructure of claim 18, the preventing the second feature from being used to train the second ML model comprising a gate between the offline store and the second ML model.
Applicant’s Claim 21 merely teaches the mental step of preventing data from being included in a larger data set. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 21 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 22
Claim 22 recites:
22. (New) The cloud infrastructure of claim 17, further comprising:
a first virtual cloud network (VCN) comprising a local peering gateway (LPG) communicatively coupled to a secure shell (SSH) VCN via the LPG;
wherein the LPG is contained in a control plane VCN and the SSH VCN is communicatively coupled to a data plane VCN.
Applicant’s Claim 22 merely teaches no step of storage, so the claimed storage is only generally linked to the abstract idea. Note that M.P.E.P. § 2106.05(h) recites:
2106.05(h) Field of Use and Technological Environment [R-10.2019]
Another consideration when determining whether a claim integrates the judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible “simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use.” Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
The courts often cite to Parker v. Flook as providing a classic example of a field of use limitation. See, e.g., Bilski v. Kappos, 561 U.S. 593, 612, 95 USPQ2d 1001, 1010 (2010) (“Flook established that limiting an abstract idea to one field of use or adding token postsolution components did not make the concept patentable”) (citing Parker v. Flook, 437 U.S. 584, 198 USPQ 193 (1978)). In Flook, the claim recited steps of calculating an updated value for an alarm limit (a numerical limit on a process variable such as temperature, pressure or flow rate) according to a mathematical formula “in a process comprising the catalytic chemical conversion of hydrocarbons.” 437 U.S. at 586, 198 USPQ at 196. Processes for the catalytic chemical conversion of hydrocarbons were used in the petrochemical and oil-refining fields. Id. Although the applicant argued that limiting the use of the formula to the petrochemical and oil-refining fields should make the claim eligible because this limitation ensured that the claim did not preempt all uses of the formula, the Supreme Court disagreed. 437 U.S. at 588-90, 198 USPQ at 197-98. Instead, the additional element in Flook regarding the catalytic chemical conversion of hydrocarbons was not sufficient to make the claim eligible, because it was merely an incidental or token addition to the claim that did not alter or affect how the process steps of calculating the alarm limit value were performed. Further, the Supreme Court found that this limitation did not amount to an inventive concept. 437 U.S. at 588-90, 198 USPQ at 197-98. The Court reasoned that to hold otherwise would “exalt[] form over substance”, because a competent claim drafter could attach a similar type of limitation to almost any mathematical formula. 437 U.S. at 590, 198 USPQ at 197.
In contrast, the additional elements in Diamond v. Diehr as a whole provided eligibility and did not merely recite calculating a cure time using the Arrhenius equation “in a rubber molding process”. Instead, the claim in Diehr recited specific limitations such as monitoring the elapsed time since the mold was closed, constantly measuring the temperature in the mold cavity, repetitively calculating a cure time by inputting the measured temperature into the Arrhenius equation, and opening the press automatically when the calculated cure time and the elapsed time are equivalent. 450 U.S. at 179, 209 USPQ at 5, n. 5. These specific limitations act in concert to transform raw, uncured rubber into cured molded rubber. 450 U.S. at 177-78, 209 USPQ at 4.
Secondly, the claimed “feature store” is well-understood, routine and conventional. Note that paragraph [0032] teaches that the claimed “features” are “any measurable input that can be used in a predictive model (i.e., any type of ML or artificial intelligence model)”:
[0032] Feature store 10, in general, is a data management layer for machine learning that allows to users to share discovered/generated features and create more effective machine learning pipelines. Feature store 50, in embodiments, further includes a data drift layer that provides that data drift detection functionality disclosed herein. Features are considered any measurable input that can be used in a predictive model (i.e., any type of ML or artificial intelligence model). For example, a recommendation application may use the total amount per purchase or product category as one of its many features. Features are used to train ML models and make predictions. In general, the more data, the better the predictions.
Further, paragraph [0068] teaches that the “ML data” are hosted/stored in a cloud architecture:
[0068] Figs. 5-8 illustrate an example cloud infrastructure that can incorporate the secure on-premises to cloud connector framework system in accordance to embodiments. The cloud infrastructure of Fig. 5-8 can be used to implement network/cloud 104 of Fig. 1 and host ML data and inference drift detection layer system 10.
Further, paragraph [0068] teaches that the cloud structure has many generic “infrastructure components” such as “storage devices…, or the like”.
[0069] As disclosed above, infrastructure as a service ("laaS") is one particular type of cloud computing. laaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In an laaS model, a cloud computing provider can host the infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., a hypervisor layer), or the like). In some cases, an laaS provider may also supply a variety of services to accompany those infrastructure components (e.g., billing, monitoring, logging, security, load balancing and clustering, etc.). Thus, as these services may be policy-driven, laaS users may be able to implement policies to drive load balancing to maintain application availability and performance.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 22 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 9, and 17 are rejected under 35 U.S.C. § 103 as being unpatentable over Breck, et al., Data Validation for Machine Learning, Proceedings of the 2nd SysML Conference, 2019, pp. 1-14, in view of Davis, et al., Calibration Drift Among Regression and Machine Learning Models for Hospital Mortality, AMIA Annu Symp Proc., 16 APR 2018, pp. 625–634, in their entireties. Specifically:
Claim 1
Claim 1’s “1. A method of detecting data drift associated with machine learning (ML) models, the method comprising:” is taught by Breck, et al., page 6, left column, first full paragraph, where it recites:
There are certain anomalies that only manifest when two or more batches of data are considered together, e.g., drift in the distribution of feature values across multiple batches of data. In this section, we first cover the different types of such anomalies, discuss the reasons why they occur, and finally present some common techniques that can be used to detect them.
Claim 1’s “identifying a first feature stored by a feature store, wherein the feature store comprises an offline store and an online store;” is taught by Breck, et al., page 6, left column, second full paragraph, where it recites:
Training-Serving Skew One of the issues that frequently occurs in production pipelines is skew between training and serving data. Several factors contribute to this issue but the most common is different code paths used for generation of training and serving data. These different code paths are required due to widely different latency and throughput characteristics of offline training data generation versus online serving data generation.
Claim 1’s “determining one or more first trained ML models that are using the first feature;” is taught by Breck, et al., page 10, left column, first full paragraph, where it recites:
Stand-alone data validation. Some product teams are setting up ML pipelines solely for the purpose of running our data-validation system, without doing any training or serving of models. These teams have existing infrastructure (predating the ML platform) to train and serve models, but they are lacking a systematic mechanism for data validation and, as a result, they are suffering from the data-related issues that we described earlier in the paper. To address this shortcoming in their infrastructure, these teams have set up pipelines that solely monitor the training and serving data and alert the on-call when an error is detected (while training and serving still happen using the existing infrastructure).
The prior art ML must use at least one feature to operate. That one feature would be the first feature.
Claim 1’s “invoking the first trained ML model using synthetic data or validation data;” is taught by Breck, et al., page 8, left column, second full paragraph, where it recites:
In fact, it has worked so well that we have packaged this type of testing as a unit test over training algorithms, and included the test in the standard templates of our ML platform. Our users routinely execute these unit tests to validate changes to the training logic of their pipelines. To our knowledge, this application of unit testing in the context of ML and for the purpose of data validation is a novel aspect of our work.
Claim 1’s “generating metrics to determine an accuracy of the first trained ML model; and” is taught by Breck, et al., page 6, right column, second full paragraph, where it recites:
In addition to validating individual batches of training data, the Data Validator also monitors for skew between training and serving data, continuously.
Claim 1’s “when the accuracy is below a threshold, generating an alert notifying of a first data drift for the first trained ML model.” is taught by Breck, et al., page 6, right column, second full paragraph, where it recites:
Any detected skew is summarized and presented to the user using the ML platform’s standard alerting infrastructure.
Applicant did not specify a particular threshold range. Therefore “Any detected skew…” is an accuracy measure that anticipates the claimed threshold.
Claim 1’s “the metrics corresponding to each first trained ML model and comprising one or more of an area under a receiver operating characteristic curve, precision or recall” is not expressly taught by Breck, et al. It is, however, taught by Davis, et al., page 626, first full paragraph, where it recites:
We divided the 7-year validation set into consecutive 3-month periods (n=28) and assessed performance of the models within each. Discrimination was measured with the area under the receiver operating characteristics curve (AUC).
Rationale – it would have been obvious to one of ordinary skill in the art, at the time of the effective filing date, to combine the machine learning system of Breck, et al. with the ROC metric of Davis, et al. because it gives an accurate measure of the discrimination capacity of the machine learning system.
Claim 9
Claim 9’s “identifying a first feature stored by a feature store, wherein the feature store comprises an offline store and an online store;” is taught by Breck, et al., page 6, left column, second full paragraph, where it recites:
Training-Serving Skew One of the issues that frequently occurs in production pipelines is skew between training and serving data. Several factors contribute to this issue but the most common is different code paths used for generation of training and serving data. These different code paths are required due to widely different latency and throughput characteristics of offline training data generation versus online serving data generation.
Claim 9’s “determining one or more first trained ML models that are using the first feature;” is taught by Breck, et al., page 10, left column, first full paragraph, where it recites:
Stand-alone data validation. Some product teams are setting up ML pipelines solely for the purpose of running our data-validation system, without doing any training or serving of models. These teams have existing infrastructure (predating the ML platform) to train and serve models, but they are lacking a systematic mechanism for data validation and, as a result, they are suffering from the data-related issues that we described earlier in the paper. To address this shortcoming in their infrastructure, these teams have set up pipelines that solely monitor the training and serving data and alert the on-call when an error is detected (while training and serving still happen using the existing infrastructure).
The prior art ML must use at least one feature to operate. That one feature would be the first feature.
Claim 9’s “invoking the first trained ML model using synthetic data or validation data;” is taught by Breck, et al., page 8, left column, second full paragraph, where it recites:
In fact, it has worked so well that we have packaged this type of testing as a unit test over training algorithms, and included the test in the standard templates of our ML platform. Our users routinely execute these unit tests to validate changes to the training logic of their pipelines. To our knowledge, this application of unit testing in the context of ML and for the purpose of data validation is a novel aspect of our work.
Claim 9’s “generating metrics to determine an accuracy of the first trained ML model; and” is taught by Breck, et al., page 6, right column, second full paragraph, where it recites:
In addition to validating individual batches of training data, the Data Validator also monitors for skew between training and serving data, continuously.
Claim 9’s “when the accuracy is below a threshold, generating an alert notifying of a first data drift for the first trained ML model.” is taught by Breck, et al., page 6, right column, second full paragraph, where it recites:
Any detected skew is summarized and presented to the user using the ML platform’s standard alerting infrastructure.
Applicant did not specify a particular threshold range. Therefore “Any detected skew…” is an accuracy measure that anticipates the claimed threshold.
Claim 9’s “the metrics corresponding to each first trained ML model and comprising one or more of an area under a receiver operating characteristic curve, precision or recall” is not expressly taught by Breck, et al. It is, however, taught by Davis, et al., page 626, first full paragraph, where it recites:
We divided the 7-year validation set into consecutive 3-month periods (n=28) and assessed performance of the models within each. Discrimination was measured with the area under the receiver operating characteristics curve (AUC).
Rationale – it would have been obvious to one of ordinary skill in the art, at the time of the effective filing date, to combine the machine learning system of Breck, et al. with the ROC metric of Davis, et al. because it gives an accurate measure of the discrimination capacity of the machine learning system.
Claim 17
Claim 17’s “a plurality of machine learning (ML) models;” is taught by Breck, et al., page 10, left column, first full paragraph, where it recites:
Stand-alone data validation. Some product teams are setting up ML pipelines solely for the purpose of running our data-validation system, without doing any training or serving of models. These teams have existing infrastructure (predating the ML platform) to train and serve models, but they are lacking a systematic mechanism for data validation and, as a result, they are suffering from the data-related issues that we described earlier in the paper. To address this shortcoming in their infrastructure, these teams have set up pipelines that solely monitor the training and serving data and alert the on-call when an error is detected (while training and serving still happen using the existing infrastructure).
The prior art ML must use at least one feature to operate. That one feature would be the first feature.
Claim 17’s “a feature store comprising an offline store and an online store;” is taught by Breck, et al., page 6, left column, second full paragraph, where it recites:
Training-Serving Skew One of the issues that frequently occurs in production pipelines is skew between training and serving data. Several factors contribute to this issue but the most common is different code paths used for generation of training and serving data. These different code paths are required due to widely different latency and throughput characteristics of offline training data generation versus online serving data generation.
Claim 17’s “a data drift layer coupled to the feature store configured to detect data drift associated with the ML models, the detecting comprising:” is taught by Breck, et al., page 6, left column, second full paragraph, where it recites:
Training-Serving Skew One of the issues that frequently occurs in production pipelines is skew between training and serving data. Several factors contribute to this issue but the most common is different code paths used for generation of training and serving data. These different code paths are required due to widely different latency and throughput characteristics of offline training data generation versus online serving data generation.
Claim 17’s “identifying a first feature stored by a feature store, wherein the feature store comprises an offline store and an online store;” is taught by Breck, et al., page 6, left column, second full paragraph, where it recites:
Training-Serving Skew One of the issues that frequently occurs in production pipelines is skew between training and serving data. Several factors contribute to this issue but the most common is different code paths used for generation of training and serving data. These different code paths are required due to widely different latency and throughput characteristics of offline training data generation versus online serving data generation.
Claim 17’s “determining one or more first trained ML models that are using the first feature; ” is taught by Breck, et al., page 10, left column, first full paragraph, where it recites:
Stand-alone data validation. Some product teams are setting up ML pipelines solely for the purpose of running our data-validation system, without doing any training or serving of models. These teams have existing infrastructure (predating the ML platform) to train and serve models, but they are lacking a systematic mechanism for data validation and, as a result, they are suffering from the data-related issues that we described earlier in the paper. To address this shortcoming in their infrastructure, these teams have set up pipelines that solely monitor the training and serving data and alert the on-call when an error is detected (while training and serving still happen using the existing infrastructure).
The prior art ML must use at least one feature to operate. That one feature would be the first feature.
Claim 17’s “invoking the first trained ML model using synthetic data or validation data;” is taught by Breck, et al., page 8, left column, second full paragraph, where it recites:
In fact, it has worked so well that we have packaged this type of testing as a unit test over training algorithms, and included the test in the standard templates of our ML platform. Our users routinely execute these unit tests to validate changes to the training logic of their pipelines. To our knowledge, this application of unit testing in the context of ML and for the purpose of data validation is a novel aspect of our work.
Claim 17’s “generating metrics to determine an accuracy of the first trained ML model; and” is taught by Breck, et al., page 6, right column, second full paragraph, where it recites:
In addition to validating individual batches of training data, the Data Validator also monitors for skew between training and serving data, continuously.
Claim 17’s “when the accuracy is below a threshold, generating an alert notifying of a first data drift for the first trained ML model.” is taught by Breck, et al., page 6, right column, second full paragraph, where it recites:
Any detected skew is summarized and presented to the user using the ML platform’s standard alerting infrastructure.
Applicant did not specify a particular threshold range. Therefore “Any detected skew…” is an accuracy measure that anticipates the claimed threshold.
Claim 17’s “the metrics corresponding to each first trained ML model and comprising one or more of an area under a receiver operating characteristic curve, precision or recall” is not expressly taught by Breck, et al. It is, however, taught by Davis, et al., page 626, first full paragraph, where it recites:
We divided the 7-year validation set into consecutive 3-month periods (n=28) and assessed performance of the models within each. Discrimination was measured with the area under the receiver operating characteristics curve (AUC).
Rationale – it would have been obvious to one of ordinary skill in the art, at the time of the effective filing date, to combine the machine learning system of Breck, et al. with the ROC metric of Davis, et al. because it gives an accurate measure of the discrimination capacity of the machine learning system.
Response to Arguments
Applicant's arguments filed 15 DEC 2025 have been fully considered but they are not persuasive. Specifically, Applicant argues:
Argument 1
The present specification discloses the technical solution of the present invention concerning data drift in ML models, in comparison to known solutions:
In contrast to embodiments of the invention, known systems are generally reactive. For example, a new model may be trained with some drifted data and deployed in production. A user will begin using it, and then the system will detect the performance is degraded and take action. In these known systems, the users first need to invoke the inference endpoint to detect the possible degraded performance of the model due to drifted trained data, which is after the prediction event happened. In contrast, with embodiments, even if users are not using a drifted model, inference detector 320 will detect it proactively, even though no user has yet to invoke the endpoint.
See specification at paragraph [0061].
Firstly, the limitations of the Specification cannot be read into the claim.
Secondly, detection by inference is a mental step.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 2
The Claims Recite Non-Conventional Elements
The present claims recite non-conventional elements. These additional elements are not conventional elements, and consistent with Berkheimer v. HP Inc., 881 F.3d 1360, 1369 (Fed. Cir. 2018), should be considered another reason that the claims are subject matter eligible. For example, the independent claims recite "a feature store, wherein the feature store comprises an offline store and an online store." At least claim 8 recites "a gate between the offline store and the second ML model." Claim 22 recites "a first virtual cloud network (VCN) comprising a local peering gateway (LPG) communicatively coupled to a secure shell (SSH) VCN via the LPG; wherein the LPG is contained in a control plane VCN and the SSH VCN is communicatively coupled to a data plane VCN."
Firstly, for Claim 1, there is no step of storage, so the claimed storage is only generally linked to the abstract idea. Note that M.P.E.P. § 2106.05(h) recites:
2106.05(h) Field of Use and Technological Environment [R-10.2019]
Another consideration when determining whether a claim integrates the judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible “simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use.” Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
The courts often cite to Parker v. Flook as providing a classic example of a field of use limitation. See, e.g., Bilski v. Kappos, 561 U.S. 593, 612, 95 USPQ2d 1001, 1010 (2010) (“Flook established that limiting an abstract idea to one field of use or adding token postsolution components did not make the concept patentable”) (citing Parker v. Flook, 437 U.S. 584, 198 USPQ 193 (1978)). In Flook, the claim recited steps of calculating an updated value for an alarm limit (a numerical limit on a process variable such as temperature, pressure or flow rate) according to a mathematical formula “in a process comprising the catalytic chemical conversion of hydrocarbons.” 437 U.S. at 586, 198 USPQ at 196. Processes for the catalytic chemical conversion of hydrocarbons were used in the petrochemical and oil-refining fields. Id. Although the applicant argued that limiting the use of the formula to the petrochemical and oil-refining fields should make the claim eligible because this limitation ensured that the claim did not preempt all uses of the formula, the Supreme Court disagreed. 437 U.S. at 588-90, 198 USPQ at 197-98. Instead, the additional element in Flook regarding the catalytic chemical conversion of hydrocarbons was not sufficient to make the claim eligible, because it was merely an incidental or token addition to the claim that did not alter or affect how the process steps of calculating the alarm limit value were performed. Further, the Supreme Court found that this limitation did not amount to an inventive concept. 437 U.S. at 588-90, 198 USPQ at 197-98. The Court reasoned that to hold otherwise would “exalt[] form over substance”, because a competent claim drafter could attach a similar type of limitation to almost any mathematical formula. 437 U.S. at 590, 198 USPQ at 197.
In contrast, the additional elements in Diamond v. Diehr as a whole provided eligibility and did not merely recite calculating a cure time using the Arrhenius equation “in a rubber molding process”. Instead, the claim in Diehr recited specific limitations such as monitoring the elapsed time since the mold was closed, constantly measuring the temperature in the mold cavity, repetitively calculating a cure time by inputting the measured temperature into the Arrhenius equation, and opening the press automatically when the calculated cure time and the elapsed time are equivalent. 450 U.S. at 179, 209 USPQ at 5, n. 5. These specific limitations act in concert to transform raw, uncured rubber into cured molded rubber. 450 U.S. at 177-78, 209 USPQ at 4.
Secondly, the claimed “feature store” is well-understood, routine and conventional. Note that paragraph [0032] teaches that the claimed “features” are “any measurable input that can be used in a predictive model (i.e., any type of ML or artificial intelligence model)”:
[0032] Feature store 10, in general, is a data management layer for machine learning that allows to users to share discovered/generated features and create more effective machine learning pipelines. Feature store 50, in embodiments, further includes a data drift layer that provides that data drift detection functionality disclosed herein. Features are considered any measurable input that can be used in a predictive model (i.e., any type of ML or artificial intelligence model). For example, a recommendation application may use the total amount per purchase or product category as one of its many features. Features are used to train ML models and make predictions. In general, the more data, the better the predictions.
Further, paragraph [0068] teaches that the “ML data” are hosted/stored in a cloud architecture:
[0068] Figs. 5-8 illustrate an example cloud infrastructure that can incorporate the secure on-premises to cloud connector framework system in accordance to embodiments. The cloud infrastructure of Fig. 5-8 can be used to implement network/cloud 104 of Fig. 1 and host ML data and inference drift detection layer system 10.
Further, paragraph [0068] teaches that the cloud structure has many generic “infrastructure components” such as “storage devices…, or the like”.
[0069] As disclosed above, infrastructure as a service ("laaS") is one particular type of cloud computing. laaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In an laaS model, a cloud computing provider can host the infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., a hypervisor layer), or the like). In some cases, an laaS provider may also supply a variety of services to accompany those infrastructure components (e.g., billing, monitoring, logging, security, load balancing and clustering, etc.). Thus, as these services may be policy-driven, laaS users may be able to implement policies to drive load balancing to maintain application availability and performance.
Regarding Claim 8, paragraph [0050] teaches that the term “gate” is actually a “quality gate”.
[0050] In one embodiment, inference detector 320 generates some synthetic data for a particular feature. Synthetic data is artificial data that mimics real-world observations and is used to train machine learning models when actual data is difficult or expensive to get. In embodiments, a "python" package can be used to generate synthetic data from real data used as a reference. In other embodiments, inference detector 320 gets a validation data set from offline store 312 to use (i.e., data that passes through quality gate 351 by drift detector 312).
A “quality gate” is known in the art as a checkpoint in software development where specific quality criteria are checked. The checking in this process may also be a determination made by a human…and is, therefore, an abstract mental step.
Regarding Clam 22, its components are “generally linked” to the abstract idea. M.P.E.P. § 2106.05(h) recites:
2106.05(h) Field of Use and Technological Environment [R-10.2019]
Another consideration when determining whether a claim integrates the judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible “simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use.” Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
The courts often cite to Parker v. Flook as providing a classic example of a field of use limitation. See, e.g., Bilski v. Kappos, 561 U.S. 593, 612, 95 USPQ2d 1001, 1010 (2010) (“Flook established that limiting an abstract idea to one field of use or adding token postsolution components did not make the concept patentable”) (citing Parker v. Flook, 437 U.S. 584, 198 USPQ 193 (1978)). In Flook, the claim recited steps of calculating an updated value for an alarm limit (a numerical limit on a process variable such as temperature, pressure or flow rate) according to a mathematical formula “in a process comprising the catalytic chemical conversion of hydrocarbons.” 437 U.S. at 586, 198 USPQ at 196. Processes for the catalytic chemical conversion of hydrocarbons were used in the petrochemical and oil-refining fields. Id. Although the applicant argued that limiting the use of the formula to the petrochemical and oil-refining fields should make the claim eligible because this limitation ensured that the claim did not preempt all uses of the formula, the Supreme Court disagreed. 437 U.S. at 588-90, 198 USPQ at 197-98. Instead, the additional element in Flook regarding the catalytic chemical conversion of hydrocarbons was not sufficient to make the claim eligible, because it was merely an incidental or token addition to the claim that did not alter or affect how the process steps of calculating the alarm limit value were performed. Further, the Supreme Court found that this limitation did not amount to an inventive concept. 437 U.S. at 588-90, 198 USPQ at 197-98. The Court reasoned that to hold otherwise would “exalt[] form over substance”, because a competent claim drafter could attach a similar type of limitation to almost any mathematical formula. 437 U.S. at 590, 198 USPQ at 197.
In contrast, the additional elements in Diamond v. Diehr as a whole provided eligibility and did not merely recite calculating a cure time using the Arrhenius equation “in a rubber molding process”. Instead, the claim in Diehr recited specific limitations such as monitoring the elapsed time since the mold was closed, constantly measuring the temperature in the mold cavity, repetitively calculating a cure time by inputting the measured temperature into the Arrhenius equation, and opening the press automatically when the calculated cure time and the elapsed time are equivalent. 450 U.S. at 179, 209 USPQ at 5, n. 5. These specific limitations act in concert to transform raw, uncured rubber into cured molded rubber. 450 U.S. at 177-78, 209 USPQ at 4.
Applicant's arguments are unpersuasive.
The rejections stand.
Argument 3
Similarly, present claims 2, 10 and 18 recite "in response to the determining, labeling the second feature and preventing the second feature from being used to train a second ML model." The action of preventing in the present claims establishes eligibility.
Based at least on the foregoing, the claims should be subject matter eligible.
Preventing data from being used is merely removing the data from the available record. This is mere record keeping. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iii. Electronic recordkeeping, Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 225, 110 USPQ2d 1984 (2014) (creating and maintaining “shadow accounts”); Ultramercial, 772 F.3d at 716, 112 USPQ2d at 1755 (updating an activity log);
Applicant's argument is unpersuasive.
The rejections stand.
Argument 4
The Claims are Allowable because the Prior Art Fails to Disclose, For ML Models that are Using a Specific Feature, Invoking the Models and Using Metrics to Determine Data Drift for Each Model
Claims 1, 9 and 17 are rejected under 35 U.S.C. §102(a)(1) as being taught by Breck, et al., Data Validation for Machine Learning, Proceedings of the 2nd SysML Conference, 2019, pp. 1-14 ("Breck"). Reconsideration of these rejections is respectfully requested because the prior art fails to disclose, for ML models that are using a specific feature, invoking the models and using metrics to determine data drift for each model.
Embodiments detect data drift associated with machine learning ("ML") models. See specification at [0012]. Embodiments identifying a first feature stored 50 by a feature store, where the feature store comprises an offline store and an online store. Id. at [0031]. Embodiments determine one or more first trained ML models, of a plurality of ML models, that are using the first feature. Id. at [0038]. For each of the first trained ML models, before each of the first trained ML models have been deployed, embodiments invoke the first trained ML model using synthetic data or validation data. Id. at [0050].
Embodiments generating metrics to determine an accuracy of the first trained ML model, the metrics corresponding to each first trained ML model and including one or more of an area under a receiver operating characteristic curve, precision or recall. Id. at [0052]. When the accuracy is below a threshold, embodiments generate an alert notifying of a first data drift for the first trained ML model. Id. at [0053].
Breck discloses "a data validation system that is designed to detect anomalies specifically in data fed into machine learning pipelines." Breck at Abstract. Therefore, although Breck is directed generally at detecting data drift, Breck focuses on the incoming data rather than invoking each model on a per feature basis, and determining metrics for the model itself rather than merely the data.
The argued “invoking each model on a per feature basis” is only partially claimed. Specifically, the limitation “per feature” is not in the claim. The argued “invoking each model” is taught by Breck, et al., page 8, left column, second full paragraph, where it recites:
In fact, it has worked so well that we have packaged this type of testing as a unit test over training algorithms, and included the test in the standard templates of our ML platform. Our users routinely execute these unit tests to validate changes to the training logic of their pipelines. To our knowledge, this application of unit testing in the context of ML and for the purpose of data validation is a novel aspect of our work.
The prior art “testing as a unit test over training algorithms” executes the machine learning models being trained. This is an “invocation” of the models.
The argued “determining metrics for the model itself rather than merely the data” is a paraphrasing of what is claimed and is taught by Breck, et al., page 6, right column, second full paragraph, where it recites:
In addition to validating individual batches of training data, the Data Validator also monitors for skew between training and serving data, continuously.
Specifically, the argued “metrics for the model itself” are the determinations of “skew” for the model. Another place in Breck where this is explained is Breck, et al., page 6, left column, bottom of next to last full paragraph, where it recites:
Now, if the training data is generated by querying the same database then it is likely that the click count for each impression will appear higher compared to the serving data, since it includes all the clicks that happened between when the data was served and when the training data was generated. This skew would bias the resulting model against a different distribution of click rates compared to what is observed at serving time, which is likely to affect model quality.
Note that Applicant never explained how or when model quality was to be detected. In the amended claims, Applicant specifies a “receiver operating characteristic (ROC)” as the method. Examiner applies 35 U.S.C. § 103 art for this amendment.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 5
For at least these reasons, amended independent claim 1, and amended independent claims 9 and 17, which recite similar limitations, should now be allowable over the cited prior art.
Similar arguments for similar claims are similarly unpersuasive.
The rejections stand.
Argument 6
The remaining claims depend from one of the above independent claims and should also be allowable for at least the above reasons.
Since there is no eligible matter in the independent claims, there is no such matter that may be incorporated by reference to the dependent claims.
The rejections stand.
Conclusion
Any inquiries concerning this communication or earlier communications from the examiner should be directed to Wilbert L. Starks, Jr., who may be reached Monday through Friday, between 8:00 a.m. and 5:00 p.m. EST. or via telephone at (571) 272-3691 or email: Wilbert.Starks@uspto.gov.
If you need to send an Official facsimile transmission, please send it to (571) 273-8300.
If attempts to reach the examiner are unsuccessful the Examiner’s Supervisor (SPE), Kakali Chaki, may be reached at (571) 272-3719.
Hand-delivered responses should be delivered to the Receptionist @ (Customer Service Window Randolph Building 401 Dulany Street, Alexandria, VA 22313), located on the first floor of the south side of the Randolph Building.
Finally, information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Moreover, status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have any questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) toll-free @ 1-866-217-9197.
/WILBERT L STARKS/
Primary Examiner, Art Unit 2122
WLS
19 MAR 2026
Prosecution Insights