Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 101 being directed to an abstract idea without significantly more
Regarding Claim 1
Step 1: “A computer-implemented method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: generating, with at least one processor, a first plot based on the first score for each first data record of the plurality of first data records and the second score for each second data record of the plurality of second data records is a mental process that can be done with the aid of pen and paper, a person can plot a graph based off of data
generating, with at least one processor, a plurality of second plots associated with at least a subset of the plurality of features, each respective second plot of the plurality of second plots generated based on a respective first field of the plurality of first fields associated with a respective feature of the plurality of features and a respective second field of the plurality of second fields associated with the respective feature is a mental process that can be done with the aid of pen and paper, a person can plot a graph based off of features
Step 2A Prong 2: The additional limitations receiving, with at least one processor, a plurality of first data records, each first data record of the plurality of first data records comprising a plurality of first fields associated with a plurality of features and a first score field associated with a first score generated by a machine learning model based on the plurality of first fields is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g) – examiners note: receiving data here amounts to nothing more than mere data gathering
receiving, with at least one processor, a plurality of second data records, each second data record of the plurality of second data records comprising a plurality of second fields associated with the plurality of features and a second score field associated with a second score generated by the machine learning model based on the plurality of second fields; generating, with at least one processor, a first plot based on the first score for each first data record of the plurality of first data records and the second score for each second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g) – examiners note: receiving data here amounts to nothing more than mere data gathering
displaying, with at least one processor, the first plot is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a plot
and displaying, with at least one processor, the plurality of second plots. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a plot
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations receiving, with at least one processor, a plurality of first data records, each first data record of the plurality of first data records comprising a plurality of first fields associated with a plurality of features and a first score field associated with a first score generated by a machine learning model based on the plurality of first fields is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore, the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II), 2106.05(g) – examiners note: receiving data here amounts to nothing more than mere data gathering
receiving, with at least one processor, a plurality of second data records, each second data record of the plurality of second data records comprising a plurality of second fields associated with the plurality of features and a second score field associated with a second score generated by the machine learning model based on the plurality of second fields; generating, with at least one processor, a first plot based on the first score for each first data record of the plurality of first data records and the second score for each second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore, the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II), 2106.05(g) – examiners note: receiving data here amounts to nothing more than mere data gathering
displaying, with at least one processor, the first plot is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a plot
and displaying, with at least one processor, the plurality of second plots. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a plot
Regarding Claim 2
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: ranking, with at least one processor, the plurality of features based on at least one mismatch metric determined based on the respective first field associated with each respective feature and the respective second field associated with each respective feature is a mental process that can be done with the aid of pen and paper, a person can rank features based off of a metric
wherein generating the plurality of second plots comprises generating the plurality of second plots associated with a selected number of the plurality of features based on the ranking of the plurality of features is a mental process that can be done with the aid of pen and paper, a person can generate a plot based off of feature rankings
Step 2A Prong 2: The additional limitations wherein displaying the plurality of second plots comprises displaying the plurality of second plots in order based on the ranking of the plurality of features. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a plot based off feature rankings
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations wherein displaying the plurality of second plots comprises displaying the plurality of second plots in order based on the ranking of the plurality of features. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a plot based off of feature rankings
Regarding Claim 3
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of claim 2
Step 2A Prong 2: the additional limitations wherein the at least one mismatch metric comprises at least one of a correlation metric, a Pearson correlation metric, a gain metric determined based on at least one decision tree model, a weight metric determined based on the at least one decision tree model, a cover metric determined based on the at least one decision tree model, a SHapley Additive exPlanations (SHAP) metric determined based on the at least one decision tree model, or any combination thereof. is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations wherein the at least one mismatch metric comprises at least one of a correlation metric, a Pearson correlation metric, a gain metric determined based on at least one decision tree model, a weight metric determined based on the at least one decision tree model, a cover metric determined based on the at least one decision tree model, a SHapley Additive exPlanations (SHAP) metric determined based on the at least one decision tree model, or any combination thereof. is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Regarding Claim 4
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: determining a first difference based on the first score for each first data record and the second score for each second data record is a mental process that can be done with the aid of pen and paper, a person can look at two sets of scores and point out the differences
determining a plurality of second differences comprising a respective second difference based on the respective first field associated with each respective feature and the respective second field associated with each respective feature; is a mental process that can be done with the aid of pen and paper, a person can look at two sets of scores and point out the differences
and determining the Pearson correlation metric for each respective feature based on the first difference and the respective second difference. Is a mathematical concept – See MPEP § 2106.04(a)(2)(I)(B).
Step 2A Prong 2 and Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Regarding Claim 5
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: determining a first difference based on the first score for each first data record and the second score for each second data record; is a mental process that can be done with the aid of pen and paper, a person can look at two sets of scores and point out the differences
Determining a plurality of second differences comprising a respective second difference based on the respective first field associated with each respective feature and the respective second field associated with each respective feature is a mental process that can be done with the aid of pen and paper, a person can look at two sets of scores and point out the differences
and regressing the first difference onto the plurality of second differences based on the at least one decision tree model to provide at least one of the gain metric, the weight metric, the cover metric, the SHAP metric, or any combination thereof. Is a mathematical concept – See MPEP § 2106.04(a)(2)(I)(B).
Step 2A Prong 2 and Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Regarding Claim 6
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of claim 1
Step 2A Prong 2: The additional limitation receiving, with at least one processor, a selection of a first point in the first plot associated with a respective first data record of the plurality of first data records and a respective second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g) – examiners note: receiving a selection of points here amounts to nothing more than mere data gathering
and automatically selecting, with at least one processor, a respective second point in each second plot of the plurality of second plots associated with the respective first data record and the respective second data record. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of automatically selecting a point associated with the two data records.
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations receiving, with at least one processor, a selection of a first point in the first plot associated with a respective first data record of the plurality of first data records and a respective second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore, the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II), 2106.05(g) – examiners note: receiving a selection of points here amounts to nothing more than mere data gathering
and automatically selecting, with at least one processor, a respective second point in each second plot of the plurality of second plots associated with the respective first data record and the respective second data record. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of automatically selecting a point associated with the two data records.
Regarding Claim 7
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of Claim 1
Step 2A Prong 2: The additional limitations receiving, with at least one processor, a selection of a second point in one of the plurality of second plots associated with a respective first data record of the plurality of first data records and a respective second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g) – examiners note: receiving a selection of points here amounts to nothing more than mere data gathering
automatically selecting, with at least one processor, a first point in the first plot associated with the respective first data record and the respective second data record is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of selecting a point associated with the two data records
and automatically selecting, with at least one processor, a respective second point associated with the respective first data record and the respective second data record in each second plot of the plurality of second plots other than the one of the plurality of second plots. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of selecting a point associated with the two data records
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations receiving, with at least one processor, a selection of a second point in one of the plurality of second plots associated with a respective first data record of the plurality of first data records and a respective second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore, the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II), 2106.05(g) – examiners note: receiving a selection of points here amounts to nothing more than mere data gathering
automatically selecting, with at least one processor, a first point in the first plot associated with the respective first data record and the respective second data record is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of selecting a point associated with the two data records
and automatically selecting, with at least one processor, a respective second point associated with the respective first data record and the respective second data record in each second plot of the plurality of second plots other than the one of the plurality of second plots. is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d),
Regarding Claim 8
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of claim 1
Step 2A Prong 2: The additional limitations wherein the first plot comprises a first scatterplot and the plurality of second plots comprises a plurality of second scatterplots, the method further comprising is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
converting, with at least one processor, the first scatterplot to a first density plot is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of converting a scatterplot to a density plot
converting, with at least one processor, the plurality of second scatterplots to a plurality of second density plots is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of converting a scatterplot to a density plot
displaying, with at least one processor, the first density plot is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a density plot
and displaying, with at least one processor, the plurality of second density plots is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a density plot
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations wherein the first plot comprises a first scatterplot and the plurality of second plots comprises a plurality of second scatterplots, the method further comprising is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
converting, with at least one processor, the first scatterplot to a first density plot is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of converting a scatterplot to a density plot
converting, with at least one processor, the plurality of second scatterplots to a plurality of second density plots is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of converting a scatterplot to a density plot
displaying, with at least one processor, the first density plot is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a density plot
and displaying, with at least one processor, the plurality of second density plots is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of displaying a density plot
Regarding Claim 9
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of claim 1
Step 2A Prong 2: The additional limitations indexing, with at least one processor, a plurality of first points of the first plot based on a first quadtree data structure is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of indexing points based on a data structure
and indexing, with at least one processor, a respective plurality of second points of each respective second plot of the plurality of second plots based on a respective second quadtree data structure is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of indexing points based on a data structure
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations indexing, with at least one processor, a plurality of first points of the first plot based on a first quadtree data structure is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of indexing points based on a data structure
and indexing, with at least one processor, a respective plurality of second points of each respective second plot of the plurality of second plots based on a respective second quadtree data structure is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2) – examiners note: high level recitation of indexing points based on a data structure
Regarding Claim 10
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of Claim 1
Step 2A Prong 2: wherein the plurality of first data records are from a first environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
wherein the plurality of second data records are from a second environment different than the first environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
and wherein each respective first data record of the plurality of first data records has a corresponding second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g) – examiners note: ensuring that the first data record has a corresponding second data record amounts to nothing more than mere data gathering
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations wherein the plurality of first data records are from a first environment are an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
wherein the plurality of second data records are from a second environment different than the first environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
and wherein each respective first data record of the plurality of first data records has a corresponding second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore, the additional element is directed to electronic recordkeeping, which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II), 2106.05(g) – examiners note: ensuring that the first data record has a corresponding second data record amounts to nothing more than mere data gathering
Regarding Claim 11
Step 1: “The method” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of claim 1
Step 2A Prong 2: the additional limitation wherein the first environment comprises an offline environment, and wherein the second environment comprises an online environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitation wherein the first environment comprises an offline environment, and wherein the second environment comprises an online environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Regarding Claim 12
Step 1: “A system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 1
Step 2A Prong 2: See the analysis of Claim 1
Step 2B: See the analysis of Claim 1
Regarding Claim 13
Step 1: “The system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 2
Step 2A Prong 2: See the analysis of Claim 2
Step 2B: See the analysis of Claim 2
Regarding Claim 14
Step 1: “The system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 3
Step 2A Prong 2: See the analysis of Claim 3
Step 2B: See the analysis of Claim 3
Regarding Claim 15
Step 1: “The system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 4
Step 2A Prong 2: See the analysis of Claim 4
Step 2B: See the analysis of Claim 4
Regarding Claim 16
Step 1: “The system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 5
Step 2A Prong 2: See the analysis of Claim 5
Step 2B: See the analysis of Claim 5
Regarding Claim 17
Step 1: “The system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 6
Step 2A Prong 2: See the analysis of Claim 6
Step 2B: See the analysis of Claim 6
Regarding Claim 18
Step 1: “The system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 7
Step 2A Prong 2: See the analysis of Claim 7
Step 2B: See the analysis of Claim 7
Regarding Claim 19
Step 1: “The system” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: Recites the abstract ideas of Claim 1
Step 2A Prong 2: The additional limitations wherein the plurality of first data records are from a first environment are an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
wherein the plurality of second data records are from a second environment different than the first environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
wherein each respective first data record of the plurality of first data records has a corresponding second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore, the additional element is directed to electronic recordkeeping, which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II), 2106.05(g) – examiners note: ensuring that the first data record has a corresponding second data record amounts to nothing more than mere data gathering
and wherein the first environment comprises an offline environment and the second environment comprises an online environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Step 2B: The additional elements, taken either alone or in combination with other limitations of the claim, do not amount to significantly more than the abstract idea itself. The additional limitations wherein the plurality of first data records are from a first environment are an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
wherein the plurality of second data records are from a second environment different than the first environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
wherein each respective first data record of the plurality of first data records has a corresponding second data record of the plurality of second data records is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore, the additional element is directed to electronic recordkeeping, which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II), 2106.05(g) – examiners note: ensuring that the first data record has a corresponding second data record amounts to nothing more than mere data gathering
and wherein the first environment comprises an offline environment and the second environment comprises an online environment is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Regarding Claim 20
Step 1: “A computer program product” falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater).
Step 2A Prong 1: See the analysis of Claim 1
Step 2A Prong 2: See the analysis of Claim 1
Step 2B: See the analysis of Claim 1
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-7, 10-15, 17-20 are rejected under U.S.C. 103 as being unpatentable over Ferlito et al (NPL: Comparative analysis of data-driven methods online and offline trained to the forecasting of grid-connected photovoltaic plant production) (“Ferlito”) in view of Cueto-Lopez (NPL: A comparative study on feature selection for a risk prediction model for colorectal cancer) (“Cueto-Lopez”)
Regarding Claim 1, Ferlito teaches A computer-implemented method for coordinated analysis of output scores and input features of machine learning models in different environments, comprising: receiving, with at least one processor, a plurality of first data records, each first data record of the plurality of first data records comprising a plurality of first fields associated with a plurality of features and a first score field associated with a first score generated by a machine learning model based on the plurality of first fields ([Fig 3] teaches that methodology for online machine learning model testing is being done on a dataset. With the dataset being described as containing five different variables [Page 3: 37-43] which teaches a part of the limitation of data record comprising a plurality of first fields associated with a plurality of features. [Fig 4] showcases the accuracy results for the dataset across each year, the accuracy here being the score generated by the machine learning model and that in combination with the variables and the years teaching the features and fields would teach the data records limitation.)
receiving, with at least one processor, a plurality of second data records, each second data record of the plurality of second data records comprising a plurality of second fields associated with the plurality of features and a second score field associated with a second score generated by the machine learning model based on the plurality of second fields ([Fig 3] teaches that methodology for offline machine learning model testing is being done on a dataset. With the dataset being described as containing five different variables [Page 3: 37-43] which teaches the limitation of data record comprising a plurality of first fields associated with a plurality of features. [Fig 7] showcases the accuracy results for the dataset across each year, the accuracy here being the score generated by the machine learning model which teaches this limitation)
generating, with at least one processor, a first plot based on the first score for each first data record of the plurality of first data records and the second score for each second data record of the plurality of second data records (Fig 4] teaches a bar graph being used to represent the accuracy of the different models trained online across the different years of the dataset which teaches this limitation)
Ferlito does not teach displaying, with at least one processor, the first plot
generating, with at least one processor, a plurality of second plots associated with at least a subset of the plurality of features, each respective second plot of the plurality of second plots generated based on a respective first field of the plurality of first fields associated with a respective feature of the plurality of features and a respective second field of the plurality of second fields associated with the respective feature
However, Cueto-Lopez does teach displaying, with at least one processor, the first plot ([Fig 5] teaches plotting the different metrics on a graph)
generating, with at least one processor, a plurality of second plots associated with at least a subset of the plurality of features, each respective second plot of the plurality of second plots generated based on a respective first field of the plurality of first fields associated with a respective feature of the plurality of features and a respective second field of the plurality of second fields associated with the respective feature ([Fig 5] teaches plotting multiple different metrics which can function as a second plot with the algorithms used functioning as fields. The different plot points in the graph are dependent on the feature amount used which teaches this limitation)
Cueto-Lopez and Ferlito are analogous art because they both focus analyzing the performance of models using machine learning
It would have been obvious to a person skilled in the art before the effective filling date of the
claimed invention to combine Ferlito with the ranking capabilities of Cueto-Lopez. Doing so would allow for more effective evaluations when it comes to assessing a model’s performance ([Abstract: Cueto-Lopez])
Regarding Claim 2, Ferlito and Cueto-Lopez teaches all the limitations of claim 1
Cueto-Lopez also teaches ranking, with at least one processor, the plurality of features based on at least one mismatch metric determined based on the respective first field associated with each respective feature and the respective second field associated with each respective feature, wherein generating the plurality of second plots comprises generating the plurality of second plots associated with a selected number of the plurality of features based on the ranking of the plurality of features, and wherein displaying the plurality of second plots comprises displaying the plurality of second plots in order based on the ranking of the plurality of features. ([Page 5-Column 2: Lines 5-8] Teaches selecting the best feature selection strategy depending on the metric and the different features and [Fig 5] shows the plotting of the different metrics on a graph which would teach the plotting with a selected number of features.)
Regarding Claim 3, Ferlito and Cueto-Lopez teaches all the limitations of claim 2
Cueto-Lopez also teaches wherein the at least one mismatch metric comprises at least one of a correlation metric, a Pearson correlation metric, a gain metric determined based on at least one decision tree model, a weight metric determined based on the at least one decision tree model, a cover metric determined based on the at least one decision tree model, a SHapley Additive exPlanations (SHAP) metric determined based on the at least one decision tree model, or any combination thereof. ([Page 3-Section 2.1] teaches using the Pearson correlation coefficient which teaches this limitation)
Regarding Claim 4, Ferlito and Cueto-Lopez teaches all the limitations of claim 3
Cueto-Lopez also teaches wherein the at least one metric comprises the Pearson correlation metric, and wherein ranking the plurality of features comprises: determining a first difference based on the first score for each first data record and the second score for each second data record; determining a plurality of second differences comprising a respective second difference based on the respective first field associated with each respective feature and the respective second field associated with each respective feature; and determining the Pearson correlation metric for each respective feature based on the first difference and the respective second difference ([Table 2] teaches determining the Pearson correlation metric for each respective number of features. The Pearson correlation coefficient looks at how well each feature is correlated with the target [page 2 section 2.1] with the Pearson correlation here functioning as a score generated by a machine learning model and with the class functioning as the fields this teaches the data records limitation. With the Pearson correlation being calculated based on how well a feature correlates with a target across different machine learning algorithms it can be reasonably interpreted that a difference in scores is being calculated to come to the Pearson correlation metric. Which teaches this limitation)
Regarding Claim 6, Ferlito and Cueto-Lopez teaches all the limitations of claim 1
Cueto-Lopez also teaches receiving, with at least one processor, a selection of a first point in the first plot associated with a respective first data record of the plurality of first data records and a respective second data record of the plurality of second data records; and automatically selecting, with at least one processor, a respective second point in each second plot of the plurality of second plots associated with the respective first data record and the respective second data record. ([Fig 5] Teaches a graph that is plotting the AUC for the different metrics used based on the feature subset. The plotting of the points is being associated with automatically selecting a point and with different metrics being used that teaches the first and second point of the limitation)
Regarding Claim 7, Ferlito and Cueto Lopez teaches all the limitations of claim 1
Cueto-Lopez also teaches receiving, with at least one processor, a selection of a second point in one of the plurality of second plots associated with a respective first data record of the plurality of first data records and a respective second data record of the plurality of second data records; (The Pearson correlation coefficient looks at how well each feature is correlated with the target [page 2 section 2.1] with the Pearson correlation here functioning as a score generated by a machine learning model and with the class functioning as the fields this teaches the data records limitation. The plotting of the points in [Fig 5] is being interpreted as a selection of a point based off the two data records)
automatically selecting, with at least one processor, a first point in the first plot associated with the respective first data record and the respective second data record; ([Fig 5] teaches plotting the different techniques based on the different feature subsets. The plotting of the different techniques is being interpreted as automatically selecting a point associated with the two different data records)
and automatically selecting, with at least one processor, a respective second point associated with the respective first data record and the respective second data record in each second plot of the plurality of second plots other than the one of the plurality of second plots. ([Fig 5] teaches plotting the different techniques based off the feature set. The different techniques used with [Page 2-Column 2: Lines 1-3] using different data samples with different ranking techniques applied teaches the two different data records. The graph contains multiple plot points for each technique which teaches plots other than the plurality of second plots.)
Regarding Claim 10, Ferlito and Cueto Lopez teaches all the limitations of claim 1
Ferlito also teaches wherein the plurality of first data records are from a first environment, wherein the plurality of second data records are from a second environment different than the first environment, and wherein each respective first data record of the plurality of first data records has a corresponding second data record of the plurality of second data records. ([Abstract Lines 17-18] Teaches using two different training methodologies to identify the most performing training mode which teaches this limitation)
Regarding Claim 11, Ferlito and Cueto Lopez teaches all the limitations of claim 10
Ferlito also teaches wherein the first environment comprises an offline environment, and wherein the second environment comprises an online environment ([Abstract Lines 17-18] Teaches using two different training methodologies to identify the most performing training mode, the two different methodologies being online and offline which teaches this limitation)
Regarding Claim 12, See the analysis of claim 1
Regarding Claim 13, See the analysis of claim 2
Regarding Claim 14, See the analysis of claim 3
Regarding Claim 15, See the analysis of claim 4
Regarding Claim 17, See the analysis of claim 6
Regarding Claim 18, See the analysis of claim 7
Regarding Claim 19, Ferlito and Cueto Lopez teaches all the limitations of claim 1
Ferlito also teaches wherein the plurality of first data records are from a first environment, wherein the plurality of second data records are from a second environment different than the first environment, wherein each respective first data record of the plurality of first data records has a corresponding second data record of the plurality of second data records, and wherein the first environment comprises an offline environment and the second environment comprises an online environment ([Abstract Lines 17-18] Teaches using two different training methodologies to identify the most performing training mode, the two different methodologies being online and offline which teaches this limitation)
Regarding Claim 20, See the analysis of claim 1 because they both focus analyzing the performance of models using machine learning
Claim 5 and 16 are rejected are rejected under U.S.C. 103 as being unpatentable over Ferlito et al (NPL: Comparative analysis of data-driven methods online and offline trained to the forecasting of grid-connected photovoltaic plant production) (“Ferlito”) in view of Cueto-Lopez (NPL: A comparative study on feature selection for a risk prediction model for colorectal cancer) (“Cueto-Lopez”) and Wang et al (US20240177071A1) (“Wang”)
Regarding Claim 5, Ferlito and Cueto-Lopez teaches all the limitations of claim 3
Ferlito does not teach wherein ranking the plurality of features comprises: determining a first difference based on the first score for each first data record and the second score for each second data record; determining a plurality of second differences comprising a respective second difference based on the respective first field associated with each respective feature and the respective second field associated with each respective feature; and regressing the first difference onto the plurality of second differences based on the at least one decision tree model to provide at least one of the gain metric, the weight metric, the cover metric, the SHAP metric, or any combination thereof.
However, Wang does teach wherein ranking the plurality of features comprises: determining a first difference based on the first score for each first data record and the second score for each second data record; ([0139] teaches that performance difference revealed by numerical metrics that may not be sufficient to choose between models. With the different models finding different features with the emails functioning as fields this would teach the data records limitation. The presence of metrics here is being interpreted as a score for the two different data records)
determining a plurality of second differences comprising a respective second difference based on the respective first field associated with each respective feature and the respective second field associated with each respective feature; ([0036] teaches determining the difference in features between two different subsets of features with the subsets being the two different fields which teaches this limitation)
and regressing the first difference onto the plurality of second differences based on the at least one decision tree model to provide at least one of the gain metric, the weight metric, the cover metric, the SHAP metric, or any combination thereof. ([0024] teaches calculated SHAP values for each feature value with [0167] talking about the presence of gradient boosting trees which teaches the tree model teaching this limitation)
Ferlito, Cueto-Lopez, and Wang are analogous art because they all deal with analyzing the performance of models using machine learning
It would have been obvious to a person skilled in the art before the effective filling date of the
claimed invention to combine to combine Ferlito with the ranking capabilities of Cueto-Lopez with the SHAP regressing evaluation of Wang. Doing so would allow for a more thorough comparison on what model is outperforming the other based off metrics ([Wang-0010])
Regarding Claim 16, See the analysis of Claim 5
Claim 8 is rejected are rejected are rejected under U.S.C. 103 as being unpatentable over Ferlito et al (NPL: Comparative analysis of data-driven methods online and offline trained to the forecasting of grid-connected photovoltaic plant production) (“Ferlito”) in view of Cueto-Lopez (NPL: A comparative study on feature selection for a risk prediction model for colorectal cancer) (“Cueto-Lopez”) and Gorgens
Regarding Claim 8, Ferlito and Cueto-Lopez teaches all the limitations of claim 1
Ferlito does not teach wherein the first plot comprises a first scatterplot and the plurality of second plots comprises a plurality of second scatterplots, the method further comprising: converting, with at least one processor, the first scatterplot to a first density plot; converting, with at least one processor, the plurality of second scatterplots to a plurality of second density plots; displaying, with at least one processor, the first density plot; and displaying, with at least one processor, the plurality of second density plots.
However, Gorgens does teach [Fig 1] teaches a scatterplot of volumes predicted by 4 different model techniques. The multiple different scatter plots for the different methodologies are being interpreted as the first and second plot and the figures are displaying the results of the prediction which teaches this limitation ([Fig 1] teaches a scatterplot of volumes predicted by 4 different model techniques. The multiple different scatter plots for the different methodologies are being interpreted as the first and second plot and the figures are displaying the results of the prediction which teaches this limitation)
Ferlito, Cueto-Lopez, and Gorgens are analogous art because they all deal with analyzing the performance of models using machine learning
It would have been obvious to a person skilled in the art before the effective filling date of the
claimed invention to combine to combine Ferlito with the ranking capabilities of Cueto-Lopez with the scatterplot of Gorgens. Doing so would lead to more effective techniques when exploring metric sets ([Abstract-Gorgen])
Claim 9 is rejected are rejected are rejected under U.S.C. 103 as being unpatentable over Ferlito et al (NPL: Comparative analysis of data-driven methods online and offline trained to the forecasting of grid-connected photovoltaic plant production) (“Ferlito”) in view of Cueto-Lopez (NPL: A comparative study on feature selection for a risk prediction model for colorectal cancer) (“Cueto-Lopez”) and Jewsbury (NPL: A QuadTree Image Representation for Computational Pathology) (“Jewsbury”)
Regarding Claim 9, Ferlito and Cueto-Lopez teaches all the limitations of claim 1
Ferlito does not teach further comprising: indexing, with at least one processor, a plurality of first points of the first plot based on a first quadtree data structure; and indexing, with at least one processor, a respective plurality of second points of each respective second plot of the plurality of second plots based on a respective second quadtree data structure.
However, Jewsbury does teach further comprising: indexing, with at least one processor, a plurality of first points of the first plot based on a first quadtree data structure; and indexing, with at least one processor, a respective plurality of second points of each respective second plot of the plurality of second plots based on a respective second quadtree data structure. ([Figure 4] teaches AUROC curve plots being trained on quadtree nodes. With their different graphs present it can be reasonably interpreted that different quadtree nodes are being used which would teach this limitation)
Ferlito, Cueto-Lopez and Jewsbury are analogous art because of analyzing the performance of models using machine learning
It would have been obvious to a person skilled in the art before the effective filling date of the
claimed invention to combine to combine Ferlito with the ranking capabilities of Cueto-Lopez with the quadtree method of Jewsbury. Doing so would lead to more accurate image data ([Abstract-Jewsbury])
Conclusion
The prior arts are made of record and relied upon is considered to applicant’s disclosure
Stein et al US 20190325351 A1 (2019-10-24) ([Abstract] “The disclosed embodiments provide a system for processing data. During operation, the system selects a set of entity keys associated with reference feature values used with one or more machine learning models, wherein the reference feature values are generated in a first environment. Next, the system matches the set of entity keys to feature values from a second environment. The system then compares the feature values and the reference feature values to assess a consistency of a feature across the first and second environments. Finally, the system outputs a result of the assessed consistency for use in managing the feature in the first and second environments.”)
Jain et al US 20230196185 A1 (2023-06-22) ([Abstract] “This disclosure describes a feature family system that, as part of an inter-network facilitation system, can intelligently generate and maintain a feature family repository for quickly and efficiently retrieving and providing machine learning features upon request. For example, the disclosed systems can generate a feature family repository as a centralized network location of feature references indicating network locations where different machine learning features are stored. In some cases, the disclosed systems identify a stored feature family that matches the request and retrieves the stored features from their respective network locations. The disclosed systems can generate feature families for online features as well as offline features and can automatically update feature values associated with various machine learning features on a period basis or in response to trigger events.”)
Nyati et al US 20220083445 A1 (2022-03-17) ([Abstract] “Systems and/or techniques for facilitating online-monitoring of machine learning models are provided. In various embodiments, a system can receive monitoring settings associated with a machine learning model to be monitored. In various cases, the monitoring settings can identify a first set of data features that are generated as output by the machine learning model. In various cases, the monitoring settings can identify a second set of data features that are received as input by the machine learning model. In various aspects, the system can compute a first set of statistical metrics based on the first set of data features. In various cases, the first set of statistical metrics can characterize a performance quality of the machine learning model. In various instances, the system can compute a second set of statistical metrics based on the second set of data features. In various cases, the second set of statistical metrics can characterize trends or distributions of input data associated with the machine learning model. In various aspects, the system can store the first set of statistical metrics and the second set of statistical metrics in a data warehouse that is accessible to an operator. In various embodiments, the system can render the first set of statistical metrics and the second set of statistical metrics on an electronic interface, such that the first set of statistical metrics and the second set of statistical metrics are viewable to the operator.”
Akinsola et al NPL: Performance Evaluation of Supervised Machine Learning Algorithms Using Multi-Criteria Decision Making Techniques (08-2022) ([Abstract] “The choice of classification algorithm in Machine Learning (ML) is a major issue cutting across several disciplines due to the uncertainty in human judgment in the ranking of performance metrics. The process of algorithm selection can be modelled as Multi-Criteria Decision Making (MCDM) problem which involves more than one criterion. In this work, seven classification algorithms, and ten performance criteria were considered to test the proposed Fuzzy Analytical Hierarchical Process (FAHP) and Technique or Order of Preference by Similarity to Ideal Solution (TOPSIS) model. The model was developed using respective priority weights based on AHP and fuzzy logic principle. Pairwise comparison matrix was formulated based on decision makers’ judgments that were aggregated and normalized. The study applied FAHP in assigning weights to the criteria and ranking the performance criteria, while Simple Additive Weighting (SAW) and TOPSIS were implemented in MATLAB to rank the classifiers for comparison. Fuzzification was done using Triangular Fuzzy Numbers (TFNs) and defuzzification was done using Graded Mean Integration (GMI) approach. Consistency of the decision makers’ judgments were obtained using Saaty’s Eigen value and Eigen vector approach. Unlike the usual practice, in addition to Accuracy as the benchmark for selecting an algorithm, the Kappa Statistic measure was also considered. The result of algorithm performance evaluation shows that Logistic Regression (LRN) from Waikato Environment for Knowledge Analysis (WEKA) has the highest Kappa Statistic. Also, FAHP result for criteria weights determination shows that Kappa Statistic has the highest priority weight then Accuracy based on decision makers’ judgments. FAHP Consistency Ratio (CR) has a value of 0.017, which is less than 10%. Hence, criteria weights results are reliable. The TOPSIS ranking result of ML algorithms shows that LRN has the highest ranking. The study concluded that LRN being the algorithm with the highest ranking is considered as the best classifier. Therefore, MCDM techniques can be used in selecting the best Supervised Machine Learning Algorithm for classification and regression.”)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to URIAH V MOORE whose telephone number is (571)384-8341. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571)270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/URIAH VENDELL MOORE/ Examiner, Art Unit 2142 /Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights