Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 119(e) as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed application, Provisional Application No. 63484362, fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application.
The limitations of claims 2-3, 5-8, 10-11, 13-16, and 18-19 lack written support in the prior-filed application. Accordingly, claims 2-3, 5-8, 10-11, 13-16, and 18-19 are not entitled to the benefit of the prior application.
Claim Objections
Claims 1, 9, and 17 are objected to because of the following informalities: sparsity off training data should read sparsity of training data. Appropriate correction is required.
Claims 8 and 16 are objected to because of the following informalities: intermediate predicated target variables should read intermediate predicted target variables. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-4, 10-12 and 18-20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 2-4, 10-12 and 18-20 each recite the limitation wherein identifying the rare event comprises. There is insufficient antecedent basis for this limitation of the above claims because the independent claims 1, 9 and 17, on which they depend, do not disclose a rare event identification step.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the invention is directed at an abstract idea without significantly more.
Regarding claim 17, in step 1 of the 101 analysis set forth in MPEP 2106.03, the claim recites A computer-readable storage medium that includes instructions which, when executed by one or more processors, cause the one or more processors to perform operations. Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the broadest reasonable interpretation of the “computer-readable storage medium” of claim 17 encompasses signals per se. The specification states that “A computer storage medium is not a propagated signal.” Since the claim reads “A computer-readable storage medium”, the disclaimer “A computer storage medium is not a propogated signal.” does not exclude transitory signals per se from the scope of the claim. Explicitly, this is due to the wording difference between the specification and the claim. As understood in light of the specification, the broadest reasonable interpretation of claim 17 encompasses signals per se which is not within one of the four statutory categories of invention. See MPEP 2106.03(I). It is suggested that claim 17 be amended to recite a “computer storage medium” or to recite a “non-transitory” computer-readable storage medium to overcome this rejection.
Also regarding claim 17, if the above rejection is overcome, the following rejection under 35 U.S.C. 101 would still apply:
Subject Matter Eligibility Analysis Step 1:
Claim 17 recites A computer-readable storage medium that includes instructions which, when executed by one or more processors, cause the one or more processors to perform operations. The claim recites a machine. A machine is one of the four statutory categories of invention.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 17 recites
determining that a sparsity of training data exists for the rare event in the particular subset of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing the training data and making a mental determination, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
in response to determining that a sparsity off training data exists for the rare event in the particular subset of the training data, selecting other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing subsets of data and selecting them based on a criteria, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
identifying feature sets that are associated with the rare event from the selected other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing feature sets and determing their association with the rare event, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
generating a special training data set that includes the feature sets that are associated with the rare events from the selected other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like writing down and grouping the feature sets associated with the rare events, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
determining an ensembled predicted target variable based on the intermediate predicted target variables; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing the intermediate predicted target variables and then manually combining them or choosing one to keep, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
Therefore, claim 17 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 17 further recites additional elements of
A computer-readable storage medium that includes instructions which, when executed by one or more processors, cause the one or more processors to perform operations, (i.e., the generic computer components recited in this limitation merely add the words “apply it”, or an equivalent, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f))).
comprising: training a naïve machine learning model to generate a predicted target variable for a rare event using a particular subset of the training data; (This element does not integrate the abstract idea into a practical application because it amounts to mere "apply it on a computer" (see MPEP 2106.05(f))).
training a special machine learning model to generate the predicted target variable using the special training data set; (This element does not integrate the abstract idea into a practical application because it amounts to mere "apply it on a computer" (see MPEP 2106.05(f))).
obtaining, as intermediate predicted target variables, the predicted target variable from each of the special machine learning model and the naïve machine learning model; (i.e., the broadest reasonable interpretation of receiving a data instance is mere data gathering, which is an insignificant extra solution activity (MPEP 2106.05(d)(II))).
providing the ensembled predicted target variable for output. (i.e., the broadest reasonable interpretation of outputting a data instance is mere data outputting, which is an insignificant extra solution activity (MPEP 2106.05(d)(II))).
Therefore, claim 17 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 17 do not provide significantly more than the abstract idea itself, taken alone or in combination because
A computer-readable storage medium that includes instructions which, when executed by one or more processors, cause the one or more processors to perform operations, merely recites steps that apply generic computer components as a tool to perform judicial exceptions, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)).
comprising: training a naïve machine learning model to generate a predicted target variable for a rare event using a particular subset of the training data; merely recites steps that train a generic machine learning model, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)).
training a special machine learning model to generate the predicted target variable using the special training data set; merely recites steps that train a generic machine learning model, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)).
obtaining, as intermediate predicted target variables, the predicted target variable from each of the special machine learning model and the naïve machine learning model; is the well understood, routine, and conventional activity of "transmitting or receiving data over a network" (see MPEP 2106.05(d)(II); OIP Techs., Inc., V. Amazon.com, Inc., 788F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
providing the ensembled predicted target variable for output. is the well understood, routine, and conventional activity of "transmitting or receiving data over a network" (see MPEP 2106.05(d)(II); OIP Techs., Inc., V. Amazon.com, Inc., 788F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Therefore, claim 17 is subject-matter ineligible.
Regarding claim 18, it is dependent upon claim 17 and fails to resolve the deficiencies identified
above by integrating the subject matter into a statutory category. For example, claim 18 recites wherein identifying the rare event comprises receiving a user selection of a rare event type. Under the broadest reasonable interpretation, this limitation does not exclude transitory signals per se from the claim.
Also regarding claim 18, the above limitation recites steps of mere data gathering which has been recognized by the courts as being a well-understood, routine, and conventional function. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra solution activity (See MPEP 2106.05(d)(II)). Therefore, claim 18 does not solve the deficiencies of claim 17.
Regarding claim 19, it is dependent upon claim 17 and fails to resolve the deficiencies identified above by integrating the subject matter into a statutory category. For example, claim 19 recites wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence. Under the broadest reasonable interpretation, this limitation does not exclude transitory signals per se from the claim.
Also regarding claim 19, the above limitation recites making a mental determination based on an observed confidence level which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 19 does not solve the deficiencies of claim 17.
Regarding claim 20, it is dependent upon claim 17 and fails to resolve the deficiencies identified above by integrating the subject matter into a statutory category. For example, claim 20 recites wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold. Under the broadest reasonable interpretation, this limitation does not exclude transitory signals per se from the claim.
Also regarding claim 20, the above limitation recites counting a quantity of feature sets and making a mental determination based on the quantity which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 20 does not solve the deficiencies of claim 17.
Regarding claim 1, in step 1 of the 101 analysis set forth in MPEP 2106.03, the claim recites A computer-implemented method. The claim recites a method. A method is one of the four statutory categories of invention.
In Step 2A, Prong 1 of the 101 analysis set forth in MPEP 2106.04, the examiner has determined that the following limitations recite a process that, under broadest reasonable interpretation, covers a mental process or mathematical concept but for the recitation of generic computer components:
determining that a sparsity of training data exists for the rare event in the particular subset of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing the training data and making a mental determination, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
in response to determining that a sparsity off training data exists for the rare event in the particular subset of the training data, selecting other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing subsets of data and selecting them based on a criteria, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
identifying feature sets that are associated with the rare event from the selected other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing feature sets and determing their association with the rare event, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
generating a special training data set that includes the feature sets that are associated with the rare events from the selected other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like writing down and grouping the feature sets associated with the rare events, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
determining an ensembled predicted target variable based on the intermediate predicted target variables; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing the intermediate predicted target variables and then manually combining them or choosing one to keep, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
If the claim limitations, under their broadest reasonable interpretation, covers activities classified under Mental processes: concepts performed in the human mind (including observation, evaluation, judgement, or opinion) (see MPEP 2106.04(a)(2), subsection (III)) or Mathematical concepts: mathematical relationships, mathematical formulas or equations, or mathematical calculations (see MPEP 2106.04(a)(2), subsection (I)). Accordingly, the claim recites an abstract idea.
In Step 2A, Prong 2 of the 101 analysis, set forth in MPEP 2106.04, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
training a naïve machine learning model to generate a predicted target variable for a rare event using a particular subset of the training data; (This element does not integrate the abstract idea into a practical application because it amounts to mere "apply it on a computer" (see MPEP 2106.05(f))).
training a special machine learning model to generate the predicted target variable using the special training data set; (This element does not integrate the abstract idea into a practical application because it amounts to mere "apply it on a computer" (see MPEP 2106.05(f))).
obtaining, as intermediate predicted target variables, the predicted target variable from each of the special machine learning model and the naïve machine learning model; (i.e., the broadest reasonable interpretation of receiving a data instance is mere data gathering, which is an insignificant extra solution activity (MPEP 2106.05(d)(II))).
providing the ensembled predicted target variable for output. (i.e., the broadest reasonable interpretation of outputting a data instance is mere data outputting, which is an insignificant extra solution activity (MPEP 2106.05(d)(II))).
Since the claim does not contain any other additional elements, that amount to integration into a practical application, the claim is directed to an abstract idea.
In Step 2B of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception:
Regarding limitations (VIII) and (IX), under the broadest reasonable interpretation, recite steps of mere data gathering/outputting, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering/outputting as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018)).
Examiner uses Berkheimer: Option 2, a citation to one or more of the court decisions discussed in MPEP 2106.05(d)(II) as noting well-understood, routine, and conventional nature of the additional elements:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
Further, limitation (VI) and (VII), under the broadest reasonable interpretation, merely recite steps that train generic machine learning models, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Considering additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 2, it is dependent upon claim 1 and fails to resolve the deficiencies identified
above by integrating the judicial exception into a practical application, or introducing significantly more
than the judicial exception. For example, claim 2 recites wherein identifying the rare event comprises receiving a user selection of a rare event type. Under the broadest reasonable interpretation, this limitation recites steps of mere data gathering which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra solution activity (See MPEP 2106.05(d)(II)). Therefore, claim 2 does not solve the deficiencies of claim 1.
Regarding claim 3, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 3 recites wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence. Under the broadest reasonable interpretation, the limitation recites making a mental determination based on an observed confidence level which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 3 does not solve the deficiencies of claim 1.
Regarding claim 4, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 4 recites wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold. Under the broadest reasonable interpretation, the limitation recites counting a quantity of feature sets and making a mental determination based on the quantity which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 4 does not solve the deficiencies of claim 1.
Regarding claim 5, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 5 recites wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset. Under the broadest reasonable interpretation, the limitation recites observing subsets of data and choosing them based on a common characteristic which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 5 does not solve the deficiencies of claim 1.
Regarding claim 6, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 6 recites wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset. Under the broadest reasonable interpretation, the limitation recites making a mental determination based on observing the physical distance between locations which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 6 does not solve the deficiencies of claim 1.
Regarding claim 7, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 7 recites wherein the special machine learning model comprises a gradient boosted tree (GBT). Under the broadest reasonable interpretation, the limitation merely recites steps that apply a generic gradient boosted tree, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Therefore, claim 7 does not solve the deficiencies of claim 1.
Regarding claim 8, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 8 recites wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value. Under the broadest reasonable interpretation, the limitation merely recites making a mental determination based on observing the predicted target variables which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 8 does not solve the deficiencies of claim 1.
Regarding claim 9, in step 1 of the 101 analysis set forth in MPEP 2106.03, the claim recites A system comprising: one or more processors, and one or more computer-readable storage media that includes instructions which, when executed by the one or more processors, cause the one or more processors to perform operations. The claim recites a machine. A machine is one of the four statutory categories of invention.
In Step 2A, Prong 1 of the 101 analysis set forth in MPEP 2106.04, the examiner has determined that the following limitations recite a process that, under broadest reasonable interpretation, covers a mental process or mathematical concept but for the recitation of generic computer components:
determining that a sparsity of training data exists for the rare event in the particular subset of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing the training data and making a mental determination, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
in response to determining that a sparsity off training data exists for the rare event in the particular subset of the training data, selecting other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing subsets of data and selecting them based on a criteria, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
identifying feature sets that are associated with the rare event from the selected other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing feature sets and determing their association with the rare event, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
generating a special training data set that includes the feature sets that are associated with the rare events from the selected other subsets of the training data; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like writing down and grouping the feature sets associated with the rare events, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
determining an ensembled predicted target variable based on the intermediate predicted target variables; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with a pen and paper like observing the intermediate predicted target variables and then manually combining them or choosing one to keep, which is a mental process of observation/evaluation/judgement (MPEP 2106.04(a)(2))).
If the claim limitations, under their broadest reasonable interpretation, covers activities classified under Mental processes: concepts performed in the human mind (including observation, evaluation, judgement, or opinion) (see MPEP 2106.04(a)(2), subsection (III)) or Mathematical concepts: mathematical relationships, mathematical formulas or equations, or mathematical calculations (see MPEP 2106.04(a)(2), subsection (I)). Accordingly, the claim recites an abstract idea.
In Step 2A, Prong 2 of the 101 analysis, set forth in MPEP 2106.04, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
one or more processors, (i.e., the generic computer components recited in this limitation merely add the words “apply it”, or an equivalent, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f))).
and one or more computer-readable storage media that includes instructions which, when executed by the one or more processors, cause the one or more processors to perform operations (i.e., the generic computer components recited in this limitation merely add the words “apply it”, or an equivalent, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f))).
training a naïve machine learning model to generate a predicted target variable for a rare event using a particular subset of the training data; (This element does not integrate the abstract idea into a practical application because it amounts to mere "apply it on a computer" (see MPEP 2106.05(f))).
training a special machine learning model to generate the predicted target variable using the special training data set; (This element does not integrate the abstract idea into a practical application because it amounts to mere "apply it on a computer" (see MPEP 2106.05(f))).
obtaining, as intermediate predicted target variables, the predicted target variable from each of the special machine learning model and the naïve machine learning model; (i.e., the broadest reasonable interpretation of receiving a data instance is mere data gathering, which is an insignificant extra solution activity (MPEP 2106.05(d)(II))).
providing the ensembled predicted target variable for output. (i.e., the broadest reasonable interpretation of outputting a data instance is mere data outputting, which is an insignificant extra solution activity (MPEP 2106.05(d)(II))).
Since the claim does not contain any other additional elements, that amount to integration into a practical application, the claim is directed to an abstract idea.
In Step 2B of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception:
Regarding limitations (X) and (XI), under the broadest reasonable interpretation, recite steps of mere data gathering/outputting, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering/outputting as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018)).
Examiner uses Berkheimer: Option 2, a citation to one or more of the court decisions discussed in MPEP 2106.05(d)(II) as noting well-understood, routine, and conventional nature of the additional elements:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
Further, limitations (VIII) and (IX), under the broadest reasonable interpretation, merely recite steps that train generic machine learning models, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Further, limitations (VI) and (VII), under the broadest reasonable interpretation, merely recite steps that apply generic computer components as a tool to perform judicial exceptions, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Considering additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 10, it is dependent upon claim 9 and fails to resolve the deficiencies identified
above by integrating the judicial exception into a practical application, or introducing significantly more
than the judicial exception. For example, claim 10 recites wherein identifying the rare event comprises receiving a user selection of a rare event type. Under the broadest reasonable interpretation, this limitation recites steps of mere data gathering which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra solution activity (See MPEP 2106.05(d)(II)). Therefore, claim 10 does not solve the deficiencies of claim 9.
Regarding claim 11, it is dependent upon claim 9 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 11 recites wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence. Under the broadest reasonable interpretation, the limitation recites making a mental determination based on an observed confidence level which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 11 does not solve the deficiencies of claim 9.
Regarding claim 12, it is dependent upon claim 9 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 12 recites wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold. Under the broadest reasonable interpretation, the limitation recites counting a quantity of feature sets and making a mental determination based on the quantity which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 12 does not solve the deficiencies of claim 9.
Regarding claim 13, it is dependent upon claim 9 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 13 recites wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset. Under the broadest reasonable interpretation, the limitation recites observing subsets of data and choosing them based on a common characteristic which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 13 does not solve the deficiencies of claim 9.
Regarding claim 14, it is dependent upon claim 9 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 14 recites wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset. Under the broadest reasonable interpretation, the limitation recites making a mental determination based on observing the physical distance between locations which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 14 does not solve the deficiencies of claim 9.
Regarding claim 15, it is dependent upon claim 9 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 15 recites wherein the special machine learning model comprises a gradient boosted tree (GBT). Under the broadest reasonable interpretation, the limitation merely recites steps that apply a generic gradient boosted tree, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Therefore, claim 15 does not solve the deficiencies of claim 9.
Regarding claim 16, it is dependent upon claim 9 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 16 recites wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value. Under the broadest reasonable interpretation, the limitation merely recites making a mental determination based on observing the predicted target variables which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 16 does not solve the deficiencies of claim 9.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 9, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Xue, US Pre-Grant Publication US20230385605A1 (“Xue”) in view of Abbasi, et al., Non-Patent Literature “A large-scale benchmark dataset for anomaly detection and rare event classification for audio forensics” (“Abbasi”).
Regarding claim 1, Xue discloses:
A computer-implemented method comprising: training a naïve machine learning model to generate a predicted target variable for a rare event using a particular subset of the training data; (“Xue, paragraph [0027], “The autoencoder 210 is trained on only negatively labeled data [A computer-implemented method comprising: training a naïve machine learning model…using a particular subset of the training data;]. After training, the autoencoder 210 has learned the features of the normal, majority class. A well-trained autoencoder 210 will predict any new data that is coming from the majority class identified by a low reconstruction error 240. However, in trying to reconstruct a datapoint from a rare event, the autoencoder 210 will struggle and result in a high reconstruction error. Such a high reconstruction error can be noted and labeled as a rare event. [to generate a predicted target variable for a rare event]”).
determining that a sparsity of training data exists for the rare event in the particular subset of the training data; (Xue, paragraph [0026], “negatively labeled majority data at 225, which is a subset of training data 230 that includes only classes that are more common and not the rare or extremely rare culprits. [determining that a sparsity of training data exists for the rare event in the particular subset of the training data;]”).
in response to determining that a sparsity off training data exists for the rare event in the particular subset of the training data, selecting other subsets of the training data; (Xue, paragraph [0027]-[0028], “the training data is divided into two parts: positively labeled data 235 and negatively labeled data 225…a minority centric classifier, such as a few-shot DNN 250 can be trained on the rare or extremely rare positive labeled data [in response to determining that a sparsity off training data exists for the rare event in the particular subset of the training data, selecting other subsets of the training data;]”).
training a special machine learning model to generate the predicted target variable using the special training data set; (Xue, paragraph [0028], “a minority centric classifier, such as a few-shot DNN 250 can be trained on the rare or extremely rare positive labeled data 235. [training a special machine learning model to generate the predicted target variable using the special training data set;]”).
obtaining, as intermediate predicted target variables, the predicted target variable from each of the special machine learning model and the naïve machine learning model; (Xue, abstract, “A first machine learning network trained to classify events in a majority class is executed on the received information representative of the event. [obtaining, as intermediate predicted target variables, the predicted target variable from…the naïve machine learning model;] A second machine learning network trained to classify events in a minority class is executed on the received information representative of the event. [from the special machine learning model]”).
determining an ensembled predicted target variable based on the intermediate predicted target variables; (Xue, paragraph [0042], “At operation 540, classifications of the first and second machine learning models are combined to predict the class of the information representative of the event. [determining an ensembled predicted target variable based on the intermediate predicted target variables;]”).
and providing the ensembled predicted target variable for output. (Xue, paragraph [0019], “The pair of complementary neural networks combines the discrimination power of complemental models and computes a prediction output [and providing the ensembled predicted target variable for output.]”).
While Xue teaches a method of training machine learning models and aggregating training data,
Xue does not explicitly teach:
identifying feature sets that are associated with the rare event from the selected other subsets of the training data;
generating a special training data set that includes the feature sets that are associated with the rare events from the selected other subsets of the training data;
However, Abbasi teaches identifying feature sets that are associated with the rare event from the selected other subsets of the training data; (Abbasi, pg. 5 col 1-2, “Every audio signal contains a variety of characteristics/features. We must, however, extract the features related to the event that we will detect. [identifying feature sets that are associated with the rare event from the selected other subsets of the training data;]”).
Abbasi also teaches generating a special training data set that includes the feature sets that are associated with the rare events from the selected other subsets of the training data; (Abbasi, pg. 5 col 2, “We selected only suitable features and removed all non-useful features using Principal Component Analysis [32]. In the end, 65 most important features passed to models for anomaly detection. [generating a special training data set that includes the feature sets that are associated with the rare events from the selected other subsets of the training data;]”).
Xue and Abbasi are both in the same field of endeavor (i.e. machine learning for rare events). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue and Abbasi to teach the above limitations. The motivation for doing so is that identifying the features associated with rare events and using them as training data makes the machine learning model better at recognizing rare events in production (cf. Abbasi, pg. 4 col 1, “we need to remove these non-useful features and pass only useful features to machine learning classifiers for better classification results.”).
Regarding claim 9, it is similar to claim 1, taught by Xue in view of Abbasi, with the following additional limitations:
A system comprising: one or more processors,
and one or more computer-readable storage media that includes instructions which, when executed by the one or more processors, cause the one or more processors to perform operations
Xue teaches A system comprising: one or more processors, (Xue, claim 18, “A device comprising: a processor; [A system comprising: one or more processors,]”).
Xue also teaches and one or more computer-readable storage media that includes instructions which, when executed by the one or more processors, cause the one or more processors to perform operations (Xue, paragraph [0010], “computer executable instructions stored on computer readable media…may be executed on a digital signal processor, ASIC, microprocessor, or other type of processor [and one or more computer-readable storage media that includes instructions which, when executed by the one or more processors, cause the one or more processors to perform operations]”).
Xue and Abbasi are both in the same field of endeavor (i.e. machine learning for rare events). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue and Abbasi to teach the above limitations. The motivation for doing so is that identifying the features associated with rare events and using them as training data makes the machine learning model better at recognizing rare events in production (cf. Abbasi, pg. 4 col 1, “we need to remove these non-useful features and pass only useful features to machine learning classifiers for better classification results.”).
Regarding claim 17, it is similar to claim 1, taught by Xue in view of Abbasi, with the following additional limitation:
A computer-readable storage medium that includes instructions which, when executed by one or more processors, cause the one or more processors to perform operations
Xue teaches A computer-readable storage medium that includes instructions which, when executed by one or more processors, cause the one or more processors to perform operations (Xue, paragraph [0010], “computer executable instructions stored on computer readable media…may be executed on a digital signal processor, ASIC, microprocessor, or other type of processor [A computer-readable storage medium that includes instructions which, when executed by one or more processors, cause the one or more processors to perform operations]”).
Xue and Abbasi are both in the same field of endeavor (i.e. machine learning for rare events). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue and Abbasi to teach the above limitations. The motivation for doing so is that identifying the features associated with rare events and using them as training data makes the machine learning model better at recognizing rare events in production (cf. Abbasi, pg. 4 col 1, “we need to remove these non-useful features and pass only useful features to machine learning classifiers for better classification results.”).
Claims 2, 8, 10, 16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Xue, US Pre-Grant Publication US20230385605A1 (“Xue”) in view of Abbasi, et al., Non-Patent Literature “A large-scale benchmark dataset for anomaly detection and rare event classification for audio forensics” (“Abbasi”) and further in view of Kuklyte, Non-Patent Literature “Unusual Event Detection in Real-World Surveillance Applications” (“Kuklyte”).
Regarding claim 2, Xue in view of Abbasi teaches The method of claim 1. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises receiving a user selection of a rare event type.
However, Kuklyte teaches wherein identifying the rare event comprises receiving a user selection of a rare event type. (Kuklyte, pg. 18, “This work focuses on unusual event detection which can be formulated as a classification task between two types of scenarios- usual and unusual. [identifying the rare event]”. Kuklyte, pg. 13, “The Abstraction process starts with acquiring raw video data in a form of a sequence of video frames...which are then transformed into D-dimensional feature vectors f. The labels, or annotations, to the vectors f can be obtained through the manual assessment of the data in an off-line manner. It can also be collected from the user in an online method via user feedback. [comprises receiving a user selection of a rare event type]”).
Xue in view of Abbasi and Kuklyte are both in the same field of endeavor (i.e. rare event detection using machine learning). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuklyte to teach the above limitations. The motivation for doing so is that allowing users to select which events are considered rare improves the machine learning model’s ability to detect those events in the data compared to when it must learn which events are rare on its own. (cf. Kuklyte, pg. 158, “a supervised classification approach which when combined with the unsupervised approach improves the overall performance of the event detection system. The experimental results show the increased accuracy when the two methods are combined when compared to the unsupervised only approach. Therefore, making the application more reliable when deployed in the real-world environment.”).
Regarding claim 8, Xue in view of Abbasi teaches The method of claim 1. Xue in view of Abbasi does not teach:
wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value.
However, Kuklyte teaches wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value. (Kuklyte, pg. 151, “MAX-fusion method compares the predictions of the participating classifiers and the highest probability value for being unusual is set to be the final result. [wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value.]”).
Xue in view of Abbasi and Kuklyte are both in the same field of endeavor (i.e. rare event detection using machine learning). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuklyte to teach the above limitations. The motivation for doing so is that this ensembling technique improves model performance when it’s more important to avoid false negative results than to avoid false positive detections. (cf. Kuklyte, pg. 151, “This method is suitable for an event detection task where missed unusual events are less tolerable than false detections.”).
Regarding claim 10, Xue in view of Abbasi teaches The system of claim 9. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises receiving a user selection of a rare event type.
However, Kuklyte teaches wherein identifying the rare event comprises receiving a user selection of a rare event type. (Kuklyte, pg. 18, “This work focuses on unusual event detection which can be formulated as a classification task between two types of scenarios- usual and unusual. [identifying the rare event]”. Kuklyte, pg. 13, “The Abstraction process starts with acquiring raw video data in a form of a sequence of video frames...which are then transformed into D-dimensional feature vectors f. The labels, or annotations, to the vectors f can be obtained through the manual assessment of the data in an off-line manner. It can also be collected from the user in an online method via user feedback. [comprises receiving a user selection of a rare event type]”).
Xue in view of Abbasi and Kuklyte are both in the same field of endeavor (i.e. rare event detection using machine learning). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuklyte to teach the above limitations. The motivation for doing so is that allowing users to select which events are considered rare improves the machine learning model’s ability to detect those events in the data compared to when it must learn which events are rare on its own. (cf. Kuklyte, pg. 158, “a supervised classification approach which when combined with the unsupervised approach improves the overall performance of the event detection system. The experimental results show the increased accuracy when the two methods are combined when compared to the unsupervised only approach. Therefore, making the application more reliable when deployed in the real-world environment.”).
Regarding claim 16, Xue in view of Abbasi teaches The system of claim 9. Xue in view of Abbasi does not teach:
wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value.
However, Kuklyte teaches wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value. (Kuklyte, pg. 151, “MAX-fusion method compares the predictions of the participating classifiers and the highest probability value for being unusual is set to be the final result. [wherein determining the ensembled predicted target variable comprises selecting one of the intermediate predicated target variables that has a maximum value.]”).
Xue in view of Abbasi and Kuklyte are both in the same field of endeavor (i.e. rare event detection using machine learning). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuklyte to teach the above limitations. The motivation for doing so is that this ensembling technique improves model performance when it’s more important to avoid false negative results than to avoid false positive detections. (cf. Kuklyte, pg. 151, “This method is suitable for an event detection task where missed unusual events are less tolerable than false detections.”).
Regarding claim 18, Xue in view of Abbasi teaches The medium of claim 17. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises receiving a user selection of a rare event type.
However, Kuklyte teaches wherein identifying the rare event comprises receiving a user selection of a rare event type. (Kuklyte, pg. 18, “This work focuses on unusual event detection which can be formulated as a classification task between two types of scenarios- usual and unusual. [identifying the rare event]”. Kuklyte, pg. 13, “The Abstraction process starts with acquiring raw video data in a form of a sequence of video frames...which are then transformed into D-dimensional feature vectors f. The labels, or annotations, to the vectors f can be obtained through the manual assessment of the data in an off-line manner. It can also be collected from the user in an online method via user feedback. [comprises receiving a user selection of a rare event type]”).
Xue in view of Abbasi and Kuklyte are both in the same field of endeavor (i.e. rare event detection using machine learning). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuklyte to teach the above limitations. The motivation for doing so is that allowing users to select which events are considered rare improves the machine learning model’s ability to detect those events in the data compared to when it must learn which events are rare on its own. (cf. Kuklyte, pg. 158, “a supervised classification approach which when combined with the unsupervised approach improves the overall performance of the event detection system. The experimental results show the increased accuracy when the two methods are combined when compared to the unsupervised only approach. Therefore, making the application more reliable when deployed in the real-world environment.”).
Claims 3, 11, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Xue, US Pre-Grant Publication US20230385605A1 (“Xue”) in view of Abbasi, et al., Non-Patent Literature “A large-scale benchmark dataset for anomaly detection and rare event classification for audio forensics” (“Abbasi”) and further in view of Brandes, et al., US Pre-Grant Publication US20200364520A1 (“Brandes”).
Regarding claim 3, Xue in view of Abbasi teaches The method of claim 1. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence.
However, Brandes teaches wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence. (Brandes, paragraph [0048], “the system 100 may comprise an evaluator engine 104 adapted for a determination of an underrepresented class in the prediction classes. For this, the evaluator engine 104 is using a confidence level distribution of the predictions. This may be implemented as a relative thresholding scheme. If, for one class the confidence level for the prediction of the class is comparably low and other classes may show a comparably high value, this may speak for an uncertainty in regard to the specific prediction.” Brandes, paragraph [0056], “a rare case is determined if the evaluator engine determines that a difference between the main prediction class and a next prediction class in a ranked series of prediction classes is below a predefined difference threshold value. [wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence.]”).
Xue in view of Abbasi and Brandes are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Brandes to teach the above limitations. The motivation for doing so is that evaluating confidences levels for predictions allows a system or method to automatically determine when a prediction class corresponds to a rare event and autonomously enhance its training data. (cf. Brandes, paragraph [0015], “The proposed techniques may supervise itself and care about itself and find alternatives autonomously if a prediction of, e.g., a classifier is uncertain about its own results. An internal evaluator detects such imbalances and may look automatically for other examples to be used for enhancing a training data set.”).
Regarding claim 11, Xue in view of Abbasi teaches The system of claim 9. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence.
However, Brandes teaches wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence. (Brandes, paragraph [0048], “the system 100 may comprise an evaluator engine 104 adapted for a determination of an underrepresented class in the prediction classes. For this, the evaluator engine 104 is using a confidence level distribution of the predictions. This may be implemented as a relative thresholding scheme. If, for one class the confidence level for the prediction of the class is comparably low and other classes may show a comparably high value, this may speak for an uncertainty in regard to the specific prediction.” Brandes, paragraph [0056], “a rare case is determined if the evaluator engine determines that a difference between the main prediction class and a next prediction class in a ranked series of prediction classes is below a predefined difference threshold value. [wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence.]”).
Xue in view of Abbasi and Brandes are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Brandes to teach the above limitations. The motivation for doing so is that evaluating confidences levels for predictions allows a system or method to automatically determine when a prediction class corresponds to a rare event and autonomously enhance its training data. (cf. Brandes, paragraph [0015], “The proposed techniques may supervise itself and care about itself and find alternatives autonomously if a prediction of, e.g., a classifier is uncertain about its own results. An internal evaluator detects such imbalances and may look automatically for other examples to be used for enhancing a training data set.”).
Regarding claim 19, Xue in view of Abbasi teaches The medium of claim 17. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence.
However, Brandes teaches wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence. (Brandes, paragraph [0048], “the system 100 may comprise an evaluator engine 104 adapted for a determination of an underrepresented class in the prediction classes. For this, the evaluator engine 104 is using a confidence level distribution of the predictions. This may be implemented as a relative thresholding scheme. If, for one class the confidence level for the prediction of the class is comparably low and other classes may show a comparably high value, this may speak for an uncertainty in regard to the specific prediction.” Brandes, paragraph [0056], “a rare case is determined if the evaluator engine determines that a difference between the main prediction class and a next prediction class in a ranked series of prediction classes is below a predefined difference threshold value. [wherein identifying the rare event comprises determining that a target variable cannot be predicted for a rare event with a threshold level of confidence.]”).
Xue in view of Abbasi and Brandes are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Brandes to teach the above limitations. The motivation for doing so is that evaluating confidences levels for predictions allows a system or method to automatically determine when a prediction class corresponds to a rare event and autonomously enhance its training data. (cf. Brandes, paragraph [0015], “The proposed techniques may supervise itself and care about itself and find alternatives autonomously if a prediction of, e.g., a classifier is uncertain about its own results. An internal evaluator detects such imbalances and may look automatically for other examples to be used for enhancing a training data set.”).
Claims 4, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Xue, US Pre-Grant Publication US20230385605A1 (“Xue”) in view of Abbasi, et al., Non-Patent Literature “A large-scale benchmark dataset for anomaly detection and rare event classification for audio forensics” (“Abbasi”) and further in view of Kuhn, et al., US Pre-Grant Publication US20250149178A1 (“Kuhn”).
Regarding claim 4, Xue in view of Abbasi teaches The method of claim 1. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold.
However, Kuhn teaches wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold. (Kuhn, paragraph [0109], “As part of the analysis nuclear and/or cell masks are generated and features extracted (e.g., over 700 features pulled from the 4 fluorescent images)…Rare events are defined as 1) events within the smallest population clusters…In a refinement, rare events are below a predetermined rarity threshold. In a further refinement, the rare events are below a rarity threshold of 1.5%. [wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold.]”).
Xue in view of Abbasi and Kuhn are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuhn to teach the above limitations. The motivation for doing so is that counting the number of corresponding features is a straightforward way in which to define rare events. (cf. Kuhn, paragraph [0137], “Rare events were detected by the third-generation of our computational algorithm for unsupervised clustering, as previously described (34). In brief, this approach allows for the classification of cells into common and rare groups based on principal component analysis of cells' morphometric features and subsequent hierarchical clustering”).
Regarding claim 12, Xue in view of Abbasi teaches The system of claim 9. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold.
However, Kuhn teaches wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold. (Kuhn, paragraph [0109], “As part of the analysis nuclear and/or cell masks are generated and features extracted (e.g., over 700 features pulled from the 4 fluorescent images)…Rare events are defined as 1) events within the smallest population clusters…In a refinement, rare events are below a predetermined rarity threshold. In a further refinement, the rare events are below a rarity threshold of 1.5%. [wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold.]”).
Xue in view of Abbasi and Kuhn are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuhn to teach the above limitations. The motivation for doing so is that counting the number of corresponding features is a straightforward way in which to define rare events. (cf. Kuhn, paragraph [0137], “Rare events were detected by the third-generation of our computational algorithm for unsupervised clustering, as previously described (34). In brief, this approach allows for the classification of cells into common and rare groups based on principal component analysis of cells' morphometric features and subsequent hierarchical clustering”).
Regarding claim 20, Xue in view of Abbasi teaches The medium of claim 17. Xue in view of Abbasi does not teach:
wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold.
However, Kuhn teaches wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold. (Kuhn, paragraph [0109], “As part of the analysis nuclear and/or cell masks are generated and features extracted (e.g., over 700 features pulled from the 4 fluorescent images)…Rare events are defined as 1) events within the smallest population clusters…In a refinement, rare events are below a predetermined rarity threshold. In a further refinement, the rare events are below a rarity threshold of 1.5%. [wherein identifying the rare event comprises determining that a quantity of feature sets in the particular subset of training data fails to satisfy a threshold.]”).
Xue in view of Abbasi and Kuhn are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Kuhn to teach the above limitations. The motivation for doing so is that counting the number of corresponding features is a straightforward way in which to define rare events. (cf. Kuhn, paragraph [0137], “Rare events were detected by the third-generation of our computational algorithm for unsupervised clustering, as previously described (34). In brief, this approach allows for the classification of cells into common and rare groups based on principal component analysis of cells' morphometric features and subsequent hierarchical clustering”).
Claims 5-6 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Xue, US Pre-Grant Publication US20230385605A1 (“Xue”) in view of Abbasi, et al., Non-Patent Literature “A large-scale benchmark dataset for anomaly detection and rare event classification for audio forensics” (“Abbasi”) and further in view of Leite, et al., US Pre-Grant Publication US20240061278A1 (“Leite”).
Regarding claim 5, Xue in view of Abbasi teaches The method of claim 1. Xue in view of Abbasi does not teach:
wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset.
However, Leite teaches wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset. (Leite, paragraph [0117], “Once the first neural network is trained, results from the training process are used to generate a second training dataset for training the second neural network. The second neural network is trained by only using data that is close to the landmark feature in question (for example the three-dimensional position coordinates are within a certain range of the initial estimates of the landmark position outputted by the first neural network). [wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset.]”).
Xue in view of Abbasi and Leite are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Leite to teach the above limitations. The motivation for doing so is that training a second machine learning model on data that shares a common characteristic with the first model’s data results in better ensemble predictions compared to training them on unrelated data. (cf. Leite, paragraph [0116], “It has been found that the sequential training of the first and second neural networks in this manner can result in a particularly accurate landmark detection process.”).
Regarding claim 6, Xue in view of Abbasi teaches The method of claim 1. Xue in view of Abbasi does not teach:
wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset.
However, Leite teaches wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset. (Leite, paragraph [0117], “Once the first neural network is trained, results from the training process are used to generate a second training dataset for training the second neural network. The second neural network is trained by only using data that is close to the landmark feature in question (for example the three-dimensional position coordinates are within a certain range of the initial estimates of the landmark position outputted by the first neural network). [wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset.]”).
Xue in view of Abbasi and Leite are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Leite to teach the above limitations. The motivation for doing so is that training a second machine learning model on data that shares a common characteristic with the first model’s data (such as physical distance) results in better ensemble predictions compared to training them on unrelated data. (cf. Leite, paragraph [0116], “It has been found that the sequential training of the first and second neural networks in this manner can result in a particularly accurate landmark detection process.”).
Regarding claim 13, Xue in view of Abbasi teaches The system of claim 9. Xue in view of Abbasi does not teach:
wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset.
However, Leite teaches wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset. (Leite, paragraph [0117], “Once the first neural network is trained, results from the training process are used to generate a second training dataset for training the second neural network. The second neural network is trained by only using data that is close to the landmark feature in question (for example the three-dimensional position coordinates are within a certain range of the initial estimates of the landmark position outputted by the first neural network). [wherein selecting the other subsets comprises determining that the other subsets share a common characteristic with the particular subset.]”).
Xue in view of Abbasi and Leite are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Leite to teach the above limitations. The motivation for doing so is that training a second machine learning model on data that shares a common characteristic with the first model’s data results in better ensemble predictions compared to training them on unrelated data. (cf. Leite, paragraph [0116], “It has been found that the sequential training of the first and second neural networks in this manner can result in a particularly accurate landmark detection process.”).
Regarding claim 14, Xue in view of Abbasi teaches The system of claim 9. Xue in view of Abbasi does not teach:
wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset.
However, Leite teaches wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset. (Leite, paragraph [0117], “Once the first neural network is trained, results from the training process are used to generate a second training dataset for training the second neural network. The second neural network is trained by only using data that is close to the landmark feature in question (for example the three-dimensional position coordinates are within a certain range of the initial estimates of the landmark position outputted by the first neural network). [wherein selecting the other subsets comprises determining that the other subsets are associated with physical locations that are within a predetermined distance of a physical location that is associated with the particular subset.]”).
Xue in view of Abbasi and Leite are both in the same field of endeavor (i.e. rare event training data augmentation). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Leite to teach the above limitations. The motivation for doing so is that training a second machine learning model on data that shares a common characteristic with the first model’s data (such as physical distance) results in better ensemble predictions compared to training them on unrelated data. (cf. Leite, paragraph [0116], “It has been found that the sequential training of the first and second neural networks in this manner can result in a particularly accurate landmark detection process.”).
Claims 7 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Xue, US Pre-Grant Publication US20230385605A1 (“Xue”) in view of Abbasi, et al., Non-Patent Literature “A large-scale benchmark dataset for anomaly detection and rare event classification for audio forensics” (“Abbasi”) and further in view of Gupta, et al., Non-Patent Literature “Wind ramp event prediction with parallelized gradient boosted regression trees” (“Gupta”).
Regarding claim 7, Xue in view of Abbasi teaches The method of claim 1. Xue in view of Abbasi does not teach:
wherein the special machine learning model comprises a gradient boosted tree (GBT).
However, Gupta teaches wherein the special machine learning model comprises a gradient boosted tree (GBT). (Gupta, pg. 1 col 1, “A parallelized gradient boosted regression tree based technique has been proposed to accurately classify the normal as well as rare extreme wind power ramp events. [wherein the special machine learning model comprises a gradient boosted tree (GBT).]”).
Xue in view of Abbasi and Gupta are both in the same field of endeavor (i.e. rare event detection). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Gupta to teach the above limitations. The motivation for doing so is that gradient boosted trees are an effective and versatile ensemble learning technique. (cf. Gupta, pg. 5 col 2, “Gradient Boosted regression trees are a powerful class of models for learning patterns from a wide variety of datasets. They have been shown to be robust to overfitting, easy to train and compact to use at test time.”).
Regarding claim 15, Xue in view of Abbasi teaches The system of claim 9. Xue in view of Abbasi does not teach:
wherein the special machine learning model comprises a gradient boosted tree (GBT).
However, Gupta teaches wherein the special machine learning model comprises a gradient boosted tree (GBT). (Gupta, pg. 1 col 1, “A parallelized gradient boosted regression tree based technique has been proposed to accurately classify the normal as well as rare extreme wind power ramp events. [wherein the special machine learning model comprises a gradient boosted tree (GBT).]”).
Xue in view of Abbasi and Gupta are both in the same field of endeavor (i.e. rare event detection). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Xue in view of Abbasi and Gupta to teach the above limitations. The motivation for doing so is that gradient boosted trees are an effective and versatile ensemble learning technique. (cf. Gupta, pg. 5 col 2, “Gradient Boosted regression trees are a powerful class of models for learning patterns from a wide variety of datasets. They have been shown to be robust to overfitting, easy to train and compact to use at test time.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Qian, US Pre-Grant Publication US20220398501A1 “Dividing training data for aggregating results of multiple machine learning elements” discloses dividing training data into groups with different labels to train a group of ensemble classifiers and recursively combining their results based on different criteria.
Qi, et al., US Pre-Grant Publication US20200042904A1 “Advanced detection of rare events and corresponding interactive graphical user interface” discloses a system which receives parameters to categorize events before training, converts raw event data into features, and predicts which events belong to a rare category.
Tarler, et al., US Pre-Grant Publication US20210042824A1 “Methods, systems, and apparatuses for improved fraud detection and reduction” discloses a system which receives a selection of transaction event types for review. It separates transactions into the event types and uses their features to train a fraud detection model. The model is also used to generate fraud detection rules/functions.
Kim, et al., “Feature Selection Techniques for Improving Rare Class Classification in Semiconductor Manufacturing Process” discloses various methods of selecting training data features for a rare class prediction model including Euclidean distance, standard deviation, and over-sampling.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL BROWN whose telephone number is (571)270-5401. The examiner can normally be reached Monday-Friday (9:30 am - 5:30 pm) ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T Bechtold can be reached at 571-431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.B./Examiner, Art Unit 2148
/MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights