Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/27/2024 and 08/26/2024 have been considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. JP2023-079271, filed on 05/12/2023.
Claim Objections
Claims 4, 6, and 8 are objected to because of the following informalities:
Claim 4 (line 7) recites in part "false positive result" this should be --false-positive result-- as is consistent with Applicant specification.
Claim 6 (line 1) recites in part “the label unit:” this should be –the labeling unit— wherein claim 5 declares “a labeling unit” as consistent with Applicant specification.
Claim 8 (line 22) recites in part “by adding a second amount of noise a fourth set of features” this should be --by adding a second amount of noise to a fourth set of features—as consistent with Applicant specification.
Appropriate corrections are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1: The claim recites “an information processing device comprising:”; therefore, it is directed to the statutory category of a machine.
Step 2A Prong 1: This claim recites, inter alia:
identify, based on the set of analysis results, a high-variance subset from the original dataset that includes a first set of subjects associated with a first analysis result that achieves a variance threshold: These limitations recite a mentally performable process with or without the aid of pen and paper of identifying a high-variance subset from the original dataset that achieves a variance threshold consistent with Applicant specification [0072].
partition the first set second and of the high-variance subset into a first data partition and a second data partition, and
generate a third data partition that is a copy of the first data partition and a fourth data partition that is a copy of the second data partition,
generate a modified third data partition by modifying the third data partition and a modified fourth data partition by modifying the fourth data partition: These limitations recite a mentally performable process with or without the aid of pen and paper of splitting the a dataset into two partitions, and generating third and fourth partitions that are copies of the first and second, and modifying said third and fourth partitions consistent with Applicant specification [0032].
generate a set of analysis results and generate a second analysis result: These limitations recite a mentally performable process with the aid of pen and paper of creating a set of analysis results to identify a subset that has a high level of variance between outcomes for classifications into categories such as true positive, false positive, true negatives and false negatives to determine prediction accuracy consistent with Applicant’s specification [0107]-[0108].
generate a final analysis result for the first set of subjects of the high-variance subset by aggregating and classifying the first analysis result and the second analysis result: These limitations recite a mentally performable process with or without the aid of pen and paper of creating a final analysis result by aggregating and classifying the first and second analysis results by using any conventional machine learning result aggregation technique consistent with Applicant specification [0061].
Thus, this claim recites a judicial exception.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of this claim are as follows:
An information processing device comprising: a processor; and a memory containing computer instructions executable by the processor to cause the processor to function as: a variance analysis unit configured to: and a partition unit configured to: and a modification unit configured to: and a result generation unit configured to: These additional elements are generic computer components used as a tool in their ordinary capacity to carry out the underlying abstract ideas. See MPEP 2106.05(f).
by processing an original dataset with a first set of machine learning models and by processing the original dataset with the second set of machine learning models: These additional elements are recited at a high level of generality and merely amount to invoking computers or other machinery, e.g. processing an original dataset with a first set of machine learning models, to apply the underlying mental processes to generate analysis results. See MPEP 2106.05(f).
train a second set of machine learning models using the first data partition, the second data partition, the modified third data partition and the modified fourth data partition: These additional elements are recited at a high level of generality to recite the idea of training a second set of machine learning models but fails to recite details of how the training reflects an inventive concept by using “the first data partition, the second data partition, the modified third data partition and the modified fourth data partition”, thus this limitation merely amounts to a recitation of the words “apply it” and does not integrate the judicial exception into a practical application. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include adding the words equivalent to “apply it” with the judicial exception. Additional elements further include invoking computers or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 2
Step 1: a machine, as in claim 1.
Step 2A Prong 1: This claim recites, inter alia:
generate the modified third data partition by adding, based on a noise level criterion, a first amount of noise to a third set of features of the third data partition; and
generate the modified fourth data partition by adding, based on the noise level criterion, a second amount of noise to a fourth set of features of the fourth data partition: This limitation encompasses a mental process with or without the aid of pen and paper of creating a copy of the third and fourth data partitions and adding unspecified amounts of noise to said copies of data partitions which encompasses a method that can be implemented using a pen and paper consistent with Applicant specification [0059].
Step 2A Prong 2 & Step 2B: There are no additional elements recited so this claim does not provide a practical application and is not considered to be significantly more. As such, this claim is patent ineligible.
Claim 3
Step 1: a machine, as in claim 2.
Step 2A Prong 1: This claim recites, inter alia:
identify, from among the set of sample prediction results, a subset of sample prediction results that satisfy a predetermined accuracy threshold; and
determine, as the noise level criterion, a maximum amount of noise that was added to the modified sample datasets associated with the subset of sample prediction results that satisfy the predetermined accuracy threshold: These limitations further encompass a mental process of using judgement/evaluation with or without the aid of pen and paper of identifying a subset of sample prediction results that satisfy a predetermined accuracy threshold and determining the maximum amount of noise added to the modified dataset that satisfies said accuracy threshold consistent with Applicant’s specification [0079].
generate, a set of sample prediction results: These limitations recite a mentally performable process with the aid of pen and paper of creating a set of analysis results to identify a subset that has a high level of variance between outcomes for classifications into categories such as true positive, false positive, true negatives and false negatives to determine prediction accuracy consistent with Applicant’s specification [0107]-[0108].
Thus, this claim recites a judicial exception.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of this claim are as follows:
by using the second set of machine learning models to iteratively process modified sample datasets to which increasing amounts of noise have been added: These additional elements merely recite insignificant extra-solution activity of mere repetitive calculations wherein the second set of machine learning models iteratively process modified sample datasets to which increasing amounts of noise have been added which is a common use of machine learning models. See MPEP 2106.05(g).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: These additional elements from Step 2A Prong 2 include insignificant extra-solution activities of output recited by “by using the second set of machine learning models to iteratively process modified sample datasets to which increasing amounts of noise have been added”. These insignificant extra-solution activities are well-understood routine and conventional activities similar to performing repetitive calculations see MPEP 2106.05(d)(II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 4
Step 1: a machine, as in claim 1.
Step 2A Prong 1: This claim recites, inter alia:
evaluate an accuracy factor of the first analysis result by comparing the first analysis result with respect to a ground-truth result for the original dataset;
partition a first subset of the first set of subjects associated with a true-positive result into the first data partition; and partition a second subset of the first set of subjects associated with a false positive result into the second data partition: this limitation further encompasses a mental process with or without the aid of pen and paper of evaluating an accuracy factor by comparing the first analysis result with respect to a ground-truth result for the original dataset and partitioning two subsets of results, one for true-positives and the other for false-positives consistent with Applicant’s specification [0077].
Step 2A Prong 2 & Step 2B: There are no additional elements recited so this claim does not provide a practical application and is not considered to be significantly more. As such, this claim is patent ineligible.
Claim 5
Step 1: a machine, as in claim 1.
Step 2A Prong 1: This claim recites, inter alia:
assigning outcome labels of true positive or false positive to the modified third data partition and the modified fourth data partition: These limitations further encompass a mental process of using judgement/evaluation with or without the aid of pen and paper of assigning outcome labels to the modified third and fourth data partitions consistent with Applicant specification [0044].
Thus, this claim recites a judicial exception.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of this claim are as follows:
a labeling unit for: This additional element is a generic computer components used as a tool in its ordinary capacity to carry out the underlying abstract ideas. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: These additional elements from Step 2A Prong 2 include invoking computers or other machinery to apply the underlying judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 6
Step 1: a machine, as in claim 5.
Step 2A Prong 1: This claim recites, inter alia:
calculates a first similarity factor between a first subset of subjects of the first data partition and a third subset of subjects of the modified third data partition;
assigns, in a case that the first similarity factor achieves a similarity threshold, a true-positive result label to the third subset of subjects;
assigns, in a case that the first similarity factor does not achieve a similarity threshold, a false-positive result label to the third subset of subjects;
calculates a second similarity factor between a second subset of subjects of the second data partition and a fourth subset of subjects of the modified fourth data partition;
assigns, in a case that the second similarity factor achieves a similarity threshold, a false-positive result label to the fourth subset of subjects; and
assigns, in a case that the second similarity factor does not achieve a similarity threshold, a true-positive result label to the fourth subset of subjects: this limitation further encompasses a mental process with or without the aid of pen and paper of calculating a similarity factor between a first subset of subjects and a third set subset of subjects between the first and modified third data partitions, assigning a true-positive or false-positive label to the third subset of subjects, and applying the same method of calculating a similarity factor between a second set of subjects and a fourth set of subjects between the second and modified fourth data partitions, assigning, again, a true-positive or false-positive label to the fourth subset of subjects with no further elaboration on the process consistent with Applicant specification [0083].
Step 2A Prong 2 & Step 2B: There are no additional elements recited so this claim does not provide a practical application and is not considered to be significantly more. As such, this claim is patent ineligible.
Claim 7
Step 1: This claim is directed to “An information processing system comprising…”; therefore, this claim is directed to the statutory category of machines.
Step 2A Prong 1: This claim recites, inter alia:
identify, based on the set of analysis results, a high-variance subset from the original dataset that includes a first set of subjects associated with a first analysis result that achieves a variance threshold: These limitations recite a mentally performable process with or without the aid of pen and paper of identifying a high-variance subset from the original dataset that achieves a variance threshold consistent with Applicant specification [0072].
partition the first set of subjects of the high-variance subset into a first data partition and a second data partition, and
generate a third data partition that is a copy of the first data partition and a fourth data partition that is a copy of the second data partition,
generate a modified third data partition by modifying the third data partition and a modified fourth data partition by modifying the fourth data partition: These limitations recite a mentally performable process with or without the aid of pen and paper of splitting the a dataset into two partitions, and generating third and fourth partitions that are copies of the first and second, and modifying said third and fourth partitions consistent with Applicant specification [0032].
generate a set of analysis results and generate a second analysis result: These limitations recite a mentally performable process with the aid of pen and paper of creating a set of analysis results to identify a subset that has a high level of variance between outcomes for classifications into categories such as true positive, false positive, true negatives and false negatives to determine prediction accuracy consistent with Applicant’s specification [0107]-[0108].
generate a final analysis result for the first set of subjects of the high-variance subset by aggregating and classifying the first analysis result and the second analysis result: These limitations recite a mentally performable process with or without the aid of pen and paper of creating a final analysis result by aggregating and classifying the first and second analysis results by using any conventional machine learning result aggregation technique consistent with Applicant specification [0061].
Thus, this claim recites a judicial exception.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of this claim are as follows:
An information processing system comprising:
an information processing device; and
a user terminal;
a processor; and
a memory containing computer instructions executable by the processor to cause the processor to function as:
a variance analysis unit configured to: and a partition unit configured to: and a modification unit configured to: and a result generation unit configured to: These additional elements are generic computer components used as a tool in its ordinary capacity to carry out the underlying abstract ideas. See MPEP 2106.05(f).
by processing an original dataset with a first set of machine learning models and by processing the original dataset with the second set of machine learning models: These additional elements are recited at a high level of generality and merely amount to invoking computers or other machinery, e.g. processing an original dataset with a first set of machine learning models, to apply the underlying mental processes to generate analysis results. See MPEP 2106.05(f).
train a second set of machine learning models using the first data partition, the second data partition, the modified third data partition and the modified fourth data partition: These additional elements are recited at a high level of generality to recite the idea of training a second set of machine learning models but fails to recite details of how the training reflects an inventive concept by using “the first data partition, the second data partition, the modified third data partition and the modified fourth data partition”, thus this limitation merely amounts to a recitation of the words “apply it” and does not integrate the judicial exception into a practical application. See MPEP 2106.05(f).
output the final analysis result to the user terminal: These additional elements are recited at a high level of generality and merely amount to invoking computers or other machinery, e.g. presenting analysis results via a user terminal as a tool, to apply the underlying mental processes corresponding to generating an analysis result. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include invoking computers or other machinery to apply the underlying judicial exception. Additional elements further include adding the words equivalent to “apply it” with the judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 8
Step 1: This claim is directed to “An information processing method carried out by a computer, the information processing method comprising steps of”; therefore, this claim is directed to the statutory category of a process.
Step 2A Prong 1: This claim recites, inter alia:
identifying, based on the set of analysis results, a high-variance subset from the original dataset that includes a first set of subjects associated with a first analysis result that achieves a variance threshold: These limitations recite a mentally performable process with or without the aid of pen and paper of identifying a high-variance subset from the original dataset that achieves a variance threshold consistent with Applicant specification [0072].
partitioning the first set of subjects of the high-variance subset into a first data partition and a second data partition;
generating a third data partition that is a copy of the first data partition and a fourth data partition that is a copy of the second data partition: These limitations recite a mentally performable process with or without the aid of pen and paper of splitting the a dataset into two partitions, and generating third and fourth partitions that are copies of the first and second partitions consistent with Applicant specification [0032]
identifying, from among the set of sample prediction results, a subset of sample prediction results that satisfy a predetermined accuracy threshold;
determining, as a noise level criterion, a maximum amount of noise that was added to the modified sample datasets associated with the subset of sample prediction results that satisfy the predetermined accuracy threshold;
generating, by adding a first amount of noise to a third set of features of the third data partition based on the noise level criterion, a modified third data partition;
generating, by adding a second amount of noise a fourth set of features of the fourth data partition based on the noise level criterion, a modified fourth data partition: These limitations encompass a mental process with or without the aid of pen and paper of identifying a set of predictions results that satisfy an accuracy threshold, determining the maximum amount of noise associated with the subsets that satisfies the accuracy threshold and generating a first and second amounts of noise to a third and fourth set of features to the third and fourth data partitions respectively consistent with the Applicant’s specification [0059].
generating a set of analysis results and generating a set of sample prediction results and generating a second analysis result: These limitations recite a mentally performable process with the aid of pen and paper of creating a set of analysis results to identify a subset that has a high level of variance between outcomes for classifications into categories such as true positive, false positive, true negatives and false negatives to determine prediction accuracy consistent with Applicant’s specification [0107]-[0108].
generating a final analysis result for the first set of subjects of the high-variance subset by aggregating and classifying the first analysis result and the second analysis result: These limitations recite a mentally performable process with or without the aid of pen and paper of creating a final analysis result by aggregating and classifying the first and second analysis results by using any conventional machine learning result aggregation technique consistent with Applicant specification [0061].
Thus, this claim recites a judicial exception.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of this claim are as follows:
An information processing method carried out by a computer, the information processing method comprising steps of: These additional elements are generic computer components used as a tool in its ordinary capacity to carry out the underlying abstract ideas. See MPEP 2106.05(f).
by processing an original dataset with a first set of machine learning models and by processing the original dataset with the second set of machine learning model: These additional elements are recited at a high level of generality and merely amount to invoking computers or other machinery, e.g. processing an original dataset with a first set of machine learning models, to apply the underlying mental processes to generate analysis results. See MPEP 2106.05(f).
training a second set of machine learning models using the first data partition, the second data partition, the modified third data partition and the modified fourth data partition: These additional elements are recited at a high level of generality to recite the idea of training a second set of machine learning models but fails to recite details of how the training reflects an inventive concept by using “the first data partition, the second data partition, the modified third data partition and the modified fourth data partition”, thus this limitation merely amounts to a recitation of the words “apply it” and does not integrate the judicial exception into a practical application. See MPEP 2106.05(f).
by using a test set of machine learning models to iteratively process modified sample datasets to which increasing amounts of noise have been added: These additional elements merely recite insignificant extra-solution activity of mere repetitive calculations wherein the second set of machine learning models iteratively process modified sample datasets to which increasing amounts of noise have been added which is a common use of machine learning models. See MPEP 2106.05(g).
Thus, the way in which the additional elements use or interact with the judicial exception when analyzed with this claim as a whole do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include insignificant extra-solution activities of adding noise to the modified datasets recited by “by using a test set of machine learning models to iteratively process modified sample datasets to which increasing amounts of noise have been added”. These insignificant extra-solution activities are well-understood routine and conventional activities similar to performing repetitive calculations see MPEP2106.05(d)(II). Additional elements further include invoking computers or other machinery to apply the underlying exception. Additional elements further include adding the words equivalent to “apply it” with the judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2016.05.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8 are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (Learning from Imbalanced Data Sets with Boosting and Data Generation: The DataBoost-IM Approach) (hereinafter Guo) in view of Hong et al. (US 2022/0188703 A1) (hereinafter Hong).
Guo and Hong were cited in the IDS filed 08/26/2024.
Regarding claim 1, Guo teaches generate, by processing an original dataset with a first set of machine learning models, a set of analysis results ([par. 3 Section 1] “Boosting produces a series of classifiers and the outputs of these classifiers are combined using weighted voting in the final prediction of the model [10]. In each step of the series, the training examples are re-weighted and selected based on the performance of earlier classifiers in the training series. This produces a set of “easy” examples with low weights and a set of hard ones with high weights.”);
identify, based on the set of analysis results, a high-variance subset from the original dataset that includes a first set of subjects associated with a first analysis result that achieves a variance threshold ([par. 3 Section 1] "Boosting is an ensemble method where the performance of weak classifiers is improved by focusing on hard examples which are difficult to classify; Bin each step of the series, the training examples are re-weighted and selected based on the performance of earlier classifiers in the training series.);
partition the first set of subjects of the high-variance subset into a first data partition and a second data partition ([par. 1 Section 2.1] “The aim of the seed selection process is to identify hard examples for both the majority and minority classes[Section 2.1 1-2]; Wherein E sub smin and E sub smaj or their subsets are the first and second data partitions, see also Table 1 and page 32 left-hand column second paragraph [Section 2.1, 9-16]”);
generate a third data partition that is a copy of the first data partition and a fourth data partition that is a copy of the second data partition, ([par. 1 Section 2] “We separately identify hard examples from, and generate synthetic examples for, the minority as well as the majority class [8-10]; Secondly, the hard examples (so called seed examples) are identified for each of these seed examples, a set of synthetic examples is generated [10]”);
a modification unit configured to generate a modified third data partition by modifying the third data partition and a modified fourth data partition by modifying the fourth data partition ([par 2 Section 2] “Secondly, the hard examples (so-called seed examples) are identified and for each of these seed examples, a set of synthetic examples is generated. During the third stage of the algorithm, the synthetic examples are added to the original training set and the class distribution and the total weights of different classes are rebalanced”);
generate, by processing the original dataset with the second set of machine learning models, a second analysis result ([Abstract, 14-17]”) “The synthetic data are then added to the original training set, and the class distribution and the total weights of the different classes in the new training set are rebalanced);
generate a final analysis result for the first set of subjects of the high-variance subset by aggregating and classifying the first analysis result and the second analysis result ([par 4 Section 2.3] Produce data generation set sizes which augment the original training set; As a result, the total weights of the majority and minority classes in the new training set are equal to 122.51, thus equally distributing the balance of the two classes.).
Guo does not expressly teach an information processing device comprising and a processor and a memory containing computer implementable instructions executable by the processor to cause the processor to function as and a variance analysis unit configured to: and a variance analysis unit configured to: and a result generation unit configured to: and train a second set of machine learning models using the first data partition, the second data partition, the modified third data partition and the modified fourth data partition.
However, Hong teaches an information processing device comprising ([0011] “a data preprocessing unit that normalizes the first dataset”) and a processor ([0148] “As illustrated in FIG. 17, a computer 200 includes a central processing unit (CPU) 201 that executes various types of computation processing”) and a memory containing computer implementable instructions executable by the processor to cause the processor to function as ([0012] “there is provided a non-transitory computer-readable recording medium storing a machine learning program for causing a computer to execute a process”) and a variance analysis unit configured to ([0018] “a validation unit that validates the generated first learning model using the first test dataset”) and a result generation unit configured to: ([0028] “semi-synthetic data generation unit”) and train a second set of machine learning models using the first data partition, the second data partition, the modified third data partition and the modified fourth data partition ([0069] “the learning model generation unit 130 may train the second machine learning model using the second training dataset, and optimize the hyperparameters of the second machine learning model”).
Because Guo and Hong are analogous arts and within the same field of endeavor, specifically computer-implemented methods of training machine learning models on small data samples with high amounts of variance, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the device structure and added 2nd step training of Hong with the DataBoost-IM approach into the classification of Guo, with a reasonable expectation of success. Implementing DataBoost-IM into a fully functioning system would have been motivated by the desire to increase accuracy of prediction and classification given small or skewed training data (Hong [0002] - [0004]).
Regarding claim 7, it is an information processing system that is substantially the same as the device of claim 1. Thus claim 7 is rejected for the same reason as the device of claim 1. In addition, Hong teaches an information processing system comprising: an information processing device; and a user terminal; wherein the information processing device includes: a processor; and a memory containing computer implementable instructions executable by the processor to cause the processor to function as: a variance analysis unit configured to: ([0031] “a data generation apparatus according to an embodiment of the present invention”).
Regarding claim 8, it is an information processing method that is substantially the same as the device of claim 1. Thus claim 8 is rejected for the same reason as the device of claim 1. In addition, Hong teaches an information processing method carried out by a computer ([0031] “A data generation method performed by a data generation apparatus according to another embodiment of the present invention”) and identifying, from among the set of sample prediction results, a subset of sample prediction results that satisfy a predetermined accuracy threshold ([par 2 Section 2] “Secondly, the hard examples (so-called seed examples) are identified and for each of these seed examples, a set of synthetic examples is generated. During the third stage of the algorithm, the synthetic examples are added to the original training set and the class distribution and the total weights of different classes are rebalanced.”) and determining, as a noise level criterion, a maximum amount of noise that was added to the modified sample datasets associated with the subset of sample prediction results that satisfy the predetermined accuracy threshold ([par 2 Section 2] “The second and third stages of the DataBoost-IM algorithm are re-executed until reaching a user specified number of iterations or the current component classifier’s error rate is worse than a threshold value”) and generating, by adding a first amount of noise to a third set of features of the third data partition based on the noise level criterion, a modified third data partition ([par 2 Section 2] During the third stage of the algorithm, the synthetic examples are added to the original training set and the class distribution and the total weights of different classes are rebalanced) and generating, by adding a second amount of noise a fourth set of features of the fourth data partition based on the noise level criterion, a modified fourth data partition ([par 2 Section 2]During the third stage of the algorithm, the synthetic examples are added to the original training set and the class distribution and the total weights of different classes are rebalanced).
Regarding claim 2, the combination of Guo and Hong teaches the information processing device according to claim 1, wherein the modification unit is configured to: generate the modified third data partition by adding, based on a noise level criterion, a first amount of noise to a third set of features of the third data partition and generate the modified fourth data partition by adding, based on the noise level criterion, a second amount of noise to a fourth set of features of the fourth data partition. ([par 2 Section 2.2] “The values are chosen to reflect the distribution of values contained in the original training attribute with respect to the particular class. This is achieved by considering, for each class, the number of occurrences of different attribute values in the original data set; These 123 values are randomly assigned to the 123 examples created during data generation”).
Regarding claim 3, the combination of Guo and Hong teaches the information processing device according to claim 2, wherein the modification unit is configured to: generate, by using the second set of machine learning models to iteratively process modified sample datasets to which increasing amounts of noise have been added, a set of sample prediction results ([par 2 Section 2] “The second and third stages of the DataBoost-IM algorithm are re-executed until reaching a user specified number of iterations or the current component classifier’s error rate is worse than a threshold value”) and identify, from among the set of sample prediction results, a subset of sample prediction results that satisfy a predetermined accuracy threshold ([par 1 Section 2.1] “The aim of the seed selection process is to identify hard examples for both the majority and minority classes. These examples are used as input for the data generation process as discussed in Section 2.2”). Additionally, Hong teaches determine, as the noise level criterion, a maximum amount of noise that was added to the modified sample datasets associated with the subset of sample prediction results that satisfy the predetermined accuracy threshold ([par 1 Section 2.2] “generate additional synthesis instances to add to the original training set Etrain. The data generation process extends our earlier work, as presented in [13, 17, 18], by generating data for the majority and minority classes separately. That is, the data generation process generates two sets of data. Firstly a total of ML sets of new majority class examples, based on each seed instance in Emaj, are generated. For each attribute included in the synthetic example, a new value is generated based on the following constraints [13, 17, 18]”).
Regarding dependent claim 4, the combination of Guo and Hong teaches the information processing device according to claim 1, wherein the partition unit is configured to: evaluate an accuracy factor of the first analysis result by comparing the first analysis result with respect to a ground-truth result for the original dataset ([par 2 Section 2] “During the third stage of the algorithm, the synthetic examples are added to the original training set and the class distribution and the total weights of different classes are rebalanced”) and partition a second subset of the first set of subjects associated with a false positive result into the second data partition ([par 3 Section 3] “Another criteria used to evaluate a classifier’s performance on skew data is the G-mean [4, 5, 7]. The G-mean is defined as curacy sqrt( NegativeAccuracy * PositiveAccuracy) where Positive Accuracy and Negative Accuracy are calculated as TP/(FN+TP) and TN/(TN+FP). This measure relates to a point on the ROC curve and the idea is to maximize the accuracy on each of the two classes while keeping these accuracies balanced”). Additionally, regarding the following limitation: partition a first subset of the first set of subjects associated with a true-positive result into the first data partition, upon virtue of parent, this is objected to, but allowable when parent is fixed.
Regarding claim 5, the combination of Guo and Hong teaches the information processing device according to claim 1, further comprising: a labeling unit for assigning outcome labels of true positive or false positive to the modified third data partition and the modified fourth data partition. (Guo [par 3 Section 3] “Another criteria used to evaluate a classifier’s performance on skew data is the G-mean [4, 5, 7]. The G-mean is defined as curacy sqrt( NegativeAccuracy * PositiveAccuracy ) where Positive Accuracy and Negative Accuracy are calculated as TP/(FN+TP) and TN/(TN+FP).”; Hong [0083] “FIG. 5 is a graph illustrating a receiver operating characteristic (ROC) of data generated for the semi-synthetic data generation method according to an embodiment of the present invention of FIG. 3. Here, the receiver operating characteristic (ROC) shows a value of a true positive rate (TP) as a ratio with respect to a value of a false positive rate (FP)”)
Regarding claim 6, the combination of Guo and Hong teaches the information processing device according to claim 5, wherein the label unit: calculates a first similarity factor between a first subset of subjects of the first data partition and a third subset of subjects of the modified third data partition; ([par 2 Section 2.3] “Before the generated data are added to the original data set, each of the synthetic examples is assigned an initial weight. The initial weight of each example is calculated by dividing the weight of the seed example by the number of instances generated from it”) and assigns, in a case that the first similarity factor achieves a similarity threshold, a true-positive result label to the third subset of subjects; ([Table 5 Section 3.2] “For each 10-fold cross validation the data set was first partitioned into 10 equal sized sets and each set was then in turn used as the test set while the classifier trains on the other nine sets. A stratified sampling technique was applied here to ensure that each of the sets had the same proportion of different classes”) and calculates a second similarity factor between a second subset of subjects of the second data partition and a fourth subset of subjects of the modified fourth data partition; ([par 2 Section 2.3] “Before the generated data are added to the original data set, each of the synthetic examples is assigned an initial weight. The initial weight of each example is calculated by dividing the weight of the seed example by the number of instances generated from it”).
Guo does not expressly teach assigns, in a case that the second similarity factor does not achieve a similarity threshold, a true-positive result label to the fourth subset of subjects and assigns, in a case that the second similarity factor achieves a similarity threshold, a false-positive result label to the fourth subset of subjects However, Hong teaches assigns, in a case that the second similarity factor does not achieve a similarity threshold, a true-positive result label to the fourth subset of subjects and assigns, in a case that the second similarity factor achieves a similarity threshold, a false-positive result label to the fourth subset of subjects. ([0084] “In Table 1, the precision is expressed as a ratio of a relationship between the true positive rate and the false positive rate.”)
Because Guo and Hong are analogous arts and within the same field of endeavor, specifically computer-implemented methods of training machine learning models on small data samples with high amounts of variance, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to assign the true-positive and false-positive labels to the fourth set of subjects to calculate the ratio of the relationship between the true-positives and false-positives into of Huong into the data boosting of Guo. Obtaining the ratio of true-positives to true-negatives is motivated by the ability to obtain the precision of the classification which can be calculated given the formula TP/(TP+FP) [0084].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Okawa et al. (US 20240202601 A1) machine learning program, method and apparatus that uses a small batch of data to continuously train and update the parameters of a machine learning model, using clustering techniques to modify the training data between iterations for increased prediction accuracy.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER W CHAY whose telephone number is (571)270-5714. The examiner can normally be reached Monday-Friday (7:30am - 5:00pm) Off every other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER WON-BIN CHAY/Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
