DETAILED ACTION
Status of the Claims
The following is a non-final Office Action in response to claims filed 28 February 2024.
Claims 1-15 are pending.
Claims 1-15 have been examined.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 28 February 2024 are being considered by the Examiner.
Priority
Applicant’s claim for the benefit of a prior-filed application(s) European Patent Application Number 23305291.9, filed 6 March 2023 under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, or 365(c) is acknowledged.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims are directed to a process (an act, or series of acts or steps), a machine (a concrete thing, consisting of parts, or of certain devices and combination of devices), and a manufacture (an article produced from raw or prepared materials by giving these materials new forms, qualities, properties, or combinations, whether by hand labor or by machinery). Thus, each of the claims falls within one of the four statutory categories (Step 1). The claims recite a method (process) and apparatus, however, the claim(s) recite(s) training machine learning models which is an abstract idea of a mathematical concept.
The limitations of “generating a series of training datasets wherein a first training dataset of said series of training datasets comprises an equal ratio of non-target cases and target cases and wherein the series of training datasets comprise a ratio of non-target to target cases that increases for each consecutive training dataset of the series of training data sets, training the machine learning model using the machine learning algorithm on each training dataset of the series of training datasets that is generated and recording the performance score that is obtained at each iteration, determining a maximum performance score among the performance score that is recorded at said each iteration, determining a ratio of target to non-target cases for the maximum performance score that is determined, training the machine learning model using the machine learning algorithm on a training dataset having said ratio of target to non-target cases that is determined, said training dataset comprising an optimized training dataset that is trained to obtain an optimized model,” as drafted, is a process that, under its broadest reasonable interpretation, covers mathematical concepts—mathematical relationships, mathematical formulas or equations, mathematical calculations but for the recitation of generic computer components (Step 2A Prong 1). Method claim 1 is devoid of structure whatsoever and thus is only directed towards an abstract idea. Next, claims 12 and 13, other than reciting “A non-transitory computer program comprising instructions which, when the non-transitory computer program is executed by a computer, cause the computer to carry out a method for optimizing detection of target cases in an imbalanced dataset by a machine learning model trained using a machine learning algorithm, said machine learning algorithm outputting a performance score when applied to a dataset, said imbalanced dataset comprising a number of non-target cases grouped in a majority class and a number of target cases grouped in a minority class for at least one given parameter, said method comprising:,” and “A device that optimizes a detection of target cases in an imbalanced dataset with a machine learning model trained using a machine learning algorithm, said machine learning algorithm outputting a performance score when applied to a dataset, said imbalanced dataset comprising a number of non-target cases grouped in a majority class and a number of target cases grouped in a minority class for at least one given parameter, said device comprising: a processor, and a memory, wherein said processor is configured to” respectively, nothing in the claim element precludes the step from the mathematical concept grouping. For example, but for the “...executed by a computer” in claim 12 or “...wherein said processor is configured to...” in claim 13 language, “determining” in the context of this claim encompasses the user manually generating or adjusting the ratio of target and non-target data points for training datasets for a machine learning algorithm, based upon some determined performance score, in order to train and optimize the model which is a fundamental mathematical concept of optimization including statistical analysis and data sampling, such as the Pareto distribution of 80% of outcomes come from 20% of causes as a way to reduce factors or causes within a dataset. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation as a mathematical concept, but for the recitation of generic computer components, then it falls within the “Mathematical Concepts” grouping of abstract ideas. Accordingly, the claim(s) recite(s) an abstract idea (Step 2A, Prong One: YES).
This judicial exception is not integrated into a practical application (Step 2A Prong Two). As noted above, method claim 1 is devoid of structure whatsoever and thus cannot integrate the claims into a practical application. Next, claims 12 and 13 only recites one additional element – using a computer or processor to perform the steps. The computer or processor in the steps is recited at a high-level of generality (i.e., as a generic processor performing a generic computer function of sorting data and performing arithmetic upon said data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Specifically the claims amount to nothing more than an instruction to apply the abstract idea using a generic computer or invoking computers as tools by adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.04(d)(I) discussing MPEP 2106.05(f). The recitation of “machine learning algorithm” in the limitations also merely indicates a field of use or technological environment in which the judicial exception is performed. Although the additional element “machine learning algorithm” limits the identified judicial exceptions, this type of limitation merely confines the use of the abstract idea to a particular technological environment (machine learning) and thus fails to add an inventive concept to the claims. See MPEP 2106.05(h). Accordingly, the combination of these additional elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea, even when considered as a whole (Step 2A Prong Two: NO).
The claim does not include a combination of additional elements that are sufficient to amount to significantly more than the judicial exception (Step 2B). As noted above, method claim 1 is devoid of structure whatsoever and thus cannot amount to significantly more. As discussed above with respect to integration of the abstract idea into a practical application (Step 2A Prong 2), the combination of additional elements of using a computer or processor in claims 12 and 13 to perform the steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.. Therefore, when considering the additional elements alone, and in combination, there is no inventive concept in the claim. As such, the claim(s) is/are not patent eligible, even when considered as a whole (Step 2B: NO).
Claims 2-11 and 14-15 recite(s) the additional limitation(s) further limiting the data and how the data is used (score and datasets) which is still directed towards the abstract idea previously identified and is not an inventive concept that meaningfully limits the abstract idea. Again, as discussed with respect to claims 1, 12, and 13, the claims are simply limitations which are no more than mere instructions to apply the exception using a computer or with computing components. Accordingly, the additional element(s) does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Even when considered as a whole, the claims do not integrate the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 1-15 are therefore not eligible subject matter, even when considered as a whole.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chakravorty et al. (US PG Pub. 2023/0306079).
As per claims 1, 12 and 13, Chakravorty discloses a method and non-transitory computer program comprising instructions which, when the non-transitory computer program is executed by a computer, cause the computer to carry out a method for; and a device comprising a processor and memory wherein the processor is configured to: optimizing detection of target cases in an imbalanced dataset by a machine learning model trained using a machine learning algorithm, said machine learning algorithm outputting a performance score when applied to a dataset, said imbalanced dataset comprising a number of non-target cases grouped in a majority class and a number of target cases grouped in a minority class for at least one given parameter, said method comprising (process for reducing class imbalance in a training dataset for machine learning, Chakravorty ¶99 and Fig. 5; methods, systems, computer-readable media, ¶5; processor, ¶6):
generating a series of training datasets wherein a first training dataset of said series of training datasets comprises an equal ratio of non-target cases and target cases and wherein the series of training datasets comprise a ratio of non-target to target cases that increases for each consecutive training dataset of the series of training data sets (Accordingly, systems, methods, and non-transitory computer-readable media are disclosed for reducing class imbalance in a training dataset for machine learning. An objective of embodiments is to increase the proportion of time series of a minority class in a training dataset by generating synthetic time series of the minority class and/or reducing the number of time series of the majority class, to thereby reduce class imbalance in the training dataset (e.g., within a tolerance). A further objective of some embodiments is to generate synthetic time series of the minority class using neighboring time series for each of a sampling of time series in the training dataset, for example, based on a distance between time samples in a pair of time series. A further objective of some embodiments is to train a machine-learning algorithm, such as a binary classifier, using a training dataset for which the class imbalance has been reduced by the disclosed techniques. Advantageously, the reduction of class imbalance in the training dataset produces greater accuracy in machine-learning classifiers that are trained by the more balanced training dataset, Chakravorty ¶60; begins with uniform distribution, ¶81) (Examiner notes the uniform distribution as including the equal ratio of targeted and non-targeted cases),
training the machine learning model using the machine learning algorithm on each training dataset of the series of training datasets that is generated and recording the performance score that is obtained at each iteration (train machine learning algorithm, Chakravorty ¶16),
determining a maximum performance score among the performance score that is recorded at said each iteration (In the event of class imbalance in the training dataset, there is the potential for giving unequal importance to false positives and false negatives. Thus, a good candidate for capturing the prediction accuracy of a machine-learning classifier is the F.sub.β score, which is also known as the F.sub.1 score when β=1. The F.sub.β score is a generalization of the harmonic mean of precision and recall and is given by:
PNG
media_image1.png
66
234
media_image1.png
Greyscale
, Chakravorty ¶112-¶113; iterations until ratio is reached, ¶19-¶21),
determining a ratio of target to non-target cases for the maximum performance score that is determined (In each embodiment, the goal may be to attain a ratio of the number of time series, belonging to the minority class, to the number of majority time series, belonging to the majority class, in the training dataset, that satisfies a predefined threshold or is within a tolerance range of a predefined threshold. It should be understood that, for a binary machine-learning classifier (i.e., having only two possible classes), the ideal value of this ratio is 1.0, Chakravorty ¶62),
training the machine learning model using the machine learning algorithm on a training dataset having said ratio of target to non-target cases that is determined, said training dataset comprising an optimized training dataset that is trained to obtain an optimized model (Experimentation demonstrated that, as measured by the F.sub.1 score, the prediction accuracy of a machine-learning model improves when trained on a dataset that has been resampled to reduce class imbalance according to the disclosed resampling techniques, Chakravorty ¶114; guaranteed to be within the top 10%, ¶86).
While Chakravorty discloses the ability to balance classification in machine learning data sets, Chakravorty does not expressly disclose the “non-target cases” and “target cases.”
However, the Examiner asserts that the cases, classification, or type of data to be balanced is simply a label for the data and adds little, if anything, to the claimed acts or steps and thus does not serve to distinguish over the prior art. Any differences related merely to the meaning and information conveyed through labels (i.e., a targeted case versus a desired data type) which does not explicitly alter or impact the steps of the method does not patentably distinguish the claimed invention from the prior art in terms of patentability (MPEP 2144.04).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to include “non-target cases” and “target cases” since the specific type of desired data does not functionally alter or relate to the steps of the method and merely labeling the information differently from that in the prior art does not patentably distinguish the claimed invention.
As per claims 2 and 14, Chakravorty discloses as shown above with respect to claims 1 and 13. Chakravorty further discloses extracting a subset dataset comprising a test dataset from the imbalanced dataset, applying the optimized model to said test dataset that is extracted to obtain a test performance score, comparing the test performance score to the maximum performance score, validating the optimized model when a difference between the maximum performance score and the test performance score is smaller than a predetermined model-optimized threshold (experiments of resampling techniques, Chakravorty ¶113; iterations until dataset within a tolerance threshold, ¶19-¶21).
As per claim 3, Chakravorty discloses as shown above with respect to claim 1. Chakravorty further discloses wherein, in the series of training datasets, an increase in count of the non-target cases is realized in increments from a second training dataset of the series of training datasets (In summary, up-sampling algorithm 300 may sample time series from the original dataset, and then iterate through pairings of each time series with one of its k-nearest neighbors that belongs to a minority class to create synthetic time series. The utilization of a neighborhood of k-nearest neighbors and a distance metric respects the temporal correlation between time samples, since it ensures that synthesized time samples will not be too far from the original samples. For each feature for each time sample, a synthetic feature value is generated for a synthetic time sample. When possible, each synthetic feature value is calculated to be closer to the feature value of a time sample that belongs to a minority class. In other words, the synthetic feature values mimic the feature values of time samples that belong to the minority class. As a result, the synthetic time series, which are created from these synthetic feature values, mimic time series that belong to the minority class. These synthetic time series can then be incorporated into a training dataset to increase the ratio of the number of time series that belong to the minority class to the total number of time series and/or the number of time series that belong to the majority class. Notably, up-sampling algorithm 300 is able to reuse already available time-series data, such that new time-series data do not need to be acquired, and can achieve the desired ratio while retaining the entire original dataset (e.g., without down-sampling), Chakravorty ¶90).
As per claims 4 and 15, Chakravorty discloses as shown above with respect to claims 1 and 13. Chakravorty further discloses wherein the series of training datasets are generated based on the imbalanced dataset by extracting of a portion of data of said imbalanced dataset, said portion of data comprising a reference training dataset, and then modifying said reference training dataset to obtain the training datasets of the series of training datasets with predefined increasing ratios (In summary, up-sampling algorithm 300 may sample time series from the original dataset, and then iterate through pairings of each time series with one of its k-nearest neighbors that belongs to a minority class to create synthetic time series. The utilization of a neighborhood of k-nearest neighbors and a distance metric respects the temporal correlation between time samples, since it ensures that synthesized time samples will not be too far from the original samples. For each feature for each time sample, a synthetic feature value is generated for a synthetic time sample. When possible, each synthetic feature value is calculated to be closer to the feature value of a time sample that belongs to a minority class. In other words, the synthetic feature values mimic the feature values of time samples that belong to the minority class. As a result, the synthetic time series, which are created from these synthetic feature values, mimic time series that belong to the minority class. These synthetic time series can then be incorporated into a training dataset to increase the ratio of the number of time series that belong to the minority class to the total number of time series and/or the number of time series that belong to the majority class. Notably, up-sampling algorithm 300 is able to reuse already available time-series data, such that new time-series data do not need to be acquired, and can achieve the desired ratio while retaining the entire original dataset (e.g., without down-sampling), Chakravorty ¶90).
As per claim 5, Chakravorty discloses as shown above with respect to claim 1. Chakravorty further discloses extracting a subset dataset, comprising a reference training dataset, from the imbalanced dataset reference training dataset, applying the machine learning model to said reference training dataset to obtain a reference baseline model and a reference performance score, comparing the maximum performance score with said reference performance score and validating the optimized model when the reference performance score is below the maximum performance score (As discussed elsewhere herein, up-sampling algorithm 300 may generate synthetic time series based on the dataset received in subprocess 510. Essentially, up-sampling algorithm 300 generates the synthetic time series by calculating average information from neighboring time series, to thereby reduce temporal bias by randomization and convex combination techniques. Up-sampling algorithm 300 ensures that there are sufficient time series, representing the minority class, to properly balance heavily imbalanced datasets, Chakravorty ¶102).
As per claim 6, Chakravorty discloses as shown above with respect to claim 2. Chakravorty further discloses wherein the extracting comprises splitting the imbalanced dataset between the test dataset and a reference training dataset, said reference training dataset being disjoint from the test dataset (subset of training dataset and neighboring time series dataset, Chakravorty ¶6-¶8).
As per claim 7, Chakravorty discloses as shown above with respect to claim 6. Chakravorty discloses wherein from the imbalanced dataset, the test dataset may be a selection of 20% of total data available, and a remaining 80% being used as a base dataset for generating the series of training datasets with their different ratios (In each embodiment, the goal may be to attain a ratio of the number of time series, belonging to the minority class, to the number of majority time series, belonging to the majority class, in the training dataset, that satisfies a predefined threshold or is within a tolerance range of a predefined threshold. It should be understood that, for a binary machine-learning classifier (i.e., having only two possible classes), the ideal value of this ratio is 1.0, Chakravorty ¶62) (Examiner interprets the ability to predefine the ratio as the ability to include a the test dataset may be a selection of 20% of total data available, and a remaining 80% being used as a base dataset for generating the series of training datasets with their different ratios).
In addition, the Examiner asserts that claim scope is not limited by claim language that suggests or makes optional but does not require steps to be performed, or by claim language that does not limit a claim to a particular structure. However, examples of claim language, although not exhaustive, that may raise a question as to the limiting effect of the language in a claim are: (A) "adapted to" or "adapted for" clauses; (B) "wherein" clauses; and (C) "whereby" clauses (See MPEP 2111.04). In the instant case, the recited wherein clause "wherein from the imbalanced dataset, the test dataset may be a selection of 20% of total data available, and a remaining 80% being used as a base dataset for generating the series of training datasets with their different ratios" is not a positive method step as it do not require any actual positive recited claim steps to be performed; nor does it modify any of the positively claimed method steps.
As per claim 8, Chakravorty discloses as shown above with respect to claim 1. Chakravorty further discloses receiving the imbalanced dataset (receive dataset, Chakravorty ¶6).
As per claim 9, Chakravorty discloses as shown above with respect to claim 1. Chakravorty further discloses selecting the machine learning algorithm (up-sampling algorithm, down-sampling algorithm, Chakravorty ¶62-¶63, ¶91-¶92).
As per claim 10, Chakravorty discloses as shown above with respect to claim 1. Chakravorty further discloses filtering the imbalanced dataset to keep only data associated with the at least one given parameter (feature or parameter as input, Chakravorty ¶17 and ¶65-¶66).
As per claim 11, Chakravorty discloses as shown above with respect to claim 1. Chakravorty further discloses enhancing the optimized model (training and deploying model, Chakravorty ¶64; resulting model is more accurate, ¶106).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure (additional art can be located on the PTO-892):
Adaboina et al. (US PG Pub. 2024/0419702) Methods and systems for automatic appeal authorization using machine learning algorithm.
Codella et al. (US PG Pub. 2016/0092789) Category Oversampling For Imbalanced Machine Learning.
Islam et al. (WO 2024253705) Machine learning fairness with limited protected attributes.
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to ANDREW B WHITAKER whose telephone number is (571)270-7563. The examiner can normally be reached on M-F, 8am-5pm, EST.
If attempts to reach the examiner by telephone are unsuccessful, the Examiner’s supervisor, Lynda Jasmin can be reached on (571) 272-6782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-
automated- interview-request-air-form
/ANDREW B WHITAKER/Primary Examiner, Art Unit 3629
Prosecution Insights