The present application, filed on or after 16 March 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to Applicant’s submission filed on 27 March 2023. THIS ACTION IS NON-FINAL.
Status of Claims
Claims 1-20 are pending.
Claim 1-20 are rejected under 35 U.S.C. 101 for being directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
There is no art rejection for claims 1-20.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Judicial Exception
Claims 1-20 of the claimed invention are directed to a judicial exception, an abstract idea, without significantly more.
(Independent Claims) With regards to claim 1 / 11 / 16, the claim recites a process / machine / article of manufacturing, which falls into one of the statutory categories.
2A – Prong 1: the claim, in part, recites
“verifying validity of a trained artificial intelligence model” (mental process), “the verifying validity including: generating a training dataset from the trained artificial intelligence model using reverse data generation of the trained artificial intelligence model” (mental process); “comparing the training dataset generated using the reverse data generation with a test dataset used to evaluate the trained artificial intelligence model, the comparing to determine a relationship between the training dataset that was generated and the test dataset” (mental process); “removing data from the test dataset determined to have a predefined relationship with the training dataset to obtain a new test dataset” (mental process); and “using the new test dataset to verify the validity of the trained artificial intelligence model” (mental process).
The limitation “verifying validity of a trained artificial intelligence model”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “verifying”, in the limitation citied above encompasses evaluating an AI model and to judge if it is valid, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
The limitation “comparing the training dataset generated using the reverse data generation with a test dataset used to evaluate the trained artificial intelligence model, the comparing to determine a relationship between the training dataset that was generated and the test dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “comparing”, “using the reverse data generation” in the limitation citied above encompasses manipulating and comparing data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
The limitation “removing data from the test dataset determined to have a predefined relationship with the training dataset to obtain a new test dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “removing data”, “determined” in the limitation citied above encompasses checking to determine data removal, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
The limitation “using the new test dataset to verify the validity of the trained artificial intelligence model”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “using … to verify”, in the limitation citied above encompasses checking model with data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
2A – Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of generic computer elements (like computer, a computing device in communication to a memory, computer executing instruction from computer readable medium), which is mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). There is no additional elements showing integration of the abstract idea into a practical application and/or providing anything significantly more to the abstract idea. The claim is directed to an abstract idea.
2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the additional element of generic computer element merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). The claim is not patent eligible.
(Dependent claims)
Claims 2-10 are dependent on claim 1 and include all the limitations of claim 1. Therefore, claims 2-10 recite the same abstract ideas.
With regards to claim 2, the claim recites further limitation of “wherein the generating the training dataset includes constructing a simulation dataset of the trained artificial intelligence model, the simulation dataset being a simulation of a distribution of data represented by the trained artificial intelligence model, and using the simulation dataset to generate the training dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “generating”, “constructing”, “using … to generate” in the limitation citied above encompasses using models to create data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 3, the claim recites further limitation of “wherein the constructing the simulation dataset includes randomly generating vector data to conform to a selected distribution model”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “generating vector data” in the limitation citied above encompasses creating data based on certain process, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 4, the claim recites further limitation of “wherein the generating the passing the simulation dataset through the trained artificial intelligence model to obtain confidence values for simulation data of the simulation dataset; comparing the confidence values to a confidence comparator value; and forming a retained dataset that includes the simulation data that have confidence values with a predetermined relationship with the confidence comparator value, the retained dataset to be used to generate the training dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “to obtain”, “comparing”, “forming … dataset”, “to generate” in the limitation citied above encompasses creating data via certain evaluation process, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 5, the claim recites further limitation of “wherein the generating the training dataset further includes filtering the retained dataset to obtain the training dataset, the filtering including removing redundancy from the retained dataset to obtain the training dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “filtering”, “removing redundancy” in the limitation citied above encompasses evaluating and removing data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 6, the claim recites further limitation of “wherein the filtering includes performing density clustering to partition the retained dataset and remove the redundancy”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “performing” in the limitation citied above encompasses data manipulation, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 7, the claim recites further limitation of “wherein the comparing is based on data distributions of the training dataset and the test dataset, and wherein test dataset data that overlaps training dataset data are removed from the test dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “comparing”, “removed” in the limitation citied above encompasses evaluating & manipulating data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 8, the claim recites further limitation of “wherein the comparing the training dataset and the test dataset includes performing anomaly detection on a mix of the training dataset and the test dataset to obtain the new test dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “performing”, “to obtain” in the limitation citied above encompasses evaluation and combining data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 9, the claim recites further limitation of “wherein the performing the anomaly detection includes using an estimation network of a selected anomaly detection technique to detect a degree of integration in the training dataset and the test dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “using … to detect” in the limitation citied above encompasses using model to evaluate data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 10, the claim recites limitation of “wherein the generating the training dataset is performed absent availability of a dataset used to train the trained artificial intelligence model”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting generic computer elements, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the language about generic computer elements, “generating” in the limitation citied above encompasses following certain process to generate training data, which is based on observation, evaluation, judgement, and/or opinion, that could be performed by human using paper / pen / calculator. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas.
Claims 12-15, 17-20 are substantially similar to claims 2-11 and 16. The arguments as given above for claims 2-11 and 16, are applied, mutatis mutandis, to claims 12-15, 17-20, therefore the rejection of claims 2-11 and 16 are applied accordingly.
Allowable Subject Matter Analysis
Claims 1-20 include allowable subject matter since when reading the claims in light of the specification, as per, MPEP §2111.01 or Toro Co. v. White Consolidated Industries Inc., 199F.3d 1295, 1301, 53 USPQ2d 1065, 1069, 1069 (Fed.Cir. 1999), none of the references of record alone or in combination disclose or suggest the combination of limitations specified in claims 1-20.
In interpreting the claims, in light of the specification filed on 27 March 2023, the Examiner finds the claimed invention to be patentably distinct from the prior arts of record.
Regarding independent claim 1, the primary reason for the allowance is the inclusion of the following specific elements, in combination with the other elements cited, which is not found in the prior art of record: “… generating a training dataset from the trained artificial intelligence model using reverse data generation of the trained artificial intelligence model; comparing the training dataset generated using the reverse data generation with a test dataset used to evaluate the trained artificial intelligence model, the comparing to determine a relationship between the training dataset that was generated and the test dataset; removing data from the test dataset determined to have a predefined relationship with the training dataset to obtain a new test dataset; and using the new test dataset to verify the validity of the trained artificial intelligence model” in combination with other limitations in the claim.
Claims 11, 16 are substantially similar to claim 1. The arguments as given above for claim 1, are applied, mutatis mutandis, to claims 11, 16, therefore the allowance reasoning of claim 1 are applied accordingly.
None of the cited prior art references, singly or in combination, fully teaches all limitations of independent claims 1, 11 and 16.
Regarding the dependent claims, which include all the limitations of the independent claims, are also allowed.
The followings are references close to the invention claimed:
Ding et al., US-PGPUB NO.20150039591A1 [hereafter Ding] teaches reverse data generation as shown in Fig.5 & [0060]. However Ding does not teach the claimed specific process of using reverse data generation of trained ML model to create & modify test data set to validate ML model.
Hong et al., US-PGPUB NO.20220188703A1 [hereafter Hong] teaches data generation for learning models. However Hong does not teach the claimed specific process of using reverse data generation of trained ML model to create & modify test data set to validate ML model.
Liu, et al., US-PGPUB NO.20230222360A1 [hereafter Liu] shows training data evaluation for ML models. However Liu does not teach the claimed specific process of using reverse data generation of trained ML model to create & modify test data set to validate ML model.
Dahmen et al., “Generation of Virtual test scenarios for training and validation of AI-based systems”, 2021 International conference on progress in informatics and computing (PIC), 2021 [hereafter Dahmen] shows generating test data for model validation. However Dahmen does not teach the claimed specific process of using reverse data generation of trained ML model to create & modify test data set to validate ML model.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSU-CHANG LEE whose telephone number is 571-272-3567. The fax number is 571-273-3567.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas, can be reached 571-272-2589.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSU-CHANG LEE/
Primary Examiner, Art Unit 2128