DETAILED ACTION
Claims 1-21 are presented for examination.
This office action is in response to submission of application on 28-OCTOBER-2025.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12-SEPTEMBER-2022 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted on 04-OCTOBER-2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Amendment
The amendment filed 28-OCTOBER-2025 in response to the non-final office action mailed 11-JUNE-2025 has been entered. Claims 1-21 remain pending in the application.
With regards to the non-final office action’s rejection under 101, the amendments to the claims do not overcome the original rejection with regards to the claims being directed towards an abstract idea.
With regards to the non-final office action’s rejection under 103, the amendment to the claims have overcome the original rejection. However, upon a new search for the amended limitations, a new 103 rejection over Park in view of McFall, further in view of new art Ramamurti has been written. In light of the new rejection, the arguments are moot.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation is:
The input device, configured to receive user input specifying a scenario of claim 13.
Because this claim limitation is being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this limitation interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation recites sufficient structure to perform the claimed function so as to avoid it being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 rejected under 35 U.S.C. 101 because the claimed invention is direction to an abstract idea without significantly more.
MPEP 2106.04(a)(2)(Ill) “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, Judgments, and opinions.
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide run) to perform the claim limitation.
MPEP 2106.04(a)(2)(I) “The mathematical concepts grouping is defined as mathematical
relationships, mathematical formulas or equations, and mathematical calculations.”
Regarding claim 1:
Step 2A, Prong 1 will now be evaluated for this claim:
A judicial exception is recited in this claim as it recites a mental process:
computer-implemented method for generating datasets for scenario- based training and testing of a machine learning system comprising a plurality of machine learning models, comprising: automatically generating a plurality of initial datasets based on at least one of production data and synthetic data
Generating datasets describes the gathering and formatting of data for a particular purpose, wherein formatting data would be a mental process.
automatically generating a scenario library based on the extracted data
This is further arranging of data.
Step 2A, Prong 2 will now be evaluated for this claim:
Furthermore, the additional elements:
testing at least one model of the machine learning system using at least one scenario based on data in the scenario library
are interpreted as a general purpose computer under MPEP 2106.05(f)
MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering and post-solution activity to be insignificant extra-solution activity.
The following steps are mere data gathering:
receiving user input specifying a scenario;
Receiving input is a form of data gathering.
automatically extracting data relevant for the user-specified scenario from the datasets
This limitation describes the gathering of data for a specific purpose.
The following steps are merely post solution activity:
storing the generated scenario library for use in testing the machine learning system;
Storing of the machine learning model for later, disconnected use would be post-solution activity.
The additional elements have been considered both individually and as an ordered combination in order to determine whether they integrate the exception into a practical application. Therefore, no meaningful limits are imposed practicing the abstract idea.
Therefore, the claim is related to an abstract idea.
Step 2B will now be discussed with regards to this claim:
The claim does not provide an inventive concept. There is no additional Insignificant Extra- Solution Activity, as identified in Step 2A Prong Two, that provides an inventive concept.
Adding insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea such as a step of obtaining information about credit card transactions so that the information can be analyzed by an abstract mental process, as discussed in CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011) (see MPEP § 2106.05(g)) does not overcome a rejection.
The additional elements have been considered both individually and as an ordered combination as to whether they whether they warrant significantly more consideration.
The claim is ineligible.
Regarding claim 2, which depends upon claim 1:
The following would be considered a mental process:
categorizing the collected data
Categorization of data describes the evaluation of which particular category data best falls into.
The following would be data gathering:
collecting data describing user activities
The claim is ineligible.
Regarding claim 3, which depends upon claim 2:
The following would be considered a mental process:
Anonymizing the collected data
This further describes a method of formatting and augmenting the data that could be performed by a human with pen and paper by removing certain values from the data.
The claim is ineligible.
Regarding claim 4, which depends upon claim 1:
The following would be considered a mental process:
parsing the received user input specifying a scenario to generate a dataset to be recorded
Parsing user input describes a particular manner of formatting user input, which is data, and hence can be performed by the human mind with the aid of pen and paper.
The claim is ineligible.
Regarding claim 5, which depends upon claim 4:
The following would be post-solution activity:
automatically generating a scenario library based on the extracted data comprises recording at least one scenario based on the parsed user input.
Recording a scenario would be the storage of the scenario, which is post-solution activity.
The claim is ineligible.
Regarding claim 6 which depends upon claim 5:
This claim further limits the scenario recording of claim 5. Further specifying the scenario recording does not overcome the parent claim’s rejection.
This claim is rejected for incorporating the parent claim in full.
This claim is ineligible.
Regarding claim 19 which depends upon claim 1:
The following would be a mental process:
wherein testing the at least one model of the machine learning system comprises comparing output of a first model of the machine learning system with output of a second model of the machine learning system
Comparison of two outputs for differences is accomplishable in the human mind given many outputs of a machine learning model.
This claim is rejected for incorporating the parent claim in full.
This claim is ineligible.
Claims 7-12 and 20 recite a non-transitory computer readable storage medium that parallels the method of claims 1-6 and 19 respectively. Therefore, the analysis discussed above with respect to claims 1-6 and 19 also applies to claims 7-12 and 20 respectively. Accordingly, claims 7-12 and 20 are rejected based on substantially the same rationale as set forth above with respect to claims 1-6 and 19 respectively.
Claims 13-18 and 21 recite a system that parallels the method of claims 1-6 and 19 respectively. Therefore, the analysis discussed above with respect to claims 1-6 and 19 also applies to claims 13-18 and 21 respectively. Accordingly, claims 13-18 and 21 are rejected based on substantially the same rationale as set forth above with respect to claims 1-6 and 19 respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-21 are rejected under 35 U.S.C. 103 as being unpatentable over Park et al. (Pub. No. US 20210406473 A1, filed June 25th 2021, hereinafter Park) in view of McFall et al. (Pub. No. WO 2017187207 A1, filed May 2nd 2017, hereinafter McFall) further in view of Ramamurti et al. (Pub. No. US 20210383276 A1, filed June 4th 2021, hereinafter Ramamurti).
Regarding claim 1:
Claim 1 recites:
A computer-implemented method for generating datasets for scenario- based training and testing of a machine learning system comprising a plurality of machine learning models, comprising: automatically generating a plurality of initial datasets based on at least one of production data and synthetic data; receiving user input specifying a scenario; automatically extracting data relevant for the user-specified scenario from the datasets; automatically generating a scenario library based on the extracted data; storing the generated scenario library for use in testing the machine learning system; and testing at least one model of the machine learning system using at least one scenario based on data in the scenario library.
Park discloses receiving user input specifying a scenario; automatically extracting data relevant for the user-specified scenario from the datasets:
Park teaches receiving an utterance by a user recognizing intention and conversation flow within it, which would be the specified scenario, as it results directly from the user input, and extracting a knowledge base element from it in order to analyze an existing scenario in a database (Paragraph 35), wherein the existing scenario would be relevant data from the datasets.
However, while the scenarios of Park are specified by user input, the user does not have direct control for this specification. This aspect is taught further below by Ramamurti.
Park discloses automatically generating a scenario library based on the extracted data; and storing the generated scenario library for use in testing the machine learning system:
Park teaches that the database of existing scenarios can additionally have a customized scenario made by a service provider (Paragraph 9), wherein the database would be the scenario library, and the storage of the database would be for use in machine learning including testing, for example, as in the use of named entity recognition (Paragraph 9).
McFall discloses generating datasets for scenario- based training and testing of a machine learning system [comprising a plurality of machine learning models], comprising: automatically generating a plurality of initial datasets based on at least one of production data and synthetic data:
McFall in the same field of endeavor of digital data processing teaches that from an initial dataset of real-world information including identifying information, wherein real-world data would be a form of production data, a new dataset can be generated through the anonymization of the first database (Page 5, lines 7-9). This would be the generation of a database based on production data.
Furthermore, databases are used in a machine learning context in Park (Paragraph 9), which would include for training and testing.
Park, McFall, and the present application are analogous art because they are all in the same field of endeavor of digital data processing.
However, McFall does not teach multiple machine learning models.
Ramamurti in the same field of endeavor of machine learning discloses a plurality of machine learning models:
Ramamurti teaches at least two machine learning models within a machine learning system (Paragraph 4), which would be a plurality.
Ramamurti and the present application are analogous art because they are in the same field of endeavor.
Ramamurti discloses user-specified scenario:
Ramamurti teaches that users may adjust or define policy definitions to tailor particular control actions to a given application (Paragraph 54) wherein the control actions are generated by a machine learning model (Paragraph 48). This would be analogous to a user-specified scenario as it demonstrates a user’s ability to influence the outputs of the model based on real application.
Ramamurti discloses testing at least one model of the machine learning system using at least one scenario based on data in the scenario library:
Ramamurti teaches testing a recommender (which are machine learning models (Paragraph 33)) based on real, user-generated feedback data (Paragraph 80, Paragraph 33). The user feedback data would be analogous to scenarios based on data in the scenario library as it is produced by a user for the purpose of directing the machine learning model.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Park and the teachings of McFall. This would have provided the advantage of performing analysis of the datasets without learning sensitive information about an individual (McFall, Page 2, lines 4-6), as well as the advantage of wider applicability of the trained models (Ramamurti, Paragraph 3).
Regarding claim 2, which depends upon claim 1:
Claim 2 recites:
The method of claim 1, wherein generating the plurality of initial datasets comprises: collecting data describing user activities; and categorizing the collected data.
Park in view of McFall further in view of Ramamurti disclose the method of claim 1 upon which claim 2 depends. Furthermore, Park discloses the limitation of claim 2:
Park teaches receiving utterance from users including the intention of the utterance (Paragraph 9) which would be data describing user activities, wherein the user activity in question is speaking or a conversation. Furthermore, the intention of the user would be an example of categorizing the collected data as an intention would be a kind of category.
Regarding claim 3, which depends upon claim 2:
Claim 3 recites:
The method of claim 2, further comprising anonymizing the collected data.
Park in view of McFall further in view of Ramamurti disclose the method of claim 2 upon which claim 3 depends. Furthermore, McFall discloses the limitation of claim 3:
McFall in the same field of endeavor of digital data processing teaches that from an initial dataset of real-world information including identifying information, a new dataset can be generated through the anonymization of the first database (Page 5, lines 7-9).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Park and the teachings of McFall. This would have provided the advantage of performing analysis of the datasets without learning sensitive information about an individual (McFall, Page 2, lines 4-6), as well as the advantage of wider applicability of the trained models (Ramamurti, Paragraph 3).
Regarding claim 4, which depends upon claim 1:
Claim 4 recites:
The method of claim 1, further comprising parsing the received user input specifying the scenario to generate a dataset to be recorded.
Park in view of McFall further in view of Ramamurti disclose the method of claim 1 upon which claim 4 depends. Furthermore, Park discloses the limitation of claim 4:
Park teaches a customized scenario made be a service provider to be stored in the scenario database (Paragraph 36). The service provider would be a user of the system as they input the customized scenario, which would be the specified scenario.
Regarding claim 5, which depends upon claim 4:
Claim 5 recites:
The method of claim 4, wherein automatically generating the scenario library based on the extracted data comprises recording at least one scenario based on the parsed user input.
Park in view of McFall further in view of Ramamurti disclose the method of claim 4 upon which claim 5 depends. Furthermore, regarding the limitation of claim 5:
Park teaches that the customized scenario is stored in a scenario database, wherein the scenario database may also comprises further scenarios (Paragraph 9). The scenario library, or the database, therefore comprises recording at least one scenario based on the parsed user input through the customized scenario.
Regarding claim 6, which depends upon claim 5:
Claim 6 recites:
The method of claim 5, wherein recording at least one scenario comprises recording at least one of the group consisting of: binary data; dataframes; and Parquet file
Park in view of McFall further in view of Ramamurti disclose the method of claim 5 upon which claim 6 depends. Furthermore, McFall discloses the limitation of claim 6:
McFall teaches datasets stored in a Parque file format (Page 45, line 30-31).
Park has previous taught the storage of scenarios as datasets.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Park and the teachings of McFall. This would have provided the advantage of performing analysis of the datasets without learning sensitive information about an individual (McFall, Page 2, lines 4-6), as well as the advantage of wider applicability of the trained models (Ramamurti, Paragraph 3).
Regarding claim 19, which depends upon claim 1:
Claim 19 recites:
The method of claim 1, wherein testing the at least one model of the machine learning system comprises comparing output of a first model of the machine learning system with output of a second model of the machine learning system
Park in view of McFall further in view of Ramamurti disclose the method of claim 1 upon which claim 19 depends. Furthermore, Ramamurti discloses the limitation of claim 19:
Ramamurti teaches comparing the performance of two models based on feedback data (Paragraph 4), wherein the performance is directly determined by the model’s output as compared to a ground truth (Paragraph 98) and therefore comparing the performance is analogous to comparing the output of the models. Furthermore, this process is used in testing the model as compared to alternate models (Paragraph 80).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Park and the teachings of McFall. This would have provided the advantage of performing analysis of the datasets without learning sensitive information about an individual (McFall, Page 2, lines 4-6), as well as the advantage of wider applicability of the trained models (Ramamurti, Paragraph 3).
Claims 7-12 and 20 recite a non-transitory computer readable storage medium that parallels the method of claims 1-6 and 19 respectively. Therefore, the analysis discussed above with respect to claims 1-6 and 19 also applies to claims 7-12 and 20 respectively. Accordingly, claims 7-12 and 20 and are rejected based on substantially the same rationale as set forth above with respect to claims 1-6 and 19 respectively.
Claims 13-18 and 21 recite a system that parallels the method of claims 1-6 and 19 respectively. Therefore, the analysis discussed above with respect to claims 1-6 and 19 also applies to claims 13-18 and 21 respectively. Accordingly, claims 13-18 and 21 are rejected based on substantially the same rationale as set forth above with respect to claims 1-6 and 19 respectively.
Response to Arguments
Applicant’s arguments filed 28-OCTOBER-2025 have been fully considered, but the examiner believes that not all are fully persuasive.
Regarding the applicant’s remarks on the non-final office action’s 101 rejection of the claims, the examiner respectfully disagrees that the amendments overcome the original rejects and requests applicant’s consideration of the following:
Regarding the applicant’s argument that the testing of at least one model step added to claims 1, 7, and 13 overcomes the 101 rejection by both a) being more than a mental process and b) improving the operation of the computing system, the examiner believes that this step is insignificant application as it details only generic testing with no further details of how the testing is performed to distinguish it from a generic computer function. Additionally, this step does not provide an improvement to the technology – rather, it could be seen as improving the scenario library as supported by paragraph 4, wherein improvement to a database (or library) is not sufficient to constitute an improvement to computer functionality (MPEP 2106.05(a) BSG Tech LLC v. Buyseasons, Inc., 899 F.3d 1281, 1287-88, 127 USPQ2d 1688, 1693-94 (Fed. Cir. 2018)). The same reasoning regarding the improvement to the technology applies to the receiving user input and automatically extracting data steps as well. For these reasons, the examiner believes that claim is directed to an abstract idea as in the previous rejection, and that the amendments, while introducing extra-solution activity, do not amount to significantly more.
Regarding the applicant’s remarks on the non-final office action’s 103 rejection of the claims, the applicant argues that McFall and Park do not teach the amended limitations of these claims. As such, the applicant argues that all claims dependent on the above would additionally not be obvious under 103. The examiner agrees that the prior art of the original office action does not teach the amended limitations. However, upon a new search of the prior art for the amended limitations, the examiner has written a new rejection under 103 to address these limitations and respectfully requests applicant’s consideration of the following:
The applicant argues with regards to claim 1 that Park in view of McFall does not teach the amended limitations, and furthermore is insufficient to address the user-specified scenarios required by claim 1. With regards to the scenarios, the examiner incorporates Ramamurti to address this interpretation of the scenarios and incorporates it with the previous teachings of Park.
Ramamurti in the same field of endeavor of machine learning discloses the amended limitation a plurality of machine learning models:
Ramamurti teaches at least two machine learning models within a machine learning system (Paragraph 4), which would be a plurality.
Ramamurti and the present application are analogous art because they are in the same field of endeavor.
Ramamurti discloses user-specified scenario:
Ramamurti teaches that users may adjust or define policy definitions to tailor particular control actions to a given application (Paragraph 54) wherein the control actions are generated by a machine learning model (Paragraph 48). This would be analogous to a user-specified scenario as it demonstrates a user’s ability to influence the outputs of the model based on real application.
Ramamurti discloses the amended limitation testing at least one model of the machine learning system using at least one scenario based on data in the scenario library:
Ramamurti teaches testing a recommender (which are machine learning models (Paragraph 33)) based on real, user-generated feedback data (Paragraph 80, Paragraph 33). The user feedback data would be analogous to scenarios based on data in the scenario library as it is produced by a user for the purpose of directing the machine learning model.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Park and the teachings of McFall. This would have provided the advantage of performing analysis of the datasets without learning sensitive information about an individual (McFall, Page 2, lines 4-6), as well as the advantage of wider applicability of the trained models (Ramamurti, Paragraph 3).
Furthermore, the examiner addresses newly added claims 19-21 through the teachings of Ramamurti, as described in the 103 section above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDRIA JOSEPHINE MILLER whose telephone number is (703)756-5684. The examiner can normally be reached Monday-Thursday: 7:30 - 5:00 pm, every other Friday 7:30 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.J.M./Examiner, Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142