DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
2. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
In view of the new 2019 Revised Patent Subject Matter Eligibility Guidance (Federal Register Vol. 84, No. 4, January 7, 2019), the Examiner has considered the claims and has determined that under step 1, claims 1-5 are to a machine, claims 1-10 are to a system, and claims 11-20 are to a process. Next under the new step 2A prong 1 analysis, the claims are considered to determine if they recite an abstract idea (judicial exception) under the following groupings: (a) mathematical concepts, (b) certain methods of organizing human activity, or (c) mental processes. The independent claims contain at least the following bolded limitations (see representative independent claims) that fall into the grouping of mental processes and/or mathematical concepts:
1. A system implemented by one or more computers to flexibly benchmark training programs, the one or more computers comprising:
a storage device; and
a processing device, communicatively connected to the storage device, to:
identify user interactions with one or more of the training programs during a period of time;
convert the user interactions into user activity data associated with user identifiers; anonymize the user activity data by removing the user identifiers;
aggregate the anonymized user activity data with respect to each of the one or more training programs;
determine a benchmark model based on a flexible benchmark schema;
calculate benchmarks for each of the one or more training programs based on the aggregated user activity data and the benchmark model; and
present the benchmarks in a graphical user interface (GUI) on a display.
11. A method implemented by one or more computers to flexibly benchmark training programs, the method comprising:
identifying user interactions with one or more of the training programs during a period of time;
converting the user interactions into user activity data associated with user identifiers;
anonymizing the user activity data by removing the user identifiers;
aggregating the anonymized user activity data with respect to each of the one or more training programs;
determining a benchmark model based on a flexible benchmark schema;
calculating benchmarks for each of the one or more training programs based on the aggregated user activity data and the benchmark model; and
displaying the benchmarks in a graphical user interface (GUI).
The limitations of "identifying user interactions with one or more of the training programs during a period of time" amounts to a mental process to observe and recognize an event, which could be performed visually by a person observing a user interacting with a training program and taking a mental or recorded note that a user interaction has occurred. The limitations of "converting the user interactions into user activity data associated with user identifiers" amounts to a mental process to organize data by associating a user activity data to a user identifier, and could be performed by an observer recording the names of each user and observations of how they interact with a training program. The limitations of "anonymizing the user activity data by removing the user identifiers" amounts to a mental process to exclude data, and could be performed by a person erasing or removing names from a recorded list. The limitations of "aggregating the anonymized user activity data with respect to each of the one or more training programs" amounts to a mental process to organize and sort the user activity data according to the one or more training programs, and could be as simple as tallying a count of user activity for each training program which could be performed mentally or on pen and paper. The limitations of "determining a benchmark model based on flexible benchmark schema" could amount to a mental process to form a judgment criteria for evaluating the data, or a mathematical concept if the creation of the benchmark model requires a mathematical equation for modeling. The limitations of "calculating benchmarks for each of the one or more training programs based on the aggregated user activity data and the benchmark model" could amount to a mental process to compare the aggregated user activity data according to the benchmark model criteria to determine a judgment result, or a mathematical concept if the analysis amounts to applying mathematical equations of the benchmark model to calculate benchmark values. It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula."(see MPEP 2106.04(a)(2) I.). Taken together, the bolded limitations in their simplest embodiment (given broadest reasonable interpretation) amount to a sequence of mental process steps that could equivalently be performed by a person recording user activity data and calculating statistics/benchmarks based on such data.
Next in step 2A prong 2, the independent claims are analyzed to determine whether there are additional elements or combination of elements that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception such that it is more than a drafting effort designed to monopolize the exception, in order to integrate the judicial exception into a practical application. These limitations have been identified and underlined above, and are not indicative of integration into a practical application because: (1) the recitations of "a system implemented by one or more computers to flexibly benchmark training programs, the one or more computers comprising: a storage device; and a processing device, communicatively connected to the storage device" and "one or more computers to flexibly benchmark training programs " amounts to mere instructions to implement an abstract idea on a computer or merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)); and (2) the recitations to "present the benchmarks in a graphical user interface (GUI) on a display" and "displaying the benchmarks in a graphical user interface (GUI)" amount to insignificant post-solution data outputting activity to the judicial exception (see MPEP 2106.05(g)).
Next in step 2B, the independent claims are considered to determine if they recite additional elements that amount to an inventive concept (“significantly more”) than the recited judicial exception.
These limitations have been identified and are also underlined above, and are not indicative of an inventive concept because: (1) the recitations of "a system implemented by one or more computers to flexibly benchmark training programs, the one or more computers comprising: a storage device; and a processing device, communicatively connected to the storage device" and "one or more computers to flexibly benchmark training programs " amounts to mere instructions to implement an abstract idea on a computer or merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f); also as recited in the MPEP, 2106.07(b), merely adding a generic computer, generic computer components, or a programmed computer to perform generic computer functions does not automatically overcome an eligibility rejection (see Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 134 S. Ct. 2347, 2359-60, 110 USPQ2d 1976, 1984 (2014). See also OIP Techs. v. Amazon.com, 788 F.3d 1359, 1364, 115 USPQ2d 1090, 1093-94)); and (2) the recitations to "present the benchmarks in a graphical user interface (GUI) on a display" and "displaying the benchmarks in a graphical user interface (GUI)" amount to insignificant post-solution data outputting activity to the judicial exception (see MPEP 2106.05(g)), as tangential outputting of informational-based data without affecting a physical change in the abilities/operations of the computer or technology.
Dependent claims 2-9 and 12-19 contain additional limitations that fall under the abstract idea grouping of a mental process and/or mathematical calculations to describe additional data-based details/definitions of the data analysis steps. Dependent claims 10 and 20 claim the application of generic machine learning (using a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)) to new data environments, without disclosing improvements to the machine learning models to be applied, and are patent ineligible under § 101 (see Recentive Analytics, Inc. v. Fox Corp., 134 F.4th 1205 (Fed. Cir. 2025)). In other words, just like "using a processor" is not enough to make a claim patent eligible, "using a machine learning (ML) model" or general "training…using training data" is not enough to make a claim patent eligible, no matter how many words go into describing the (generic) machine learning techniques in the claim.3. An invention is not rendered ineligible for patent simply because it involves an abstract concept. Applications of such concepts "to a new and useful end" remain eligible for patent protection (see Alice Corp., 134 S. Ct. at 2354 (quoting Benson, 409 U.S. at 67)). However, "a claim for a new abstract idea is still an abstract idea" (see Synopsys v. Mentor Graphics Corp. _F.3d_, 120 U.S.P.Q. 2d1473 (Fed. Cir. 2016)). There needs to be additional elements or combination of additional elements in the claim to apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception or render the claim as a whole to be significantly more than the exception itself in order to demonstrate “integration into a practical application” or an “inventive concept.” For instance, particular (non-generic) physical arrangements for actively obtaining the gathered data, or further physical applications using the calculated benchmarks to drive a transformation, change in physical operation, or repair/maintenance of a technology or technical process (beyond a mere informational-based display or further calculations) could provide integration into a practical application to demonstrate an improvement to the technology or technical field.
Allowable Subject Matter
4. Claims 1-20 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action.
5. The following is a statement of reasons for the indication of allowable subject matter:
In regards to claim 1, the closest prior art, Sharma et al. (Us Pat. Pub. 2019/0295105) at least teaches a system implemented by one or more computers to flexibly benchmark programs (Sharma paragraphs [0002] and [0004] teaches a system implemented by one or more processing devices (computers) to flexibly score (benchmark) software programs), the one or more computers comprising:
a storage device (Sharma paragraph [0004] teaches a non-transitory processor-readable storage medium); and
a processing device, communicatively connected to the storage device (Sharma paragraph [0004] teaches a processing device communicatively connected to the storage medium), to:
identify user interactions with one or more of the programs during a period of time (Sharma paragraph [0042] teaches identifying user interactions with the software programs, including but not limited to mouse cursor location/movements, scrolling position, scrolling speed, button clicks, etc., during a user session period of time);
convert the user interactions into user activity data associated with user identifiers (Sharma paragraphs [0042]-[0043] teaches converting the user interactions into metrics that correspond to a user's activity within a software program, where paragraph [0042] teaches associating user-related data such as user name is associated with the activity data).
6. However, claim 1 contains allowable subject matter because the closest prior art, Sharma et al. (US Pat. Pub. 2019/0295105) fails to anticipate or render obvious a system implemented by one or more computers to flexibly benchmark training programs, the one or more computers comprising: a processing device to: anonymize the user activity data by removing the user identifiers; aggregate the anonymized user activity data with respect to each of the one or more training programs; determine a benchmark model based on a flexible benchmark schema; and calculate benchmarks for each of the one or more training programs based on the aggregated user activity data and the benchmark model, in combination with the rest of the claim limitations as claimed and defined by the Applicant. There is no suggestion or motivation for removing user identifiers (i.e., names) in Sharma (US Pat. Pub. 2019/0295105), as doing so would be contrary to the teaching of using the names as demographic information for generating score categories, which could not be done if the name information is removed.
Similarly, claim 11 contains allowable subject matter because the closest prior art, Sharma et al. (US Pat. Pub. 2019/0295105) fails to anticipate or render obvious a method implemented by one or more computers to flexibly benchmark training programs, the method comprising: anonymizing the user activity data by removing the user identifiers; aggregating the anonymized user activity data with respect to each of the one or more training programs; determining a benchmark model based on a flexible benchmark schema; and calculating benchmarks for each of the one or more training programs based on the aggregated user activity data and the benchmark model, in combination with the rest of the claim limitations as claimed and defined by the Applicant.
7. Dependent claims 2-10 depend from claim 1 and contain allowable subject matter for at least the same reasons as given for claim 1. Dependent claims 12-20 depend from claim 11 and contain allowable subject matter for at least the same reasons as given for claim 11.
Pertinent Art
8. Applicants are directed to consider additional pertinent prior art included on the Notice of References Cited (PTOL 892) attached herewith. The Examiner has pointed out particular references contained in the prior art of record within the body of this action for the convenience of the Applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply. Applicant, in preparing the response, should consider fully the entire reference as potentially teaching all or part of the claimed invention, as well as the context of the of the passage as taught by the prior art or disclosed by the Examiner. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
B. Branson et al. (US Pat. Pub. 2009/0083717) discloses Benchmark Profiling for Distributed Systems.
C. Kratsch (US Pat. Pub. 2011/0202774) discloses System for Collection and Longitudinal Analysis of Anonymous Student Data.
D. Signer et al. (US Pat. Pub. 2015/0095083) discloses Method, Apparatus, and Computer Readable Media For Match Rating and Interview Scheduling.
E. Sawyer et al. (US Pat. Pub. 2022/0139252) discloses System and Method of Training a Student With a Simulator.
F. Baker et al. (US Pat. Pub. 2022/0230554) discloses Systems and Methods for Analyzing Learner's Roles and Performance and for Intelligently Adapting the Delivery of Education.
G. Waldron (US Pat. Pub. 2022/0276952) discloses Log-Based Automation Testing.
H. Ben-Elazar et al. (US Pat. Pub. 2023/0050034) discloses Automated Generation of Predictive Insights Classifying User Activity.
I. Bilsborough (US Pat. No. 8,924,375) discloses Item Attention Tracking System and Method.
Conclusion
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL D LEE whose telephone number is (571)270-1598. The examiner can normally be reached on M to F, 9:30 am to 6 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Arleen Vazquez can be reached at 571-272-2619. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL D LEE/Primary Examiner, Art Unit 2857 7/20/2026
Prosecution Insights