DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Step 1:
According to the first part of the analysis, in the instant case, claims 1-10 is directed to a computer-implemented method, claim 11-20 is directed to using a system comprising data processing hardware to perform the method. Thus, each of the claims falls within one of the four statutory categories (i.e. process, machine, manufacture, or composition of matter).
Regarding claim 1:
A computer-implemented method executed on data processing hardware of a user device that causes the data processing hardware to perform operations comprising:
obtaining a pre-trained machine learning model and a training embedding snapshot from a remote system;
obtaining one or more input data samples captured by the user device;
for each particular input data sample of the one or more input data samples:
processing, using an on-device machine learning model corresponding to the pre-trained machine learning model, the particular input data sample to generate a corresponding on-device embedding and one or more corresponding predicted outputs; and
generating, based on the training embedding snapshot and the corresponding on-device embedding, corresponding performance data;
aggregating the corresponding performance data for the one or more input data samples to determine one or more performance metrics for the on-device machine learning model; and
transmitting the one or more performance metrics to the remote system.
Step 2A Prong 1:
“obtaining a pre-trained machine learning model and a training embedding snapshot from a remote system” is directed to mental step of collecting data.
“obtaining one or more input data samples captured by the user device” is directed to human activity.
“for each particular input data sample of the one or more input data samples: processing, using an on-device machine learning model corresponding to the pre-trained machine learning model, the particular input data sample to generate a corresponding on-device embedding and one or more corresponding predicted outputs” is direct to mental step of analyzing data.
“aggregating the corresponding performance data for the one or more input data samples to determine one or more performance metrics for the on-device machine learning model” is directed to math because aggregation is a statistical process. Aggregation in statistics involves combining multiple data measurements into summary statistics. This involves mathematical operations such as calculating the mean, variance, standard deviation, and confidence intervals to summarize performance across numerous data samples. Performance metrics are mathematical constructs. The metrics used to evaluate a model are defined mathematically.
Each limitation recites in the claim is a process that, under BRI covers performance of the limitation in the mind but for the recitation of a generic “performance metrics” which is a mere indication of the field of use. Nothing in the claim elements precludes the steps from practically being performed in the mind. Thus, the claim recites a mental process and human activity.
Further, the claim recites the step of "aggregating the corresponding performance data for the one or more input data samples to determine one or more performance metrics for the on-device machine learning model” which as drafted, under BRI recites a mathematical calculation. The grouping of "mathematical concepts” in the 2019 PED includes "mathematical calculations" as an exemplar of an abstract idea. 2019 PEG Section |, 84 Fed. Reg. at 52. Thus, the recited limitation falls into the "mathematical concept" grouping of abstract ideas. This limitation also falls into the “mental process” group of abstract ideas, because the recited mathematical calculation is simple enough that it can be practically performed in the human mind, e.g., scientists and engineers have been solving the Arrhenius equation in their minds since it was first proposed in 1889.
Note that even if most humans would use a physical aid (e.g., pen and paper, a slide rule, or a calculator) to help them complete the recited calculation, the use of such physical aid does not negate the mental nature of this limitation. See October Update at Section I(C)(i) and (iii).
Additional Elements:
Step 2A Prong 2:
“A computer-implemented method executed on data processing hardware of a user device that causes the data processing hardware to perform operations” recited in the preamble does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining a pre-trained machine learning model and a training embedding snapshot from a remote system” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining one or more input data samples captured by the user device” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“for each particular input data sample of the one or more input data samples: processing, using an on-device machine learning model corresponding to the pre-trained machine learning model, the particular input data sample to generate a corresponding on-device embedding and one or more corresponding predicted outputs” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“generating, based on the training embedding snapshot and the corresponding on-device embedding, corresponding performance data” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“aggregating the corresponding performance data for the one or more input data samples to determine one or more performance metrics for the on-device machine learning model” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“transmitting the one or more performance metrics to the remote system” is directed to insignificant activity and does not integrate the judicial exception into a practical application. See MPEP 2106.05(g).
The claim is merely collecting data, manipulating or analyzing the data using math and mental process, and displaying the results.
This is similar to electric power: MPEP 2106.05(h) vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016).
Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, "claiming the improved speed or efficiency inherent with applying the abstract idea on a computer" does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
The claim as a whole does not meet any of the following criteria to integrate the judicial exception into a practical application:
An additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
Step 2B:
“A computer-implemented method executed on data processing hardware of a user device that causes the data processing hardware to perform operations” recited in the preamble does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining a pre-trained machine learning model and a training embedding snapshot from a remote system” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining one or more input data samples captured by the user device” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“for each particular input data sample of the one or more input data samples: processing, using an on-device machine learning model corresponding to the pre-trained machine learning model, the particular input data sample to generate a corresponding on-device embedding and one or more corresponding predicted outputs” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“generating, based on the training embedding snapshot and the corresponding on-device embedding, corresponding performance data” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“aggregating the corresponding performance data for the one or more input data samples to determine one or more performance metrics for the on-device machine learning model” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“transmitting the one or more performance metrics to the remote system” is directed to insignificant activity and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(g) and 2106.05(d)(ii), third list, (iv).
The claim is therefore ineligible under 35 USC 101.
Claim 11 is similar to claim 1 but recites a system comprising data processing hardware; and memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising the steps as in claim 1. These additional elements fail to integrate the abstract idea into a practical application. These limitations are recited at a high level of generality and do not add significantly more to the judicial exception. These elements are generic computing devices that perform generic functions. Using generic computer elements to perform an abstract idea does not integrate an abstract idea into a practical application. See 2019 Guidance, 84 Fed. Reg. at 55. Moreover, “the mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.” Alice, 573 U.S. at 223; see also FairWarninglP, LLCv. latric SysInc., 839 F.3d 1089, 1096 (Fed. Cir. 2016) (citation omitted) (“[T]he use of generic computer elements like a microprocessor or user interface do not alone transform an otherwise abstract idea into patent-eligible subject matter”). On the record before us, we are not persuaded that the hardware of claim 11 integrates the abstract idea into a practical application. Nor are we persuaded that the additional elements are anything more than well-understood, routine, and conventional so as to impart subject matter eligibility to claim 11.
Regrading claims 2 and 12, “wherein the one or more performance metrics are generated without exposing content of the one or more input data samples, the corresponding on-device embeddings, or the corresponding predicted outputs to the remote system” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claims 3 and 13, “wherein the one or more performance metrics represent a drift in one or more characteristics of the input data samples over time” is directed to math because it refers to a change in the statistical distribution of the data. This phenomenon, often called data drift or covariate shift, involves core mathematical and statistical concepts.
Regrading claims 4 and 14, “wherein the one or more performance metrics represent a drift in the on-device machine learning model over time” is directed to math because machine learning itself is built on the foundations of statistics and mathematical optimization. Model drift (or concept/data drift) occurs when the underlying statistical properties or distributions of the data change over time. The "performance metrics" used to identify drift (e.g., accuracy, precision, recall, F1-score, mean error) are all mathematical measures used to quantify a model's effectiveness.
Regrading claims 5 and 15, “wherein: the operations further comprise receiving a trigger from the remote system; and generating the corresponding performance data and transmitting the one or more performance metrics to the remote system are performed in response to receiving the trigger” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claims 6 and 16, “wherein the trigger comprises a Firebase Cloud Messaging push notification” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claims 7 and 17, “wherein the trigger comprises logic for generating the corresponding performance data” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claims 8 and 18, “wherein the trigger comprises at least one of an indication of a time period over which corresponding performance data is to be generated, an indication of how often corresponding performance data is to be generated, or an indication of how often the corresponding performance data is to be aggregated and reported” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claims 9 and 19, “wherein generating, using the training embedding snapshot and the corresponding on-device embedding, the corresponding performance data comprises determining one or more differences between the training embedding snapshot and the corresponding on-device embedding” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claims 10 and 20, “wherein the one or more differences comprise a cosine similarity or a Euclidean distance” is directed to math because Euclidean distance and cosine similarity are mathematical concepts used to measure the difference or similarity between vectors in a multi-dimensional space. Euclidean distance measures the straight-line distance between two points, while cosine similarity measures the cosine of the angle between two vectors, indicating their directional similarity rather than their magnitude.
Hence the claims 1-20 are treated as ineligible subject matter under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4 and 11-14 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Lange et al. (US 11,755,955).
Regarding claims 1 and 11, Lange et al. disclose system, and a computer-implemented method executed on data processing hardware of a user device that causes the data processing hardware to perform operations comprising: obtaining a pre-trained machine learning model and a training embedding snapshot from a remote system (abstract, claim 1, Col.8, line 51-Col.9, line 11: The system collects train data from the computer system that is under testing. The train data includes features that affect performance metrics, as defined by a selected benchmark. This train data is used in training machine learning (ML) models. The ML models create a train snapshot); obtaining one or more input data samples captured by the user device (Col.7, lines 7-14, Col.15, lines 63-65: GUI enter input); for each particular input data sample of the one or more input data samples: processing, using an on-device machine learning model corresponding to the pre-trained machine learning model, the particular input data sample to generate a corresponding on-device embedding and one or more corresponding predicted outputs (e.g. Col.4, lines 54-67); and generating, based on the training embedding snapshot and the corresponding on-device embedding, corresponding performance data (e.g. Col.5, lines 9-34: the system 100 can be configured to automatically execute any tuning recommendations 141 produced by the anomaly detection and tuning recommendation system 110, such that the level of automation in the testing process is maximized); aggregating the corresponding performance data for the one or more input data samples to determine one or more performance metrics for the on-device machine learning model (e.g. Col.6, lines 1-14: the system store the set of ML models (machine learning models, , Col.8, lines 19-28: ML models is best fit model for performance metric, Col.10, line 58-Col.11, line 13: snapshots can be analyzed in operation 335 in order to detect anomalies and classify them (if any are detected). In the embodiments, anomalies are classified in two classes); and transmitting the one or more performance metrics to the remote system (Figs.3A,3B, Col.10, line 58-Col.11, line 23: train data include performance metrics is transmitted to the remote system).
Regarding claims 2 and 12, Lange et al. disclose the one or more performance metrics are generated without exposing content of the one or more input data samples, the corresponding on-device embeddings, or the corresponding predicted outputs to the remote system (Fig. 1 shows all the data is processed at the server 110, only the recommendations appear to be transmitted to the “remote system” of the user 140 – this may read on “are generated without exposing”).
Regarding claims 3 and 13, Lange et al. disclose the one or more performance metrics represent a drift in one or more characteristics of the input data samples over time (BRI of “drift” appears to include change, e.g. see Fig. 3A “325” “Change in feature set detected”).
Regarding claims 4 and 14, Lange et al. disclose the one or more performance metrics represent a drift in the on-device machine learning model over time (e.g. see col. 9 lines 50-65 changes in the feature set/train data for the ML model).
Other Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Choi et al. (US 2020/0372407) disclose method involves performing a first set of training operations to access and prepare multiple training examples included in a set of training data by one or more computing devices. A second set of training operations to train a machine-learned model based at least in portion on the set of training data is performed by the one or more computing devices. The training example of the set of training data into the machine-learned model is input by the one or more computing devices. An output of the machine-learned model, at least one training output is received by the one or more computing devices.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN H LE whose telephone number is (571)272-2275. The examiner can normally be reached on Monday-Friday from 7:00am – 3:30pm Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shelby A. Turner can be reached on (571) 272-6334. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN H LE/Primary Examiner, Art Unit 2857