DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
According to the first part of the analysis, in the instant case, claims 1-13 are directed to a method, claims 14-19 are directed to using a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising steps, and claim 20 is directed to using a system to perform the method. Thus, each of the claims falls within one of the four statutory categories (i.e. process, machine, manufacture, or composition of matter).
Regarding claim 1:
A method comprising:
obtaining a first subset of test results, wherein the first subset of test results is based on a set of test cases as associated with a first version of a computing system;
obtaining a second subset of test results, wherein the second subset of test results is based on the set of test cases as associated with a second version of the computing system;
determining a metric based on the first subset of test results and the second subset of test results;
determining, based on a noise profile of the computing system, a precision associated with the metric; and
providing, based on the precision, an indication of whether the computing system exhibited a performance degradation between the first version and the second version thereof.
Step 2A Prong 1:
“obtaining a first subset of test results, wherein the first subset of test results is based on a set of test cases as associated with a first version of a computing system” is directed to mental step of data gathering.
“obtaining a second subset of test results, wherein the second subset of test results is based on the set of test cases as associated with a second version of the computing system” is directed to mental step of data gathering.
“determining a metric based on the first subset of test results and the second subset of test results” is directed to math, specifically relying on statistic and probability. By applying mathematical and statistical formulas, you can determine whether the differences between your test subsets.
“determining, based on a noise profile of the computing system, a precision associated with the metric” is directed to math because: Noise in computing systems is typically random. You use probability distributions and random variables to model how noise behaves and fluctuates over time. Calculating precision involves statistical measures such as variance, standard deviation, and signal-to-noise (SNR). When calculating a metric that depends on multiple noisy variables, you use the mathematical laws of error propagation to determine how those combined uncertainties affect the final precision. In digital system, noise limits the amount of valid data you can transmit or process. Mathematical concepts like entropy and channel capacity are used to determine the maximum precision achievable under specific noise conditions.
“providing, based on the precision, an indication of whether the computing system exhibited a performance degradation between the first version and the second version thereof” is directed to math because comparing performance between system versions is fundamentally a mathematical and statistical process.
The claim recites the step of " determining a metric based on the first subset of test results and the second subset of test results; determining, based on a noise profile of the computing system, a precision associated with the metric; and providing, based on the precision, an indication of whether the computing system exhibited a performance degradation between the first version and the second version thereof” which as drafted, under BRI recites a mathematical calculation. The grouping of "mathematical concepts” in the 2019 PED includes "mathematical calculations" as an exemplar of an abstract idea. 2019 PEG Section |, 84 Fed. Reg. at 52. Thus, the recited limitation falls into the "mathematical concept" grouping of abstract ideas. This limitation also falls into the “mental process” group of abstract ideas, because the recited mathematical calculation is simple enough that it can be practically performed in the human mind, e.g., scientists and engineers have been solving the Arrhenius equation in their minds since it was first proposed in 1889.
Note that even if most humans would use a physical aid (e.g., pen and paper, a slide rule, or a calculator) to help them complete the recited calculation, the use of such physical aid does not negate the mental nature of this limitation. See October Update at Section I(C)(i) and (iii).
Additional Elements:
Step 2A Prong 2:
“obtaining a first subset of test results, wherein the first subset of test results is based on a set of test cases as associated with a first version of a computing system” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining a second subset of test results, wherein the second subset of test results is based on the set of test cases as associated with a second version of the computing system” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“determining a metric based on the first subset of test results and the second subset of test results” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“determining, based on a noise profile of the computing system, a precision associated with the metric” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“determining, based on a noise profile of the computing system, a precision associated with the metric” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“providing, based on the precision, an indication of whether the computing system exhibited a performance degradation between the first version and the second version thereof” is directed to insignificant activity and does not integrate the judicial exception into a practical application. See MPEP 2106.05(g).
The claim is merely selecting data, manipulating or analyzing the data using math and mental process, and displaying the results.
This is similar to electric power: MPEP 2106.05(h) vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016).
Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, "claiming the improved speed or efficiency inherent with applying the abstract idea on a computer" does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
The claim as a whole does not meet any of the following criteria to integrate the judicial exception into a practical application:
An additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
Step 2B:
“obtaining a first subset of test results, wherein the first subset of test results is based on a set of test cases as associated with a first version of a computing system” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining a second subset of test results, wherein the second subset of test results is based on the set of test cases as associated with a second version of the computing system” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“determining a metric based on the first subset of test results and the second subset of test results” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“determining, based on a noise profile of the computing system, a precision associated with the metric” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“providing, based on the precision, an indication of whether the computing system exhibited a performance degradation between the first version and the second version thereof” is directed to insignificant activity and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(g) and 2106.05(d)(ii), third list, (iv).
The claim is therefore ineligible under 35 USC 101.
Claim 14 cites a non-transitory computer-readable storage medium having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising steps as in claim 1. This amounts to nothing more than instructions to implement the abstract idea on a computer, which fails to integrate the abstract idea into a practical application. See 2019 Guidance, 84 Fed. Reg. at 55. Additionally, using instructions to implement an abstract idea on a generic computer “is not ‘enough’ to transform an abstract idea into a patent-eligible invention.” Alice, 573 U.S. at 226. Therefore, the rejection of claim 14 for the same reason discussed above with regard to the rejection of claim 1.
Claim 20 recites a system comprising: one or more processors; and memory, containing program instructions that, upon execution by the one or more processors, cause the system to perform operations comprising steps as in claim 1. These additional elements fail to integrate the abstract idea into a practical application. These limitations are recited at a high level of generality and do not add significantly more to the judicial exception. These elements are generic computing devices that perform generic functions. Using generic computer elements to perform an abstract idea does not integrate an abstract idea into a practical application. See 2019 Guidance, 84 Fed. Reg. at 55. Moreover, “the mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.” Alice, 573 U.S. at 223; see also FairWarninglP, LLCv. latric SysInc., 839 F.3d 1089, 1096 (Fed. Cir. 2016) (citation omitted) (“[T]he use of generic computer elements like a microprocessor or user interface do not alone transform an otherwise abstract idea into patent-eligible subject matter”). On the record before us, we are not persuaded that the hardware of claim 20 integrates the abstract idea into a practical application. Nor are we persuaded that the additional elements are anything more than well-understood, routine, and conventional so as to impart subject matter eligibility to claim 20.
Regarding claim 2, “wherein the first subset of test results are from the set of test cases being performed against the first version of the computing system, and wherein the second subset of test results are from the set of test cases being performed against the second version of the computing system” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regarding claim 3, “wherein performing the set of test cases against the first version of the computing system comprises a testing system remotely accessing the first version of the computing system, and wherein performing the set of test cases against the second version of the computing system comprises the testing system remotely accessing the first version of the computing system” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regarding claim 4, “wherein the second version is different from the first version” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regarding claim 5, “wherein the metric is indicative of a difference between the first subset of test results and the second subset of test results” is directed to math because it describes a comparative analysis between two group of data, where a specific mathematical measurement (the metric) shows that the two group are not the same.
Regarding claim 6, “wherein the difference between the first subset of test results and the second subset of test results is based on a normalized difference between mean latency values of the first subset of test results and the second subset of test results” is directed to math because it describes a statistical comparison between two groups of data (test results) using specific mathematical concepts.
Regarding claims 7 and 15, “wherein the set of test cases assess functional correctness of the computing system, and wherein the first subset of test results and the second subset of test results assess performance of the computing system” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regarding claims 8 and 16, “wherein the noise profile was pre-calculated prior to obtaining the first subset of test results and the second subset of test results” is directed to math because the phrase describes a process of calibrating a mathematical baseline so that background noise can be separated from actual, meaningful result when evaluating test data.
Regarding claims 9 and 17, “wherein pre-calculating the noise profile comprises: performing a first set of pretests against the computing system; performing a second set of pretests against the computing system; determining a false alarm percentage and a false okay percentage for a delta threshold, wherein the delta threshold is based on a normalized difference between mean latency values of the first set of pretests and the second set of pretests; and determining the noise profile based on the false alarm percentage and the false okay percentage for one or more slowdown percentages” is directed to math because it relies on calculating probabilities and error rates to determine the most reliable threshold for detecting system anomalies.
Regarding claim 10, “wherein the one or more slowdown percentages are synthetic delays applied to the second set of pretests” is directed to math because it describes a quantitative algorithm where system latency is adjusted by a specific numerical value slowdown percentages refer to quantitative, numerical values used to calculate a specific duration. Synthetic delays refer to artificial time lags programmed to test system stability or how software responds to simulated lagging condition. Second set of pretests refers to the testing data or performance group used to evaluate this mathematical or system response.
Regarding claims 11 and 18, “wherein the false alarm percentage is based on a count of test runs in the second set of pretests that exceed corresponding test runs in the first set of pretests by more than the delta threshold” is directed to math because this description outlines a mathematical and statistical procedure.
Regarding claims 12 and 19, “wherein one of the slowdown percentages is applied to test runs in the second set of pretests, and wherein the false okay percentage is based on a count of the test runs in the second set of pretests that do not exceed corresponding test runs in the first set of pretests by more than the delta threshold” is directed to math because it describes a quantitative, algorithmic process for evaluating data, specifically detailing how to calculate a statistical metric.
Regarding claim 13, “wherein determining the precision associated with the metric comprises: determining, based on the metric, an applicable false alarm percentage and an applicable false okay percentage; and determining a confidence level relating to a difference between the first subset of test results and the second subset of test results” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Hence the claims 1-20 are treated as ineligible subject matter under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, 14-16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Arguelles et al. (US 9,558,465) in view of Lizon et al. (“Fundamentals of Precision ADC Noise Analysis”, Texas Instruments, September 2020, pages 1-65).
Regarding claims 1, 14, and 20, Arguelles et al. disclose system comprising: one or more processors; and memory, containing program instructions that, upon execution by the one or more processors, cause the system to perform operations (Col.14, lines 29-41) comprising: obtaining a first subset of test results, wherein the first subset of test results is based on a set of test cases as associated with a first version of a computing system; obtaining a second subset of test results, wherein the second subset of test results is based on the set of test cases as associated with a second version of the computing system; determining a metric based on the first subset of test results and the second subset of test results (see Fig.2, Col.22, lines 13-30: As noted above regarding FIG. 2, transaction generation framework 140 may be configured to compare testing and/or performance results for a service, such as network-based production service 100 that is running on multiple hosts or endpoints, such as test hosts 275A-275N. For instance, a service may be executing on multiple endpoints that have different hardware configurations and the service owners would like to compare how the service performs (e.g., in terms of throughput, latency, etc.) on the different hardware configurations. Additionally two different endpoints may be running the service, but with different settings (e.g., different cache sizes, caching algorithms, etc.) and it is desired to compare the performance of the service between different settings. Similarly, two different versions (e.g., a current and a new or proposed version) of the service may be executing on two different (possibly identical) endpoints and testers wish to validate the new version by comparing its performance to the current version).
Arguelles et al. fail to disclose determining, based on a noise profile of the computing system, a precision associated with the metric; and providing, based on the precision, an indication of whether the computing system exhibited a performance degradation between the first version and the second version thereof.
Lizon et al. teach determining, based on a noise profile of the computing system, a precision associated with the metric; and providing, based on the precision, an indication of whether the computing system exhibited a performance degradation between the first version and the second version thereof (page 34, section selecting an external amplifier: Figure 8 and Equation 6 show the modified version of the equivalent noise model, page 38: precision associated with noise performance, noise performance metric, page 60: a significant degradation in noise performance). It would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Lizon et al. with the teaching of Arguelles et al. in order to provide effective precision associated with noise performance and noise performance metric.
Regarding claim 2, Arguelles et al. disclose wherein the first subset of test results are from the set of test cases being performed against the first version of the computing system, and wherein the second subset of test results are from the set of test cases being performed against the second version of the computing system (Col.22, lines 13-30).
Regarding claim 3, Arguelles et al. disclose performing the set of test cases against the first version of the computing system comprises a testing system remotely accessing the first version of the computing system, and wherein performing the set of test cases against the second version of the computing system comprises the testing system remotely accessing the first version of the computing system (Col.26, lines 46-62).
Regarding claim 4, Arguelles et al. disclose the second version is different from the first version (Col.22, lines 26-30).
Regarding claim 5, Arguelles et al. disclose wherein the metric is indicative of a difference between the first subset of test results and the second subset of test results (Col.23, lines 8-35).
Regarding claim 6, Arguelles et al. disclose the difference between the first subset of test results and the second subset of test results is based on a normalized difference between mean latency values of the first subset of test results and the second subset of test results Col.23, lines 8-35).
Regarding claim 7, Arguelles et al. disclose the set of test cases assess functional correctness of the computing system, and wherein the first subset of test results and the second subset of test results assess performance of the computing system.
Regarding claims 8 and 16, Arguelles et al. disclose wherein the noise profile was pre-calculated prior to obtaining the first subset of test results and the second subset of test results (Col.23, lines 8-28).
Regarding claim 15, Arguelles et al. disclose wherein the set of test cases assess functional correctness of the computing system, and wherein the first subset of test results and the second subset of test results assess performance of the computing system (Col.22, lines 13-30).
Other Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Wan et al. (US 2023/0086361 A1) disclose a computer-implemented method can receive a software update for a software application deployable on a target system and perform an automatic performance test of the software application incorporating the software update on the target system. The automatic performance test can be integrated in a continuous integration and continuous delivery (CI/CD) pipeline. The method can collect a plurality of performance metrics measured when performing the automatic performance test and determine a performance score based on the plurality of performance metrics. Responsive to detecting the performance score is below a predefined performance threshold, the method can cause the CI/CD pipeline to reject the software update to the software application.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN H LE whose telephone number is (571)272-2275. The examiner can normally be reached on Monday-Friday from 7:00am – 3:30pm Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shelby A. Turner can be reached on (571) 272-6334. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN H LE/Primary Examiner, Art Unit 2857
Prosecution Insights