DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
According to the first part of the analysis, in the instant case, claims 1-16 are directed to a method, claim 17 is directed to using a computer program stored in a non-transitory computer readable medium to perform the method, and claims 18-20 are directed to using a computer system to perform the method. Thus, each of the claims falls within one of the four statutory categories (i.e. process, machine, manufacture, or composition of matter).
Regarding claim 1:
A method for providing a benchmark result, performed by a computing device, comprising:
obtaining input data comprising information related to a target model which is an artificial intelligence-based model to be benchmarked;
providing a candidate node list comprising a plurality of candidate nodes, wherein each of the plurality of candidate nodes corresponds to hardware on which an artificial intelligence-based model can be executed;
determining a target model on the candidate node list, which is a target of the benchmark result; and
providing the benchmark result comprising estimated performance of the target model on the target node or an execution result of the target model on the target node.
Step 2A Prong 1:
“obtaining input data comprising information related to a target model which is an artificial intelligence-based model to be benchmarked” is directed to mental step of data gathering.
“providing a candidate node list comprising a plurality of candidate nodes, wherein each of the plurality of candidate nodes corresponds to hardware on which an artificial intelligence-based model can be executed” is directed to mental step of data analysis. Specifically, evaluating which hardware (nodes) can support an AI model requires analyzing system data such as processing capabilities, compute throughput, memory capacity, and power efficiency.
“determining a target model on the candidate node list, which is a target of the benchmark result” is directed to math because evaluating and selecting a model based on benchmark results involves math. Selecting the best model from a candidate list requires statistical math to aggregate score, ensure reproducibility, and calculate accuracy or error margins. “Benchmarks” themselves are frequently standardized math datasets designed specifically to test an AI’s ability to solve complex algebraic and calculus.
“providing the benchmark result comprising estimated performance of the target model on the target node or an execution result of the target model on the target node” is directed to a mental step of display result.
The claim recites the step of "determining a target model on the candidate node list, which is a target of the benchmark result” which as drafted, under BRI recites a mathematical calculation. The grouping of "mathematical concepts” in the 2019 PED includes "mathematical calculations" as an exemplar of an abstract idea. 2019 PEG Section |, 84 Fed. Reg. at 52. Thus, the recited limitation falls into the "mathematical concept" grouping of abstract ideas. This limitation also falls into the “mental process” group of abstract ideas, because the recited mathematical calculation is simple enough that it can be practically performed in the human mind, e.g., scientists and engineers have been solving the Arrhenius equation in their minds since it was first proposed in 1889.
Note that even if most humans would use a physical aid (e.g., pen and paper, a slide rule, or a calculator) to help them complete the recited calculation, the use of such physical aid does not negate the mental nature of this limitation. See October Update at Section I(C)(i) and (iii).
Additional Elements:
Step 2A Prong 2:
“A method for providing a benchmark result, performed by a computing device” recited in the preamble does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining input data comprising information related to a target model which is an artificial intelligence-based model to be benchmarked” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“providing a candidate node list comprising a plurality of candidate nodes, wherein each of the plurality of candidate nodes corresponds to hardware on which an artificial intelligence-based model can be executed” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“determining a target model on the candidate node list, which is a target of the benchmark result” does not integrate the judicial exception into a practical application. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“providing the benchmark result comprising estimated performance of the target model on the target node or an execution result of the target model on the target node” is directed to insignificant activity and does not integrate the judicial exception into a practical application. See MPEP 2106.05(g).
The claim is merely selecting data, manipulating or analyzing the data using math and mental process, and displaying the results.
This is similar to electric power: MPEP 2106.05(h) vi. Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid, because limiting application of the abstract idea to power-grid monitoring is simply an attempt to limit the use of the abstract idea to a particular technological environment, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016).
Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, "claiming the improved speed or efficiency inherent with applying the abstract idea on a computer" does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
Claim 1 recites the additional element(s) of using generic AI/ML technology, i.e. an artificial intelligence-based model, to perform data evaluations or calculations, as identified under Prong 1 above. The claims do not recite any details regarding how the AI/ML algorithm or model functions or is trained. Instead, the claims are found to utilize the AI/ML algorithm as a tool that provides nothing more than mere instructions to implement the abstract idea on a general purpose computer. See MPEP 2106.05(f). Additionally, the use of the artificial intelligence-based model merely indicates a field of use or technological environment in which the judicial exception is performed. See MPEP 2106.05(h). Therefore, the use of the artificial intelligence-based model to perform steps that are otherwise abstract does not integrate the abstract idea into a practical application. See the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence; and Example 47, ineligible claim 2.
The claim as a whole does not meet any of the following criteria to integrate the judicial exception into a practical application:
An additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
Step 2B:
“A method for providing a benchmark result, performed by a computing device” recited in the preamble does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“obtaining input data comprising information related to a target model which is an artificial intelligence-based model to be benchmarked” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“providing a candidate node list comprising a plurality of candidate nodes, wherein each of the plurality of candidate nodes corresponds to hardware on which an artificial intelligence-based model can be executed” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“determining a target model on the candidate node list, which is a target of the benchmark result” does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
“providing the benchmark result comprising estimated performance of the target model on the target node or an execution result of the target model on the target node” is directed to insignificant activity and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(g) and 2106.05(d)(ii), third list, (iv).
The claim is therefore ineligible under 35 USC 101.
Claim 17 is similar to claim 1 but recites a computer program stored in a non-transitory computer readable medium, wherein the computer program allows a computing device to perform following operations to provide a benchmark result when executed by the computing device. These additional elements fail to integrate the abstract idea into a practical application. These limitations are recited at a high level of generality and do not add significantly more to the judicial exception. These elements are generic computing devices that perform generic functions. Using generic computer elements to perform an abstract idea does not integrate an abstract idea into a practical application. See 2019 Guidance, 84 Fed. Reg. at 55. Moreover, “the mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.” Alice, 573 U.S. at 223; see also FairWarninglP, LLCv. latric SysInc., 839 F.3d 1089, 1096 (Fed. Cir. 2016) (citation omitted) (“[T]he use of generic computer elements like a microprocessor or user interface do not alone transform an otherwise abstract idea into patent-eligible subject matter”).
On the record before us, we are not persuaded that the hardware of claim 17 integrates the abstract idea into a practical application. Nor are we persuaded that the additional elements are anything more than well-understood, routine, and conventional so as to impart subject matter eligibility to claim 17.
Claim 18 is directed to an abstract idea similar to claim 1. The additional elements (i.e., A computing device for providing a benchmark result, comprising: at least one processor; and a memory, wherein the at least one processor to perform the steps) are recited at a high level of generality, necessary, routine, or conventional to facilitate the application of the abstract idea. When considered separately and in combination, they do not add significantly more to the abstract idea. See Alice Corp. and 2014 Interim Guidance.
Regrading claim 2, “wherein a model type information corresponding to the target model is extracted from the input data, and wherein the plurality of candidate nodes is determined based on the model type information” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 3, “wherein the providing the candidate node list comprises, determining, based on the model type information, the plurality of candidate nodes which support an execution environment corresponding to the model type information within a group of nodes, and providing the candidate node list comprising the determined candidate nodes” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 4, “wherein the model type information comprises identification information for identifying a framework corresponding to the target model” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 5, “wherein the benchmark result comprises: time information comprising preprocessing time information required for preprocessing of inference of the target model at the target node, or inference time information required for inference of the target model at the target node; and memory size information comprising preprocessing memory usage information required for preprocessing of inference of the target model at the target node, or inference memory usage information required for inference of the target model at the target node” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 6, “wherein the information related to the target model comprises at least one of dataset, a model file, and a link for the model file” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 7, “wherein the candidate node list comprises, identification information of each of candidate nodes capable of supporting a framework corresponding to the input data” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 8, “wherein the candidate node list comprises, at least one of quantified estimated performance information related to time corresponding to each of the plurality of candidate nodes, or quantified estimated performance information related to a memory corresponding to each of the plurality of candidate nodes” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 9, “wherein the plurality of candidate nodes of the candidate node list is determined based on at least one of a framework extracted from the input data, or size information of an artificial intelligence-based model, and the plurality of candidate nodes on the candidate node list are arranged based on the quantified estimated performance information” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 10, “wherein the benchmark result comprises: memory footprint information required for executing the target model on the target node; latency information required for executing the target model on the target node; power usage information required for executing the target model on the target node; and information regarding the target node” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 11, “wherein the information regarding the target node comprises, an execution environment of the target node, a processor of the target node and a random access memory (RAM) size of the target node” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 12, “wherein the benchmark result comprises: estimated GPU usage information when executing the target model on the target node; estimated CPU usage information when executing the target model on the target node; estimated latency information when executing the target model on the target node; and estimated power usage information when executing the target model on the target node” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 13, “generating download data corresponding to the target model, based on the benchmark result, to deploy the target model at the target node” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 14, “wherein a quantization interval is determined based on the benchmark result, and the download data in which a parameter value of the target model is changed is generated based on the determined quantization interval” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 15, “wherein the obtaining the input data comprises, obtaining the input data comprising information related to the target model on a candidate model list comprising a framework corresponding to each of a plurality of candidate models and software version information corresponding to each of a plurality of candidate models” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 16, “determining whether to convert the target model based on whether the target model is supported by the determined target node, or whether an operator included in the target model is supported by the determined target node, and wherein the benchmark result comprises, estimated performance of a converted target model on the target node or an execution result of the converted target model on the target node” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 19, “wherein a model type information corresponding to the target model is extracted from the input data, and wherein the plurality of candidate nodes is determined based on the model type information” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Regrading claim 20, “estimated GPU usage information when executing the target model on the target node; estimated CPU usage information when executing the target model on the target node; estimated latency information when executing the target model on the target node; and estimated power usage information when executing the target model on the target node” does not integrate the judicial exception into a practical application. It does not amount to significantly more than the judicial exception in the claim. This additional element is merely using a computer as a tool to perform an abstract idea (see MPEP 2106.05(h)).
Hence the claims 1-20 are treated as ineligible subject matter under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 6, 7, 15, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yoon et al. (KR 2020-0052417) in view of Kwon et al. (KR 10-2461997).
Regarding claims 1, 15, and 20, Yoon et al. disclose a method for providing a benchmark result, performed by a computing device, comprising:
obtaining input data comprising information related to a target model which is an artificial intelligence-based model to be benchmarked (see paragraphs [0070]-[0072] and Figs. 3 and 4: The module inference selection device 100 of the target device loads the artificial neural network model, and computes a computational amount of the corresponding artificial neural network model); and
providing the benchmark result comprising estimated performance of the target model on the target node or an execution result of the target model on the target node (see paragraph [0083] and Fig.4: After selecting the inference module, the inference module selection device 100 of the target device performs inference by using selected inference module).
Yoon et al. fail to disclose providing a candidate node list comprising a plurality of candidate nodes, wherein each of the plurality of candidate nodes corresponds to hardware on which an artificial intelligence-based model can be executed; determining a target model on the candidate node list, which is a target of the benchmark result.
Kwon et al. teach providing a candidate node list comprising a plurality of candidate nodes, wherein each of the plurality of candidate nodes corresponds to hardware on which an artificial intelligence-based model can be executed; determining a target model on the candidate node list, which is a target of the benchmark result (see paragraphs [0099] - [0100] and Figs. 4 to 6: target model is determined from a set of neural networks that includes at least one candidate neural network based on profile information of the reference model, network space information of the reference model and/or compression ratio information, and also see Figs.8 and 9, paragraphs [0115] and [0116]). Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Kwon et al. with the teaching of Yoon et al. in order to provide effective method of the neutral network model and reduction in weight system of the neutral network model.
Regarding claims 2 and 19, Koon et al. disclose wherein a model type information corresponding to the target model is extracted from the input data, and wherein the plurality of candidate nodes is determined based on the model type information (e.g. Fig.5: In step S1000 of acquiring data on the reference model for which learning is completed, the server 1000 may acquire the reference model for which learning is completed and data on the reference model. Specifically, the server 1000 receives arbitrary data for executing the reference model, including the network type and/or structural data of the reference model, parameter (or weight) data of nodes included in the reference model, for which training is completed. can be obtained).
Regarding claim 6, Koon et al. disclose wherein the information related to the target model comprises at least one of dataset, a model file, and a link for the model file (e.g. Fig. 6: In the step (S4000) of obtaining a target model according to an embodiment of the present application, the server 1000 selects at least one candidate neural network based on profile information of the reference model, network space information of the reference model, and/or compression rate information of the reference model. A target model may be determined from among the included neural network sets).
Regarding claim 7, Koon et al. disclose identification information of each of candidate nodes capable of supporting a framework corresponding to the input data (e.g. Figs. 4-6).
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN H LE whose telephone number is (571)272-2275. The examiner can normally be reached on Monday-Friday from 7:00am – 3:30pm Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shelby A. Turner can be reached on (571) 272-6334. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN H LE/Primary Examiner, Art Unit 2857
Prosecution Insights