Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 1 recites “a method for providing a benchmark result…obtaining…providing…and wherein the first comparison option is an option…layers constituting the target model” which recites a series of steps. Therefore, it is a process.
Step 2A, Prong One: The claim recites an abstract idea as follows:
Under BRI, the steps “obtaining…, providing…, and wherein the first comparison option is an option…layers constituting the target model” which falls within both mental processes and mathematical calculation. The steps may be carried out as a mental process if the algorithm is simple enough, and as a mathematical process if the algorithm is more complicated.
The claimed invention thus recited as an abstract idea. Claim 1 recites mathematical concepts and/or mental processes, that may be carried out in human mind or with the aid of pencil and paper in simple situations. The claim does not recite a particular equation or algorithm for making the recited calculating step, this just means that the abstract idea is being recited broadly enough to monopolize all possible equations or algorithms that might be used (Please also see MPEP 2106.04(a)(2)(III)(A), (B), (C), and (D).
The claim recites “wherein the first comparison option is an option for visually comparing…target model” which is an insignificant post-solution activity, visually outputting/displaying the result of the abstract idea calculation (see MPEP. 2106.05(g), for instance). Outputting and displaying are at best the equivalent of merely adding the words “apply it” to the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
The recited method claimed steps which are recited at such a high level of generality that they cannot be considered to indicate a particular machine. The claim is therefore directed to the abstract idea.
The steps recited in claim 1 encompasses observation or evaluation (“obtaining…and providing…” fall within mental processes grouping of abstract ideas.
The recited limitation “comparing…” encompasses mathematical concepts.
At Step 2A, Prong 2, the abstract idea is not integrated into a practical application. There is no particular machine recited, and no real-world transformation takes place.
The recited steps in claim 1 are merely data gathering limitation and an insignificant extra-solution activity, namely mere data gathering recited at a high level of generality (see MPEP 2106.05(g), for instance).
The “(provided) benchmark result…” is insignificant extra-solution activity.
The claim does not recite applying the abstract idea with, or by use of, any particular machine, nor does the claim affect a real-world transformation or reduction of a particular article to a different state or thing. The claim amounts to obtaining benchmark object…and providing the benchmark result based on the configuration setting, the target model, and the target node”. Therefore, the claimed invention does not appear to be limited to the use of the mathematical technique for a particular practical application, but instead the claim appears to monopolize the mathematical technique itself, in any practical application where it might conceivably be used. The claim is therefore considered to be directed to an abstract idea.
The limitation “wherein the first comparison option is an option for visually comparing…” represents extra solution activity because it is a mere nomial or tangential addition to the claim. See MPEP 2106.05(I) for more information on this point, including explanations from judicial decisions including Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 224-26 (2014). This limitation represents extra-solution activity because it is a mere nominal or tangential addition to the claim. See MPEP 2106.05(g), discussing limitations that the Federal Circuit has considered to be insignificant extra-solution activity, for instance the step of printing a menu that was generated through an abstract process in Apple, Inc. v. Ameranth, Inc., 842 F.3d 1229, 1241-42 (Fed. Cir. 2016) and the mere generic presentation of collected and analyzed data in Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354 (Fed. Cir. 2016).
The term “visually” are generic which would not integrate the claim into a particular practical application. Further, outputting is at best the equivalent of merely adding the words “apply it” to the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
At Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception, for reasons that are analogous to the discussion of additional elements at Prong 2.
Claim 1 is therefore rejected as ineligible under 35 USC 101.
Dependent claims 2-13 add limitations which are merely data gathering extending the abstract idea without adding any additional elements. Claim 9 recites wherein a layer….are distinguishably displayed in the benchmark result is insignificant solution. Further, the limtiation represents extra solution activity because it is a mere nomial or tangential addition to the claim. See MPEP 2106.05(I) for more information on this point, including explanations from judicial decisions including Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 224-26 (2014). This limitation represents extra-solution activity because it is a mere nominal or tangential addition to the claim. See MPEP 2106.05(g), discussing limitations that the Federal Circuit has considered to be insignificant extra-solution activity, for instance the step of printing a menu that was generated through an abstract process in Apple, Inc. v. Ameranth, Inc., 842 F.3d 1229, 1241-42 (Fed. Cir. 2016) and the mere generic presentation of collected and analyzed data in Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354 (Fed. Cir. 2016).
Dependent claims 14 and 15 add limitations which can be considered as an additional element; however, the limitation is not performed by a particular device and does not integrate the judicial exception into a practical application.
Dependent claims 16-18 add limitations which are merely data gathering extending the abstract idea without adding any additional elements.
Claims 19 and 20 recite a non-transitory computer readable medium and a computing device which do not offer a meaningful limitation beyond generally linking the apparatus to a particular technological environment, that is, implementation via a processor. In other words, the medium and computing device claims are no different from the method claim 1 in substance; the method claim recites the abstract idea while the apparatus claim and medium claim recites generic components configured to implement the same abstract idea. The claims do not amount to significantly more than the underlying abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Enerzai Inc. (KR 20230022387) (English Abstract submitted by Applicants) (English translation by Examiner) (hereinafter “Enerzai”) and Myers et al. (USPAP. 20200099773)(hereinafter “Myers”).
Regarding claim 1 and similar claims 19 and 20, Enerzai discloses a method for providing a benchmark result, performed by a computing device (Pars. 88, 92, 96, according to the test generating method, apparatus, system, the trained AI model is tested for each computing environment. The structure of a deep learning model and the weights (or parameters) of the nodes of the deep learning model can be learned using a test bench generated), comprising:
obtaining a benchmark object comprising an artificial intelligence-based target model and a target node, and obtaining a benchmark configuration setting (Pars. 91, 101-103: The test bench may validate the temporary model based on performance information of the target processing unit where the final deep learning model is to be used. The test bench generation system 10 generates a test bench that includes performance information of the model, e.g. accuracy of tasks, latency, memory usage, power usage, and execution speed) for each computing environment. The structure of the deep learning model and the weight or parameter of the node of the deep learning model may be learned by using the test bench),
wherein the benchmark configuration setting comprises at least one of: a resource condition for the target node (computing environment) (Par. 104: actual memory usage, execution speed and power consumption varies depending on the actual computing environment), a first comparison option for benchmark results of each of a plurality of layers constituting the target model (Par. 91: the central server can be implemented to visualize performance information related to tasks other than latency such as accuracy, memory usage, power consumption, execution speed, and footprint), or a second comparison option for benchmark results of each of a plurality of nodes comprising the target node (Par. 91: the central server can be implemented to visualize performance information related to tasks other than latency such as accuracy, memory usage, power consumption, execution speed, and footprint. The central server (100) can perform visualization in any format by comparing the verification results of the second model according to the computing environment. and Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing), and wherein the benchmark results comprise performance information corresponding to each of the plurality of layers constituting the target model (Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing);
and providing the benchmark result based on the configuration setting, the target model, and the target node (Pars. 101, 102: at step S2400, in the obtaining of the final deep learning model, the central server obtains a final deep learning model that has a model size that is close to the learning goal and exhibits performance close to the targeted accuracy),
and wherein the first comparison option is an option for visually comparing benchmark results for each of the plurality of layers within the target model at the target node (Par. 91: the central server can be implemented to visualize performance information related to tasks other than latency such as accuracy, memory usage, power consumption, execution speed, and footprint. The central server (100) can perform visualization in any format by comparing the verification results of the second model according to the computing environment. and Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing), and wherein the benchmark results comprise performance information corresponding to each of the plurality of layers constituting the target model (Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing).
However, the recited reference discloses “obtaining a benchmark configuration setting” as explained above; but not the “indicating a customization of the benchmark result”.
Myers teaches “obtaining a benchmark configuration setting indicating a customization of the benchmark result” (Myers: Par. 260: the health scores provide accurate predictions for the performance of the object being migrated in the new environment. Benchmark results or scores for particular characteristics of the initial system are taken prior to the migration, and these benchmark values can be used to estimate performance of the system in the new environment. Par. 269: While the performance measures are generated in response to user configuration change requests, and are customized for the particular change requested (e.g., the specific servers, documents, users involved), the management server can use previously generated benchmark results and conversion measures to maintain appropriate responsiveness).
It would have been obvious to one of ordinary skilled in the art at the time of filling the Application to modify Enerzai's invention using Meyers' invention to arrive at the claimed invention specified in claim 1 to maintain appropriate responsiveness (Meyers: Par. 269).
Regarding claim 2, Enerzai and Meyers disclose everything as applied above. In addition, Enerzai discloses wherein the resource condition for the target node comprises: a condition related to a use of computing resources by applications other than an inference application of the target model on the target node, or a condition related to a use of computing resources by operations other than an inference operation of the target model on the target node, when inference of the target model is executed on the target node, and wherein the benchmark result comprises performance information obtained by executing the target model on the target node, assuming a computing resource situation corresponding to the resource condition (Enerzai: Par. 104: actual memory usage, execution speed and power consumption varies depending on the actual computing environment).
Regarding claim 3, Enerzai and Meyers disclose everything as applied above. In addition, Enerzai discloses wherein the resource condition for the target node comprises: a condition related to an occupancy of computing resources being used on the target node, when inference of the target model is to be executed on the target node (Par. 104: actual memory usage, execution speed and power consumption varies depending on the actual computing environment).
Regarding claim 4, Enerzai and Meyers disclose everything as applied above. In addition, Enerzai discloses wherein the resource condition for the target node identifies a computing resource to be included in the benchmark result, when inference of the target model is to be executed on the target node (Pars. 104-109).
Regarding claim 5, Enerzai and Meyers disclose everything as applied above. In addition, Enerzai discloses wherein the benchmark result comprises a benchmark estimation result predicted when the target model is executed in the target node (Par. 109).
Regarding claim 6, Enerzai and Meyers disclose everything as applied above. In addition, Enerzai discloses “wherein the second comparison option is an option for visually comparing benchmark results for the target model at each of the plurality of nodes comprising the target node, or an option for visually comparing benchmark results for each of the plurality of layers within the target model at each of the plurality of nodes comprising the target node” ( Par. 91: the central server can be implemented to visualize performance information related to tasks other than latency such as accuracy, memory usage, power consumption, execution speed, and footprint. The central server (100) can perform visualization in any format by comparing the verification results of the second model according to the computing environment. and Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing), and wherein the benchmark results comprise performance information corresponding to each of the plurality of layers constituting the target model (Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing).
Regarding claim 7, Enerzai and Meyers disclose everything as applied above. In addition, Enerzai discloses wherein the benchmark configuration setting further comprises a third comparison option for visually comparing benchmark results for each of a plurality of models comprising the target model at the target node (Par. 91: the central server can be implemented to visualize performance information related to tasks other than latency such as accuracy, memory usage, power consumption, execution speed, and footprint. The central server (100) can perform visualization in any format by comparing the verification results of the second model according to the computing environment. and Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing), and wherein the benchmark results comprise performance information corresponding to each of the plurality of layers constituting the target model (Par. 92: users can use test benches to easily verify, e.g. comparing or predict performance information of their developed models in different computing).
Conclusion
Claims 8-18 are patentably distinguishable over the prior art of record.
Choi et al. (USPAP. 20220335293)(submitted by Applicant) discloses a method of optimizing a neural network model includes receiving original model information about a first neural network model that is pre-trained; generating a second neural network model and compressed model information about the second neural network model by performing a compression on the first neural network model; and outputting, on a screen, at least a part of the original model information (Pars. 58-65). However, Choi does not explicitly disclose the claims 8-18.
Regarding claim 8, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the benchmark result comprises the number of calls to each of the plurality of layers, and latencies for each of the plurality of layers.” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 9, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein a layer that is able to be optimized and a layer that is not able to be optimized, for each of a plurality of layers constituting an artificial intelligence-based model, are distinguishably displayed in the benchmark result” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 10, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the benchmark result comprises: preprocessing time information required for preprocessing of inference of the target model at the target node, or inference time information required for inference of the target model at the target node; and preprocessing memory usage information used for preprocessing of inference of the target model at the target node, or inference memory usage information required for inference of the target model at the target node” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 11, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the benchmark result comprises: memory footprint information required for executing the target model at the target node; latency information required for executing the target model at the target node; and power consumption information required for executing the target model at the target node” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 12, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the benchmark result comprises at least one of: a first result comparatively indicating maximum inference latencies obtained by executing the target model assuming the slowest computing resource situation at each of the plurality of nodes comprising the target node; a second result comparatively indicating an average inference latency when the target model is executed multiple times at each of the plurality of nodes comprising the target node; or a third result comparatively indicating inference latencies for each of a plurality of layers within the target model at each of the plurality of nodes comprising the target node” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 13, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the benchmark result comprises: a fourth result comparatively indicating a processor margin value in which another operation or another application is executable in a process of inferring the target model at the target node; and a fifth result comparatively indicating a memory margin value in which another operation or another application is executable in a process of inferring the target model at the target node.” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 14, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the providing the benchmark result comprises: determining a visual element to be included in the benchmark result based on the benchmark configuration setting; obtaining performance information to be included in the benchmark result based on the target model and the target node; and providing the benchmark result in which the performance information is indicated depending on the visual element” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 15, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the visual element comprises: information identifying each of a plurality of axes to be included in the benchmark result; and information identifying a graph shape to be included in the benchmark result” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 16, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the providing the benchmark result comprises: providing, to a first module, the benchmark result comprising performance information corresponding to an input layer of the plurality of layers so that the first module which trains the target model can determine a size of input data of the target model; or providing, to a second module, the benchmark result comprising performance information for each of the plurality of layers so that the second module which generates a compressed target model by compressing the target model can determine whether to compress each of the plurality of layers of the target model” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 17, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “obtaining importance level information corresponding to visual elements to be included in the benchmark result; and providing a candidate node list comprising candidate nodes recommended for determining the target node, by using pre-obtained benchmark results for each of a plurality of nodes based on the importance level information” in combination with other limitations in the claims as defined by Applicants.
Regarding claim 18, the closest prior art of record either alone or in combination fails to anticipate or render obvious the combination wherein “wherein the benchmark configuration setting further comprises target area information for identifying a target area which is an object of a benchmark within the target model, and wherein the benchmark result comprises estimated performance information corresponding to the identified target area when the benchmark is performed at the target node” in combination with other limitations in the claims as defined by Applicants.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Park et al. (WO2024181616) discloses a method, performed by a computing device, for providing result of benchmarking. The method may comprise the steps of: acquiring first input data including an inference task and a dataset; determining a target model to be benchmarked with respect to the inference task and at least one target node in which the inference task of the target model is to be executed, the determined target model corresponding to an artificial intelligence-based model in which the inference task is benchmarked in the at least one target node on the basis of the dataset; and providing the results of benchmarking acquired from executing the target model in the at least one target node (Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUONG HUYNH whose telephone number is (571)272-2718. The examiner can normally be reached M-F: 9:00AM-5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew M Schechter can be reached at 571-272-2302. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUONG HUYNH/Primary Examiner, Art Unit 2857 March 27, 2026
Prosecution Insights