DETAILED ACTION
This action is responsive to the Application/ filed on 08/31/2023. Claims 1-8 are pending in the case. Claims 1, 5, 7 and 8 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101 because the claim are directed to an abstract idea without significantly more.
Regarding Claim 1, 5, 7 and 8:
Under step 1, claim 1 is directed to a method which is directed to a method, one of the statutory categories.
Under step 1, claim 5 is directed to a method which is directed to a method, one of the statutory categories.
Under step 1, claim 7 is directed to a method which is directed to a system, one of the statutory categories.
Under step 1, claim 8 is directed to a method which is directed to a product of manufacture, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations:
creating a simulation graph based on the machine learning system;
simulating an implementation of the machine learning system … using … the simulation graph,
the simulation being an event-based simulation;
and ascertaining the performance based on a result of the simulation.
[from claim 5]
wherein the ascertained performance is used to determine whether each of the parameters characterizing the neural network and/or parameters characterizing the processing unit are adjusted within a predefined parameter range of the parameter, and wherein the steps … simulating, and ascertaining the performance are carried out again based on the modified parameters, the procedure being repeated several times until a predefined target performance is achieved.
Each of these amount to mental evaluation because they describe manipulation of abstract data. Creating simulation graphs amount to creation of abstract representations of operations or actions taken place according to a specification and based on the claim context. Simulating such graphs amounts to evaluation of the features of the graphs. Finally, ascertaining performance and determinations about whether the network is characterized is a judgement about the abstract results, therefore these limitations can be performed in the mind, such that the claim recites a judicial exception.
Under step 2A Prong 2, The claim recites the following additional element(s):
A computer-implemented method…simulating...on the processing unit using the hardware model…[claim 5] a processing unit for running the neural network with regard to optimizing hardware performance…[claim 7] the apparatus configured to: …[claim 8] A non-transitory machine-readable storage medium on which is stored a computer program including commands for ascertaining a performance of a machine learning system on a processing unit, the commands, when executed by a computer, causing the computer to perform the following step (amounts to mere instructions to apply a computer technology to an abstract idea, see MPEP 2106.05(f))
creating a hardware model of the processing unit from a provided technical specification of the processing unit… wherein the steps of creating… based on the modified parameters, the procedure being repeated several times (that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g))
Therefore the claim is directed to a judicial exception.
Under step 2B, the additional element creating a hardware model of the processing unit from a provided technical specification of the processing unit are insignificant extra-solution activities that are considered well-understood, routine, conventional activities. In accordance with the MPEP, the following factual determination is based on the technical publication: Shen et al. “An RTL Abstraction Technique for Processor Microarchitecture Validation and Test Generation” (PTO-892)]. Shen Section 1.1 pg 68:
“In current industrial practice, simulation is the primary means of verification. Usually, the simulation based verification environment consists of an RTL design of the processor and a more abstract reference model as the specification… RTL and reference machine co-simulation and state comparison is the most widely used correctness checking technique in processor validation.”
discloses that RTL and reference co-simulation is the most widely used technique and is current industrial practice (corresponds to well-understood, routine and conventional) such co-simulation is the creation of an RTL design, or claimed hardware model of a processing unit, from a abstract reference model or technical specification (corresponds to creating a hardware model of the processing unit from a provided technical specification of the processing unit). As such, the insignificant extra-solution activities are considered well-understood, routine, conventional activities.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 2:
The rejection of claim 1 is incorporated and further:
Under Step 2A Prong 1, The claim recites the limitations:
wherein the machine learning system is a neural network, a neural network graph being converted into a tree representation … during the step of creating the simulation graph, the tree representation being converted into a Petri net graph, which is provided to the simulation
Each of these amount to mental evaluation because they describe manipulation of abstract data. Conversion of a neural network into a tree representation which is a petri-net graph amounts to translation of abstract representations of computer operations. Such a description of operations amounts to an evaluation of abstract data; therefore these limitations can be performed in the mind, such that the claim recites a judicial exception
Under step 2A Prong 2, The claim recites the following additional element(s):
by a machine learning compiler (amounts to mere instructions to apply a computer technology to an abstract idea, see MPEP 2106.05(f))
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 3:
The rejection of claim 1 is incorporated and further:
Under Step 2A Prong 1, The claim recites the limitations:
wherein internal hardware processes are simulated with an event-based simulation.
Such limitation merely further describe the recited abstract ideas.
Furthermore, under step 2A Prong 2 and 2B, the claim does not recite additional elements to consider.
Regarding Claim 4:
The rejection of claim 1 is incorporated and further:
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
Under step 2A Prong 2, The claim recites the following additional element(s):
wherein the processing unit is a hardware accelerator for the machine learning system. (is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h))
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Regarding Claim 6:
The rejection of claim 5 is incorporated and further:
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
Under step 2A Prong 2, The claim recites the following additional element(s):
once the target performance is achieved, a system is manufactured and/or configured in accordance with the parameters characterizing the neural network and the parameters characterizing the processing unit with which the simulation achieved the target performance. (is generally linking the use of the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h))
Therefore the claim is directed to a judicial exception.
Under step 2B, the recited additional elements when considered alone or in combination neither integrates the abstract idea into a practical application nor provides significantly more than the abstract idea itself.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1,3, 4, 7 and 8 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Klaiber et al. “An End-to-End HW/SW Co-Design Methodology to Design Efficient Deep Neural Network Systems using Virtual Models”
Claim 1
Klaiber teaches, A computer-implemented method for ascertaining a performance of a machine learning system on a processing unit, comprising the following steps: (abstract “Deep learning compilers introduce hardware specific transformations and are, therefore, considered a part of the design flow of virtual system models to extract end-to-end performance estimations. To validate the run-time accuracy of the proposed methodology, a system processing the DilatedVGG DNN is realized both as virtual system model and as hardware implementation. The results show that up to 92 % accuracy” the models evaluate the accuracy or performance of a processing unit realization of a DNN, a type of machine learning model.)
creating a hardware model of the processing unit from a provided technical specification of the processing unit; creating a simulation graph based on the machine learning system; ( pg 3 Section 3 “Each instance of an AVSM is described as system description file that defines the topology of the virtual hardware models of the NCE, the memory sub-system and the bus. It also contains the physical annotations, such as the frequency of the NCE or the memory frequency. The model generation engine then uses the system description file and the hardware-adapted task graph to automatically generate an executable SystemC model that is simulated in Synopsys Platform Architect.” The AVSM is a hardware model based on description file which is a technical specification of a virtual hardware model. Which is used to create a task graph or simulation graph.)
simulating an implementation of the machine learning system on the processing unit using the hardware model and the simulation graph, (pg 3 “Figure 3 shows the total run-time to build an AVSM from the system description file and simulate all layers of the DilatedVGG neural network.” The layer of the neural network are simulated)
the simulation being an event-based simulation; (pg 3 Figure 4 caption “Figure 4: Gantt chart showing simulation of tasks and usage of computation and communication resources.”
PNG
media_image1.png
378
424
media_image1.png
Greyscale
the results depict the execution time of different event thus an event based simulation.)
and ascertaining the performance based on a result of the simulation. (pg 3 Section 3 “The total processing time on an Intel Xeon CPU E5620 running at 2.40 GHz is around 20 minutes. Generation of the hardware-adapted task graph and the hardware models of the AVSM takes 16.4 seconds.” Processing time is a measure of performance)
Claim 3
Klaiber teaches claim 1
Further Klaiber teaches, wherein internal hardware processes are simulated with an event-based simulation. (pg 3 “To determine the system performance, the task graph is deployed in the virtual model of the HKP which controls the execution of the virtual model of the NCE” Section 3 “to transform the internal graph representation into a hardware-adapted task graph,” pg 4 “The virtual-system-based prototyping allows to track computation time at the level of individual operations and the traffic on the bus for each memory transaction. Therefore, a detailed analysis of the performance and efficiency for a design point of a DNN system is possible” the individual operations and bus traffic correspond to the even based simulation of internal hardware functions)
Claim 4
Klaiber teaches claim 1
Further Klaiber teaches, wherein the processing unit is a hardware accelerator for the machine learning system. (pg 3 “To compare the presented methodology quantitatively, an AVSM and an FPGA implementation of the DNN system architecture shown in Figure 2 were created. The physical prototype [4] was implemented on a Xilinx Virtex7 FPGA platform” the fpga platform is a hardware accelerator for implementing a DNN or machine learning system.)
Claim 7
Klaiber teaches, An apparatus configured to ascertain a performance of a machine learning system on a processing unit, the apparatus configured to (abstract “Deep learning compilers introduce hardware specific transformations and are, therefore, considered a part of the design flow of virtual system models to extract end-to-end performance estimations. To validate the run-time accuracy of the proposed methodology, a system processing the DilatedVGG DNN is realized both as virtual system model and as hardware implementation. The results show that up to 92 % accuracy” the models evaluate the accuracy or performance of a processing unit realization of a DNN, a type of machine learning model.)
The remaining limitations are rejected for the reasons set forth in the rejection of claim 1
Claim 8
Klaiber teaches, A non-transitory machine-readable storage medium on which is stored a computer program including commands for ascertaining a performance of a machine learning system on a processing unit, the commands, when executed by a computer, causing the computer to perform the following steps (abstract “Deep learning compilers introduce hardware specific transformations and are, therefore, considered a part of the design flow of virtual system models to extract end-to-end performance estimations. To validate the run-time accuracy of the proposed methodology, a system processing the DilatedVGG DNN is realized both as virtual system model and as hardware implementation. The results show that up to 92 % accuracy” the models evaluate the accuracy or performance of a processing unit realization of a DNN, a type of machine learning model.)
The remaining limitations are rejected for the reasons set forth in the rejection of claim 1
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 are rejected under 35 U.S.C. § 103 as being unpatentable over Klaiber further in view of Karmakar et al. US document ID US 20220366267 A1
Claim 2
Klaiber teaches claim 1
Klaiber further teaches, wherein the machine learning system is a neural network, ( pg 3 “Figure 3 shows the total run-time to build an AVSM from the system description file and simulate all layers of the DilatedVGG neural network”)
Klaiber does not explicitly teach, a neural network graph being converted into a tree representation by a machine learning compiler during the step of creating the simulation graph, the tree representation being converted into a Petri net graph, which is provided to the simulation
Karmakar, however when addressing performance modeling of machine learning models teaches, a neural network graph being converted into a tree representation by a machine learning compiler during the step of creating the simulation graph, (para 0014 “the system comprises a compiler and a petri-net simulator.” Para 0037 “converts the workloads into a common format that is ready for subsequent processing by compiler 56. The common format may comprise, for example, an Abstract Syntax Tree (AST) format” here the workloads are of AI computations, which when combined with Klaiber include neural network workloads or graphs para 0038 “At a workload transformation step 132, transformer module 60 of compiler 56 transforms workloads 48 into respective graphs”) the tree representation being converted into a Petri net graph, which is provided to the simulation. (para 0026 “In the present context, the term “petri-net simulation” refers to a discrete event simulation method built on a mathematical formalism used to describe distributed systems, where tensors are modeled as Places and operators typically as a hierarchy of Transitions and Places. The actual simulation proceeds according to the dynamic execution delay of dependent events, which models conflicts in a resource-constrained system. Examples of petri-net simulators comprise, for example, “CPN Tools” provided by Eindhoven University of Technology, the Netherlands, and a “Petri-Net Toolbox” for MATLAB” the system simulates the hierarchy of transitions and places which is a understood to be a petri-net graph.)
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify virtual performance analysis described by Klaiber to comprise petri-net simulations described by Karmakar. One would have been motivated to make such a combination because both references address platforms for identifying performance bottle necks and characterization of model performance. Further, Klaiber notes optimizing the performance is desirable “This part of the flow has not been optimized for performance yet, therefore, bears a great potential for further improvement” (Section 3 Klaiber). While Karmakar notes “The simulated performance can be used for improving the accelerator design, e.g., by iteratively testing various hardware and/or software configurations”
Claim(s) 5 and 6 are rejected under 35 U.S.C. § 103 as being unpatentable over Klaiber further in view of Chen et al. “TVM:AnAutomated End-to-End Optimizing Compiler for Deep Learning”
Claim 5
Klaiber teaches claim 1
Klaiber further teaches, A computer-implemented method for a joint optimization of a neural network configuration and a processing unit for running the neural network with regard to optimizing hardware performance, the optimizing including (abstract “Deep learning compilers introduce hardware specific transformations and are, therefore, considered a part of the design flow of virtual system models to extract end-to-end performance estimations. To validate the run-time accuracy of the proposed methodology, a system processing the DilatedVGG DNN is realized both as virtual system model and as hardware implementation. The results show that up to 92 % accuracy” the models evaluate the accuracy or performance of a processing unit realization of a DNN, a type of machine learning model.)
creating a hardware model of the processing unit from a provided technical specification of the processing unit, creating a simulation graph based on the machine learning system ( pg 3 Section 3 “Each instance of an AVSM is described as system description file that defines the topology of the virtual hardware models of the NCE, the memory sub-system and the bus. It also contains the physical annotations, such as the frequency of the NCE or the memory frequency. The model generation engine then uses the system description file and the hardware-adapted task graph to automatically generate an executable SystemC model that is simulated in Synopsys Platform Architect.” The AVSM is a hardware model based on description file which is a technical specification of a virtual hardware model. Which is used to create a task graph or simulation graph.)
simulating an implementation of the machine learning system on the processing unit using the hardware model and the simulation graph, (pg 3 “Figure 3 shows the total run-time to build an AVSM from the system description file and simulate all layers of the DilatedVGG neural network.” The layer of the neural network are simulated)
the simulation being an event-based simulation; (pg 3 Figure 4 caption “Figure 4: Gantt chart showing simulation of tasks and usage of computation and communication resources.”
PNG
media_image1.png
378
424
media_image1.png
Greyscale
the results depict the execution time of different event thus an event based simulation.)
and ascertaining the performance based on a result of the simulation. (pg 3 Section 3 “The total processing time on an Intel Xeon CPU E5620 running at 2.40 GHz is around 20 minutes. Generation of the hardware-adapted task graph and the hardware models of the AVSM takes 16.4 seconds.” Processing time is a measure of performance)
Klaiber does not explicitly teach, wherein the ascertained performance is used to determine whether each of the parameters characterizing the neural network and/or parameters characterizing the processing unit are adjusted within a predefined parameter range of the parameter, and wherein the steps of creating, simulating, and ascertaining the performance are carried out again based on the modified parameters, the procedure being repeated several times until a predefined target performance is achieved
Chen however, when addressing optimizing the performance of neural network systems teaches, wherein the ascertained performance is used to determine whether each of the parameters characterizing the neural network and/or parameters characterizing the processing unit are adjusted within a predefined parameter range of the parameter, and wherein the steps of creating, simulating, and ascertaining the performance are carried out again based on the modified parameters, the procedure being repeated several times until a predefined target performance is achieved. (pg 5 Section 4 “TVM produces efficient code for each operator by generating many valid implementations on each hardware back-end and choosing an optimized implementation.” Pg 9 Section 5.2 “We instead take a statistical approach to solve the cost modeling problem. In this approach, a schedule explorer proposes configurations that may improve an operator’s performance. For each schedule configuration, we use an ML model that takes the lowered loop program as in put and predicts its running time on a given hardware back-end. The model, trained using runtime measurement data collected during exploration, does not require the user to input detailed hardware information. We update the model periodically as we explore more configurations during optimization, which improves accuracy for other related workloads, as well. In this way, the quality of the ML model improves with more experimental trials.” This system repeatedly creates and simulates via the program the performance with each new configuration or set or modified parameters. The performance cost or accuracy is used in part to determine the optimal configuration via repetitive experimental trials or iterations.)
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify virtual performance analysis described by Klaiber to comprise compiler optimization for acceleration of end devices. One would have been motivated to make such a combination because both references address platforms for identifying performance bottle necks and characterization of model performance. Further, Klaiber notes optimizing the performance is desirable “This part of the flow has not been optimized for performance yet, therefore, bears a great potential for further improvement” (Section 3 Klaiber). While Chen notes “Experimental results show that TVM delivers performance across hardware back-ends that are competitive with state-of the-art, hand-tuned libraries” (Chen abstract)
Claim 6
Klaiber/Chen teach claim 5
Further Chen teaches, wherein, once the target performance is achieved, a system is manufactured and/or configured in accordance with the parameters characterizing the neural network and the parameters characterizing the processing unit with which the simulation achieved the target performance. (pg 4 “Possible optimizations form a large space, so we use an ML-based cost model to find optimized operators. Finally, the system packs the generated code into a deployable module.” The system arrives at optimized operators or configurations of a neural network on the processor, where optimized is the target performance level, which is then configured on a deployable module)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNATHAN R GERMICK whose telephone number is (571)272-8363. The examiner can normally be reached M-F 9:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.R.G./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights