DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Regarding Chinese Patent App. No. CN202310355429.1 (filed 4/3/2023), receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements submitted on 9/26/2025 and 5/28/2026 have been considered.
Drawings
The drawings are objected to because Figures 13 and 15-25 should be corrected to comply with the applicable sections of 37 CFR 1.84 set forth below. In particular, such figures should be drawings using India ink or its equivalent.
(a) Drawings. There are two acceptable categories for presenting drawings in utility and design patent applications.
(1) Black ink. Black and white drawings are normally required. India ink, or its equivalent that secures solid black lines, must be used for drawings; or
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 20 is rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claim 20 recites the limitation “the performing scheduling and resource allocation on each instructions in the respective second intermediate representation instruction sequences based on the hardware resources of the AI accelerator, to obtain running results of the respective second intermediate representation instruction sequences comprises:” in lines 1-5. There is insufficient antecedent basis for this limitation in the claim. The examiner suggests amending claim 20 to depend from claim 19 in over to overcome this rejection.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6 and 11-18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 20220222584 A1, hereinafter referenced as NIMMAGADDA.
Regarding Claim 1
NIMMAGADDA teaches:
A method for generating instructions of an artificial intelligence accelerator, the method comprising: (NIMMAGADDA, para. 0084: “The edge computing node 750 may include or be coupled to acceleration circuitry 764, which may be embodied by one or more artificial intelligence (AI) accelerators, a neural compute stick, neuromorphic hardware, an FPGA, an arrangement of GPUs, an arrangement of xPUs/DPUs/IPU/NPUs, one or more SoCs, one or more CPUs, one or more digital signal processors, dedicated ASICs, or other forms of specialized processors or circuitry designed to accomplish one or more specialized tasks. These tasks may include AI processing (including machine learning, training, inferencing, and classification operations), visual data processing, network data processing, object detection, rule analysis, or the like. These tasks also may include the specific edge computing tasks for service management and service operations discussed elsewhere in this document.”;
NIMMAGADDA, para. 0099: “FIG. 8 is a flow diagram of an example of a system 800 for heterogeneous compute-based artificial intelligence model partitioning, according to an embodiment. The system 800 may execute on the compute node 700 as described in FIG. 7A, edge computing node 750 as described in FIG. 7B, the edge nodes 422, 424 as described in FIG. 4, compute nodes 515, 523, 537, 544 as described in FIG. 5, etc. operating on the edge cloud 110 as described in FIGS. 1, 2, 3, and 6.”
NIMMAGADDA, para. 0112: The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).
Examiner’s Note: As shown in Fig.8, NIMMAGADDA discloses techniques for converting an input model into instructions that can run on an AI accelerator using multiple layers of intermediate representations)
determining a first intermediate representation instruction sequence corresponding to a neural network model that needs to be processed by an artificial intelligence (AI) accelerator; (NIMMAGADDA, para. 0100: “An input AI model 805 may be converted into an in-memory graph representation and high-level optimizations are applied (e.g. at operation 810) using a variety of graph compilers (e.g., an AI model inference optimization and deployment engine, TensorFlow, PyTorch, etc.) during inference (e.g., processing data for input into a trained AI model). Input models may include a variety of machine learning models that may include layers, filters, nodes, and the like. Optimizing the model during inference may improve the compute efficiency of the model and may allow for more effective partitioning of the model for distribution amongst devices having different hardware specifications, resources, types, etc. This allows a complex model to be distributed amongst a heterogeneous set of compute resources increasing flexibility of distributing portions of the model workload.”;
Examiner’s Note: As shown in Fig. 8, the output of the high-level graph optimizations 810 corresponds to the recited “first intermediate representation instruction sequence”, which takes an input model (having layers and nodes, e.g., a neural network model), so that it can ultimately run the model on the devices 880a, which as explained above can be an AI accelerator)
converting, based on a hardware resources of the AI accelerator, the first intermediate representation instruction sequence into one or more second intermediate representation instruction sequences respectively by using one or more code generation manners; and (NIMMAGADDA, para. 0102: “The IR generator 815 receives the applied graph optimizations and converts the model to an intermediate representation (IR) using an offline step or an inline step. For example, an AI model inference optimization and deployment engine may be used to convert a TensorFlow model to IR or a PyTorch to open neural network exchange (ONNX) converter may be used to export a PyTorch model to an ONNX model offline. The TensorFlow model may also be converted to an AI model inference optimization and deployment engine intermediate representation using an inline method with a TensorFlow frontend module in the AI model inference optimization and deployment engine. The TensorFlow frontend module traverses the graph after high level optimizations are applied and converts each TensorFlow operator to AI model inference optimization and deployment engine operator. The nodes are topologically sorted and a directed acyclic graph with IR is produced by the IR generator 815.”;
NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: NIMMAGADDA explains that the in-memory graph representations (the first intermediate representation) is converted to an intermediate representation (corresponding to the recited “second intermediate representation”), and then the IR is optimized to a target dependent optimization IR which takes into account the hardware capabilities of the device 880, using a TensorFlow frontend module (corresponding to recited “code generating manners”)
generating an AI accelerator-executable instruction based on the hardware resources of the AI accelerator and the one or more second intermediate representation instruction sequences. (NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: NIMMAGADDA explains that target dependent optimized IR (corresponding to the recited “second intermediate representation”) which takes into account the hardware capabilities of the device 880, is compiled and loaded on a respective device, which can be an AI accelerator as explained above.)
Regarding Claim 2
NIMMAGADDA discloses the method of claim 1 as explained above (including the “wherein the converting, based on a hardware resources of the AI accelerator, the first intermediate representation instruction sequence into one or more second intermediate representation instruction sequences respectively by using one or more code generation manners comprises:” limitation). NIMMAGADDA further teaches:
determining, based on the first intermediate representation instruction sequence, to-be-processed objects corresponding to respective neural network layers in the neural network model, and a first dependency relationship between the to-be-processed objects corresponding to the respective neural network layers; and (NIMMAGADDA, para. 0100: “An input AI model 805 may be converted into an in-memory graph representation and high-level optimizations are applied (e.g. at operation 810) using a variety of graph compilers (e.g., an AI model inference optimization and deployment engine, TensorFlow, PyTorch, etc.) during inference (e.g., processing data for input into a trained AI model). Input models may include a variety of machine learning models that may include layers, filters, nodes, and the like. Optimizing the model during inference may improve the compute efficiency of the model and may allow for more effective partitioning of the model for distribution amongst devices having different hardware specifications, resources, types, etc. This allows a complex model to be distributed amongst a heterogeneous set of compute resources increasing flexibility of distributing portions of the model workload.”;
NIMMAGADDA, para. 0101: “Target independent optimizations such as horizontal or vertical node fusions, redundant node removal, training node removal, batchnorm folding, etc. are performed. For example, node fusion using TensorFlow Grappler may fuse convolution and rectified linear unit (ReLU) operators together to form a “_FusedConv2D” operator. An example of training node removal may use a model.eval( ) call in PyTorch before running inference to remove training nodes like “Dropout” from the graph.”;
Examiner’s Note: NIMMAGADDA discloses that the conversion to an in-memory graph representation (corresponding to recited “first intermediate representation”) for layers and nodes to depict in the graph representation, where dependencies between objects are utilized to perform horizontal or vertical node fusions, or to remove redundant nodes, etc.)
generating, based on the hardware resources of the AI accelerator and the first dependency relationship, a plurality of first sub-intermediate representation instruction sequences included in a third intermediate representation instruction sequence in the one or more second intermediate representation instruction sequences. (NIMMAGADDA, para. 0102: “The IR generator 815 receives the applied graph optimizations and converts the model to an intermediate representation (IR) using an offline step or an inline step. For example, an AI model inference optimization and deployment engine may be used to convert a TensorFlow model to IR or a PyTorch to open neural network exchange (ONNX) converter may be used to export a PyTorch model to an ONNX model offline. The TensorFlow model may also be converted to an AI model inference optimization and deployment engine intermediate representation using an inline method with a TensorFlow frontend module in the AI model inference optimization and deployment engine. The TensorFlow frontend module traverses the graph after high level optimizations are applied and converts each TensorFlow operator to AI model inference optimization and deployment engine operator. The nodes are topologically sorted and a directed acyclic graph with IR is produced by the IR generator 815.”;
NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: NIMMAGADDA explains that the in-memory graph representations (the first intermediate representation) are converted to an intermediate representation, and then the IR is optimized to a target dependent optimization (corresponding to the recited “second intermediate representation”) which then creates an optimized IR (see Fig. 8) at 870b (corresponding to recited “first sub-intermediate representation instruction sequences included in a third intermediate representation instruction sequence in the one or more second intermediate representation instruction sequences”)
Regarding Claim 3
NIMMAGADDA discloses the method of claim 2 as explained above (including the “wherein the generating, based on the hardware resources of the AI accelerator and the first dependency relationship, the plurality of first sub-intermediate representation instruction sequences included in the third intermediate representation instruction sequence comprises” limitation). NIMMAGADDA further teaches:
splitting, based on the hardware resources of the AI accelerator and the first dependency relationship, each of the to-be-processed objects to obtain a plurality of to-be-processed subobjects; (NIMMAGADDA, para. 0100: “An input AI model 805 may be converted into an in-memory graph representation and high-level optimizations are applied (e.g. at operation 810) using a variety of graph compilers (e.g., an AI model inference optimization and deployment engine, TensorFlow, PyTorch, etc.) during inference (e.g., processing data for input into a trained AI model). Input models may include a variety of machine learning models that may include layers, filters, nodes, and the like. Optimizing the model during inference may improve the compute efficiency of the model and may allow for more effective partitioning of the model for distribution amongst devices having different hardware specifications, resources, types, etc. This allows a complex model to be distributed amongst a heterogeneous set of compute resources increasing flexibility of distributing portions of the model workload.”;
NIMMAGADDA, para. 0109: “The heterogeneous IR partitioner 835 takes input from the compute analyzer 820 and the compute normalizer 840 to identify optimal partitioning points of an AI model.”;
Examiner’s Note: As shown in Fig. 8, the heterogeneous IR partitioner 835 divides the model (containing the objects) into sub-objects for target-dependent optimizations based on the hardware)
determining, based on the first dependency relationship, a second dependency relationship between the plurality of to-be-processed subobjects corresponding to the respective neural network layers; and (NIMMAGADDA, para. 0101: “Target independent optimizations such as horizontal or vertical node fusions, redundant node removal, training node removal, batchnorm folding, etc. are performed. For example, node fusion using TensorFlow Grappler may fuse convolution and rectified linear unit (ReLU) operators together to form a “_FusedConv2D” operator. An example of training node removal may use a model.eval( ) call in PyTorch before running inference to remove training nodes like “Dropout” from the graph.”;
Examiner’s Note: the portioning relates to portioning of the AI model with respect to particular neural network layers, such as performing redundance node removal or horizontal or vertical node fusions solely on the partitioned IR)
generating the plurality of first sub-intermediate representation instruction sequences based on the second dependency relationship. (NIMMAGADDA, para. 0102: “The IR generator 815 receives the applied graph optimizations and converts the model to an intermediate representation (IR) using an offline step or an inline step. For example, an AI model inference optimization and deployment engine may be used to convert a TensorFlow model to IR or a PyTorch to open neural network exchange (ONNX) converter may be used to export a PyTorch model to an ONNX model offline. The TensorFlow model may also be converted to an AI model inference optimization and deployment engine intermediate representation using an inline method with a TensorFlow frontend module in the AI model inference optimization and deployment engine. The TensorFlow frontend module traverses the graph after high level optimizations are applied and converts each TensorFlow operator to AI model inference optimization and deployment engine operator. The nodes are topologically sorted and a directed acyclic graph with IR is produced by the IR generator 815.”;
NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: NIMMAGADDA explains that the in-memory graph representations (the first intermediate representation) are converted to an intermediate representation, and then the IR is optimized to a target dependent optimization (corresponding to the recited “second intermediate representation”) which then creates an optimized IR (see Fig. 8) at 870b (corresponding to recited “first sub-intermediate representation instruction sequences included in a third intermediate representation instruction sequence in the one or more second intermediate representation instruction sequences”, where the dependency between the sub-AI model and its neural networks are used when generating the optimized IR (870A, 870B, 870N)
Regarding Claim 4
NIMMAGADDA discloses the method of claim 3 as explained above (including the “wherein the generating the plurality of first sub-intermediate representation instruction sequences based on the second dependency relationship comprises” limitation). NIMMAGADDA further teaches:
determining, based on the second dependency relationship, second to-be-processed subobjects having dependency relationships with a plurality of first to-be-processed subobjects in an output object of the neural network model; and (NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: the recited first dependency relationships are at the in memory graph-level (see 810) and the recited second dependency relationships are at the optimized IR level (870A, 870B, 870N), and at the optimized IR level, sub-objects such as neural network nodes can be determined, where they each have an association with the graph-level representation of the first to-be-processed subobjects)
generating the plurality of first sub-intermediate representation instruction sequences corresponding to the plurality of first to-be-processed subobjects based on the plurality of first to-be-processed subobjects, the second to-be-processed subobjects having the dependency relationships with the respective first to-be-processed subobjects, and the dependency relationships between the first to-be-processed subobjects and the second to-be-processed subobjects. (NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: the optimized IR (870A, 870B, 870N) is generated with all of these dependencies taken into account)
Regarding Claim 5
NIMMAGADDA discloses the method of claim 2 as explained above (including the “wherein the generating, based on the hardware resources of the AI accelerator and the first dependency relationship, the plurality of first sub-intermediate representation instruction sequences included in the third intermediate representation instruction sequence comprises” limitation). NIMMAGADDA further teaches:
determining, based on the hardware resources of the AI accelerator and the first dependency relationship, operation windows of the to-be-processed objects corresponding to the respective neural network layers; (NIMMAGADDA, para. 0109: “The heterogeneous IR partitioner 835 takes input from the compute analyzer 820 and the compute normalizer 840 to identify optimal partitioning points of an AI model. If the overall compute (e.g., in FLOPs) present in the model is ‘f’ that needs to be partitioned across N input processing devices (e.g., device_1 880, device_2 885, and device_n 890), the flops ‘f.sub.n’ of the partitioned model are determined that need to be executed on an n.sup.th device using the formula f.sub.n=f*C.sub.n/Σ(C.sub.i), i=1 to N where f.sub.n is the FLOPs to be executed on the device, f is the total number of FLOPs of the model, C.sub.i is the compute normalization factor of i.sup.th device, and N is the number of devices used for partitioning. The compute normalization of the n.sup.th device C.sub.n is computed using the formula C.sub.n=(1/p.sub.n)*(1−s)*e.sub.n*T.sub.n where C.sub.n is the compute normalization factor of the device (e.g. calculated by the compute normalizer 840), p.sub.n is the precision factor 845 of the device, s is the sparsity factor 830 of the model, e.sub.n is the compute efficiency factor 850, and T.sub.n is the theoretical compute of the device (e.g., theoretical maximum compute bound of the device).”;
Examiner’s Notes: the N optimized partitioning points of an AI model, which each pertain to the amount of FLOPs of the model to be partitioned to a particular device, correspond to the recited “operation windows”, consistent with how such term is used in para. 00185 of the instant specification)
splitting, based on the operation windows of the to-be-processed objects and the first dependency relationship, each of the to-be-processed objects to obtain a plurality of to-be-processed subobjects, and determining a second dependency relationship between the plurality of to-be-processed subobjects corresponding to the respective neural network layers; and (NIMMAGADDA, para. 0100: “An input AI model 805 may be converted into an in-memory graph representation and high-level optimizations are applied (e.g. at operation 810) using a variety of graph compilers (e.g., an AI model inference optimization and deployment engine, TensorFlow, PyTorch, etc.) during inference (e.g., processing data for input into a trained AI model). Input models may include a variety of machine learning models that may include layers, filters, nodes, and the like. Optimizing the model during inference may improve the compute efficiency of the model and may allow for more effective partitioning of the model for distribution amongst devices having different hardware specifications, resources, types, etc. This allows a complex model to be distributed amongst a heterogeneous set of compute resources increasing flexibility of distributing portions of the model workload.”;
NIMMAGADDA, para. 0109: “The heterogeneous IR partitioner 835 takes input from the compute analyzer 820 and the compute normalizer 840 to identify optimal partitioning points of an AI model.”;
Examiner’s Note: As shown in Fig. 8, the heterogeneous IR partitioner 835 divides the model (containing the objects) into first sub-objects based on the N partitions (corresponding to recited “operation windows”)
generating the plurality of first sub-intermediate representation instruction sequences based on the second dependency relationship and the operation windows of the to-be-processed objects. (NIMMAGADDA, para. 0102: “The IR generator 815 receives the applied graph optimizations and converts the model to an intermediate representation (IR) using an offline step or an inline step. For example, an AI model inference optimization and deployment engine may be used to convert a TensorFlow model to IR or a PyTorch to open neural network exchange (ONNX) converter may be used to export a PyTorch model to an ONNX model offline. The TensorFlow model may also be converted to an AI model inference optimization and deployment engine intermediate representation using an inline method with a TensorFlow frontend module in the AI model inference optimization and deployment engine. The TensorFlow frontend module traverses the graph after high level optimizations are applied and converts each TensorFlow operator to AI model inference optimization and deployment engine operator. The nodes are topologically sorted and a directed acyclic graph with IR is produced by the IR generator 815.”;
NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: NIMMAGADDA explains that optimized IR (870A, 870B, 870N) is based on the N partition points (corresponding to recited “operation windows”) and the secondary relationships between sub-objects in the IR and the respective neural network layers)
Regarding Claim 6
NIMMAGADDA discloses the method of claim 1 as explained above (including the “wherein the converting, based on a hardware resources of the AI accelerator, the first intermediate representation instruction sequence into one or more second intermediate representation instruction sequences respectively by using one or more code generation manners comprises” limitation). NIMMAGADDA further teaches:
determining an operational flow of the neural network model based on the first intermediate representation instruction sequence; and (NIMMAGADDA, para. 0100: “An input AI model 805 may be converted into an in-memory graph representation and high-level optimizations are applied (e.g. at operation 810) using a variety of graph compilers (e.g., an AI model inference optimization and deployment engine, TensorFlow, PyTorch, etc.) during inference (e.g., processing data for input into a trained AI model). Input models may include a variety of machine learning models that may include layers, filters, nodes, and the like. Optimizing the model during inference may improve the compute efficiency of the model and may allow for more effective partitioning of the model for distribution amongst devices having different hardware specifications, resources, types, etc. This allows a complex model to be distributed amongst a heterogeneous set of compute resources increasing flexibility of distributing portions of the model workload.”;
Examiner’s Note: As shown in Fig. 8, the in-memory graph representations corresponds to the recited “first intermediate representation instruction sequence”, which shows the flow of operations consistent with at least Fig. 8 of the instant disclosure)
generating a fourth intermediate representation instruction sequence in the one or more second intermediate representation instruction sequences based on the hardware resources of the AI accelerator and the operational flow of the neural network model. (NIMMAGADDA, para. 0102: “The IR generator 815 receives the applied graph optimizations and converts the model to an intermediate representation (IR) using an offline step or an inline step. For example, an AI model inference optimization and deployment engine may be used to convert a TensorFlow model to IR or a PyTorch to open neural network exchange (ONNX) converter may be used to export a PyTorch model to an ONNX model offline. The TensorFlow model may also be converted to an AI model inference optimization and deployment engine intermediate representation using an inline method with a TensorFlow frontend module in the AI model inference optimization and deployment engine. The TensorFlow frontend module traverses the graph after high level optimizations are applied and converts each TensorFlow operator to AI model inference optimization and deployment engine operator. The nodes are topologically sorted and a directed acyclic graph with IR is produced by the IR generator 815.”;
NIMMAGADDA, para. 0112: “The partitioned IR output by the heterogenous IR partitioner 835 includes target dependent optimizations (e.g., based on hardware and its capabilities, etc.) 855A, 855B, and 855N that are compiled and loaded (e.g., at operations 870A, 870B, and 870N). The complied models (e.g., 875A, 875B, and 875N) are executed on the respective device (e.g., device_1 880A, device_2 880B, and device_n 880N).”
Examiner’s Note: NIMMAGADDA explains that the in-memory graph representations (the first intermediate representation) are converted to an intermediate representation, and then the IR is optimized to a target dependent optimization (corresponding to the recited “second intermediate representation”) which then creates an optimized IR (see Fig. 8) at 870n (corresponding to recited “fourth intermediate representation instruction sequence”)
Regarding Claim 11
NIMMAGADDA teaches:
A non-transient computer readable storage medium, wherein a computer program is stored thereon, which, when executed by a processor, is used for implementing a method for generating instructions of an artificial intelligence accelerator, comprising: (NIMMAGADDA, para. 0091: “ In an example, the instructions 782 provided via the memory 754, the storage 758, or the processor 752 may be embodied as a non-transitory, machine-readable medium 760 including code to direct the processor 752 to perform electronic operations in the edge computing node 750. The processor 752 may access the non-transitory, machine-readable medium 760 over the interconnect 756. For instance, the non-transitory, machine-readable medium 760 may be embodied by devices described for the storage 758 or may include specific storage units such as optical disks, flash drives, or any number of other hardware devices.”)
The remaining limitations correspond to the method of claim 1, and therefore this claim is rejected for the same reasons explained above with respect to claim 1.
Regarding Claim 12
NIMMAGADDA teaches:
An electronic apparatus, the electronic apparatus comprising: a processor; and a memory, configured to store a processor-executable instruction, wherein the processor is configured to read the executable instruction from the memory, and execute the instruction to implement the following steps of: (NIMMAGADDA, para. 0091: “ In an example, the instructions 782 provided via the memory 754, the storage 758, or the processor 752 may be embodied as a non-transitory, machine-readable medium 760 including code to direct the processor 752 to perform electronic operations in the edge computing node 750. The processor 752 may access the non-transitory, machine-readable medium 760 over the interconnect 756. For instance, the non-transitory, machine-readable medium 760 may be embodied by devices described for the storage 758 or may include specific storage units such as optical disks, flash drives, or any number of other hardware devices.”)
The remaining limitations correspond to the method of claim 1, and therefore this claim is rejected for the same reasons explained above with respect to claim 1.
Claim 13 depends from claim 12 and recites an apparatus that corresponds to the method of claim 2, and is therefore rejected for the same reasons explained above with respect to claims 2 and 12.
Claim 14 depends from claim 13 and recites an apparatus that corresponds to the method of claim 3, and is therefore rejected for the same reasons explained above with respect to claims 3 and 13.
Claim 15 depends from claim 14 and recites an apparatus that corresponds to the method of claim 4, and is therefore rejected for the same reasons explained above with respect to claims 4 and 14.
Claim 16 depends from claim 13 and recites an apparatus that corresponds to the method of claim 5, and is therefore rejected for the same reasons explained above with respect to claims 5 and 13.
Claim 17 depends from claim 12 and recites an apparatus that corresponds to the method of claim 6, and is therefore rejected for the same reasons explained above with respect to claims 6 and 12.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 7 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over NIMMAGADDA in view of US 20190392296 A1, hereinafter referenced as BRADY.
Regarding Claim 7
NIMMAGADDA discloses the method of claim 1 as explained above (including the “wherein the generating an AI accelerator-executable instruction based on the hardware resources of the AI accelerator and the one or more second intermediate representation instruction sequence comprises” limitation). However, NIMMAGADA fails to explicitly teach:
determining a target intermediate representation instruction sequence in the one or more second intermediate representation instruction sequences based on the hardware resources of the AI accelerator; and
generating the AI accelerator-executable instruction based on the target intermediate representation instruction sequence.
However, in a related field of endeavor (neural network compilers, see para. 0001), BRADY teaches and makes obvious:
determining a target intermediate representation instruction sequence in the one or more second intermediate representation instruction sequences based on the hardware resources of the AI accelerator; and (BRADY, para. 0069: “With this information, additional compilation passes may be performing (using the operator model and/or control model of the IR) to determine which operations are to be performed by which compute resources and in what order. With the assignment of compute resources and operation order set, together with the collection of tensor attribute information through preceding compilation passes, memory allocation passes may be performed (using a data model of the IR) to determine how best to allocate memory to enable fast and efficient use of the tensors to thereby optimize performance of the operations of the neural network by the particular target hardware.”;
Examiner’s Note: BRADY teaches multiple optimization passes in order to determine the best order of the sequences of the intermediate representation before going to the compiler; the NIMMAGADDA-BRADY combination now does multiple optimization passes on the intermediate representations of NIMMAGADDA (870b), and re-arranges those representations into an optimal order (corresponding to recited “target intermediate representation”))
generating the AI accelerator-executable instruction based on the target intermediate representation instruction sequence. (BRADY, para. 0069: “With this information, additional compilation passes may be performing (using the operator model and/or control model of the IR) to determine which operations are to be performed by which compute resources and in what order. With the assignment of compute resources and operation order set, together with the collection of tensor attribute information through preceding compilation passes, memory allocation passes may be performed (using a data model of the IR) to determine how best to allocate memory to enable fast and efficient use of the tensors to thereby optimize performance of the operations of the neural network by the particular target hardware.”;
Examiner’s Note: BRADY teaches multiple optimization passes in order to determine the best order of the sequences of the intermediate representation before going to the compiler; the NIMMAGADDA-BRADY combination now does multiple optimization passes on the intermediate representations of NIMMAGADDA (870b), and re-arranges those representations into an optimal order (corresponding to recited “target intermediate representation”), and now uses the optimized order when compiling code for the devices as explained in NIMMAGADDA at para. 0112)
Before the effective filing date of the application, it would have been obvious to one of ordinary skill in the art to combine the teachings of NIMMAGADDA and BRADY as disclosed herein. As disclosed by BRADY, one of ordinary skill would have been motivated to do so in order to utilize sets of compilation passes which can “identify hardware-agnostic opportunities, rooted in the underlying mathematics of the operations to be performed to implement the neural network, to generate a pared, more efficient version of the neural network (and reflect these modifications in a transformation of the intermediate representation graph).” (para. 0062).
Allowable Subject Matter
Claims 8-10 and 19-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims (and, solely with respect to claim 20, that the rejection under 35 U.S.C. 112(b) is overcome).
The following is a statement of reasons for the indication of allowable subject matter:
Claim 8 would be considered allowable because none of the references of record either alone or in combination fairly disclose or suggest the combination of limitations specified in the claim, including at least:
performing scheduling and resource allocation on each instruction in the respective second intermediate representation instruction sequences based on the hardware resources of the AI accelerator, to obtain running results of the respective second intermediate representation instruction sequences; and
determining the target intermediate representation instruction sequence based on the running results of the respective second intermediate representation instruction sequences.
The closest prior art of record discloses:
20220222584 A1, hereinafter referenced as NIMMAGADDA discloses system for compute-based artificial intelligence model partitioning. (see para. 0099, Fig. 8). As shown in Fig. 8, there are multiple layers of intermediate representations and optimizations to convert an AI model to compiled machine code on an end device. (paras. 0100-0112).
US 20190392296 A1, hereinafter referenced as BRADY teaches multiple optimization passes in order to determine the best order of the sequences of the intermediate representation before going to the compiler. (para. 0069). BRADY further teaches determining results of one or more memory allocation passes. (para. 0087).
US 20230195439 A1, hereinafter referenced as EGGER, teaches a schedule module for scheduling instructions of intermediate representation codes and outputs a scheduling result. (para. 0083).
However, the examiner has found that the distinct feature of the Applicant's claimed invention over the prior art is the explicit claiming of the aforementioned limitations in combination with all the other limitations as specified in claim 8. Moreover, the examiner finds that one of ordinary skill would not have been motivated to combine the prior art of record in the manner claimed without the hindsight aid of Applicant’s disclosure. Therefore, because the prior art of record does not anticipate nor make obvious the limitations of claim 8, claim 8 would be allowed if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 9-10 depend from claim 8 and would be allowed for the same reasons explained with respect to claim 8 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 19 depends from claim 18 and recites an apparatus that corresponds to the method of claim 8, and would therefore be allowed for the same reasons explained above with respect to claim 8 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 20 would be considered allowable because none of the references of record either alone or in combination fairly disclose or suggest the combination of limitations specified in the claim, including at least:
wherein the performing scheduling and resource allocation on each instructions in the respective second intermediate representation instruction sequences based on the hardware resources of the AI accelerator, to obtain running results of the respective second intermediate representation instruction sequences comprises:
determining a third dependency relationship between the respective instructions in the second intermediate representation instruction sequence based on the second intermediate representation instruction sequence;
determining required resources for respective instructions in the second intermediate representation instruction sequences; and
performing scheduling and resource allocation on the respective instructions in the second intermediate representation instruction sequences based on the third dependency relationship and the required resources of the respective instructions in the second intermediate representation instruction sequences, to obtain the running results of the respective second intermediate representation instruction sequences.
The closest prior art of record discloses:
20220222584 A1, hereinafter referenced as NIMMAGADDA discloses system for compute-based artificial intelligence model partitioning. (see para. 0099, Fig. 8). As shown in Fig. 8, there are multiple layers of intermediate representations and optimizations to convert an AI model to compiled machine code on an end device. (paras. 0100-0112).
US 20190392296 A1, hereinafter referenced as BRADY teaches multiple optimization passes in order to determine the best order of the sequences of the intermediate representation before going to the compiler. (para. 0069). BRADY further teaches determining results of one or more memory allocation passes. (para. 0087).
US 20230195439 A1, hereinafter referenced as EGGER, teaches a schedule module for scheduling instructions of intermediate representation codes and outputs a scheduling result. (para. 0083).
However, the examiner has found that the distinct feature of the Applicant's claimed invention over the prior art is the explicit claiming of the aforementioned limitations in combination with all the other limitations as specified in claim 20. Moreover, the examiner finds that one of ordinary skill would not have been motivated to combine the prior art of record in the manner claimed without the hindsight aid of Applicant’s disclosure. Therefore, because the prior art of record does not anticipate nor make obvious the limitations of claim 20, claim 20 would be allowed if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and provided that the rejections under 35 U.S.C. 112(b) are overcome.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20210319369 A1 (Nimmagadda). “As shown in the example of FIG. 2, the model generation pipeline 205 (which is a software pipeline) includes a shape inference process 210, a graph optimization process 220, an intermediate representation (IR) translation process 230, a compilation process 240 (e.g., a compiler), and an execution process 250. A multi-level set of cached intermediate model stages 207 includes cached stages from the processes of the pipeline 205. The multi-level set of cached intermediate model stages 207 can be stored in a common cache memory, or in different cache memories, which can be co-located or distributed in different cache locations. In embodiments, the cache memory or memories are incorporated within the pipeline 205; in some embodiments the cache memory or memories can be separate from the pipeline 205. In embodiments, the multi-level set of intermediate stages corresponds to a hierarchy of processing stages in the model pipeline 205.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL C LEE whose telephone number is (571)272-4933. The examiner can normally be reached M-F 12:00 pm - 8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL C. LEE/Examiner, Art Unit 2128
Prosecution Insights