DETAILED ACTION
This non-final office action is responsive to application 18/352,784 as submitted 14 July 2023.
Claim status is currently pending and under examination for claims 1-20 of which independent claims are 1, 10 and 20.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for domestic priority under 35 U.S.C. 119 (e), 120, 121, 365(c) or 386(c). Based on provisional filing 63/389,673 the application has an effective filing date of 07/15/2022.
Specification
The disclosure is objected to because it contains an embedded hyperlink and/or other form of browser-executable code. Particularly, specification at [0034] “https://www.automl.org/automl/.” Applicant is required to delete the embedded hyperlink and/or other form of browser-executable code; references to websites should be limited to the top-level domain name without any prefix such as http:// or other browser-executable code. See MPEP § 608.01.
Claim Objections
Claims 1, 6 and 15 are objected to because of the following informalities: claim 1 should use semicolons following each of the three limitations similar to independent claims 10 and 20 so as to improve readability as a matter of form. Claims 6 and 15 term “under utilized” should be hyphenated. Appropriate correction is requested.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 1-19 are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention. Particularly, claims 1 and 10 recite the limitation “the artificial intelligence or machine learning model” which lacks antecedent basis and should be “an artificial intelligence or machine learning model” similar to claim 20. Claims 2-9 and 11-19 depend from claims 1 and 10 without remedying the issue. Additionally, claims 7-8 and 16-17 recite limitation “the model” which could be either of the AI/ML model or the GCM model. Thus, it is uncertain which model is being referred to. Accordingly, claims 1-19 are rejected as lacking antecedent basis and indefinite under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claim is directed to non-statutory subject matter. Claim 20 is drawn to a machine-readable storage medium but fails to establish the medium as non-transitory so as to provide physical embodiment. As per MPEP 2106.03(II) “the BRI of machine readable media can encompass non-statutory forms of signal transmission, such as a propagating electrical or electromagnetic signal per se… a computer readable medium that can be a compact disc or a carrier wave covers a non-statutory embodiment and therefore should be rejected under 35 USC 101” Turning to the instant specification, per [0126] “any other medium… medium such as space or an atmosphere as an acoustic signal” non-limiting language does not narrowly define the medium to avoid or otherwise disavow the medium to comprise non-statutory subject matter. In view of the foregoing, claim 20 is rejected as being directed to non-statutory subject matter under 35 U.S.C. 101. The issue may be remedied readily by specifying the machine-readable medium as non-transitory.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 10-11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over:
Zhu et al., “ROLLER: Fast and Efficient Tensor Compilation for Deep Learning” hereinafter Zhu (Univ. of Toronto), in view of
Fu et al., “GPU Domain Specialization via Composable On-Package Architecture” hereinafter Fu (NVIDIA).
With respect to claim 1, Zhu teaches:
A system for efficiently executing an artificial intelligence or machine learning model (model) {Zhu Fig 2 “System overview of ROLLER” illustrates DNN models with ONNX, further [P.238 Sect3.3 ¶2] “An execution unit is abstracted as an rTile Execution Unit (TEU), which computes the data tiles through the Compute interface”} comprising
a composer for generating a General Chip Model (GCM) {Zhu [P.234 Sect.1 ¶3-4] “composed by three interfaces: Load, Store and Compute, acted against rTile… rTile can be derived analytically from hardware” hardware i.e. chips comprise “execution unit of the accelerator (e.g., an SM, a streaming multi-processor in a NVIDIA GPU” illustrated Fig 2 hardware abstraction layer with micro-performance model detailed Alg.1 Line17 of Fig 8, Fig 9, described [P.238 ¶1, Last¶]. The micro-performance model is model for performance of the chip noted above. See also Fig 7 ROLLER computation model} and
a compiler for compiling the artificial intelligence or machine learning model for execution by a {Zhu [P.235 Sect.3 ¶1] “DNN compiler” for the DNN models Fig 2, introduced [P.233 ¶1] “DNN model are abstracted as operators and implemented as kernels, executed on modern accelerators”, compiler described [P.239 ¶2] “ROLLER’s compilation pipeline is as follows. Its input is an ONNX graph… and generates corresponding rProgram by ROLLER’s construction algorithm” whereby [P.234 ¶3] “rProgram that saturates a single execution unit of the accelerator (e.g., an SM, a streaming multi-processor in a NVIDIA GPU)” is execution by processor, the processor’s architecture [P.239 Last2¶] “GPUs with different architectures on the streaming multi-processors” and composable is suggested as scaling [P.237 ¶4 – P.238 ¶2] the scaling leverages parallelism through replication and enlarging tiles. Zhu also considers Graphcore-IPU architecture [P.240 ¶3]} and
generating a compiled program for execution on a processor having the {Zhu see [P.235 Sect.3 ¶1] “generate efficient tensor programs (named rProgram)… generating rProgram” simply rProg() Alg.1 Lines4-5 of Fig 8, also Figs 7a, 6, 2, compiled per [P.239 ¶2] “ROLLER’s compilation pipeline is as follows. Its input is an ONNX graph… and generates corresponding rProgram by ROLLER’s construction algorithm” whereby [P.234 ¶3] “rProgram that saturates a single execution unit of the accelerator (e.g., an SM, a streaming multi-processor in a NVIDIA GPU)” is execution by processor, the processor’s architecture [P.239 Last2¶] “GPUs with different architectures on the streaming multi-processors” and composable is suggested as scaling [P.237 ¶4 – P.238 ¶2] the scaling leverages parallelism through replication and enlarging tiles. Zhu also considers Graphcore-IPU architecture [P.240 ¶3]}.
However, Zhu does not explicitly state composable processor architecture which is disclosed by Fu. Particularly, Fu [P.3 ¶3] “we propose a Composable On-PAckage GPU (COPA-GPU) architecture” shown Figs 1, 5-7 and described e.g. [P.11 ¶3] “SRAM-based COPA-GPU” also compared to Groq TSP, Graphcore IPU, Nvidia V100 GPU, and Cerebras WSE-2 [P.4 Tbl.1].
Fu is directed to composing chip hardware for deep learning thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to use the composable copa-gpu of Fu in combination as simple substitution of known processor architectures for another to obtain predictable results and/or for a motivation being [P.7 ¶3,2] “primary goals: (1) to largely preserve the existing GPU architecture to minimize design effort and maximize reuse, and (2) to provide flexibility in specializing the GPU memory system for diverging requirements across application domains. Specifically, we aim to improve the GPU’s memory… to unlock deep learning performance on GPUs” thus “benefits of a composable GPU architecture that can satisfy the unique demands of each domain while maximizing design reuse” see contributions [P.3 Last2¶].
With respect to claim 2, the combination of Zhu and Fu teaches the system of claim 1, wherein
the composer comprises a hardware composer {Fu [P.3 ¶3] “Composable On-Package GPU (COPA-GPU)” detailed [P.7-10 Sect.3] and illustrated Figs 1, 5-7}.
A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to use the hardware composer of Fu in combination with Zhu to arrive at the invention as claimed for a motivation similar to that of claim 1 with further benefit [P.9 ¶3] “COPA-GPU has the advantage that offloading of GPM area allocated to MC and HBM I/O frees up die area for the implementation of UHB links and additional compute resources” and also [P.18 ¶3] “COPA-GPUs will provide substantially better cost performance at scale.”
With respect to claim 10, the rejection of claim 1 is incorporated. The difference in scope being a method to perform limitations of system claim 1. Zhu discloses [P.234 ¶6] “the computation process is also a data processing pipeline” similar [P.235 ¶3] with algorithmic steps Fig 8. The remainder of this claim is rejected for the same rationale as claim 1.
With respect to claim 11, the combination of Zhu and Fu teaches the method of claim 10 and further teaches limitation similar to claim 2. The hardware composer of Fu’s COPA-GPU being coupled for generating by a GCM may be considered GPM shown Fig 6 [P.9] COPA-GPU GPM with silicon interposer. Motivation of combination is applied similar to rationale for rejection of claim 2.
With respect to claim 20, the rejection of claim 1 is incorporated. The difference in scope being a machine-readable storage medium comprising instructions executable by processor to perform operations of method claim 1. Zhu discloses memory Fig 1 and throughout, github open-source code is provided [P.234 Ft.Nt.] as well as algorithmic code Fig 8, processor implementation and experimental setup is disclosed e.g. [P.239 Last2¶ - P.240 Last2¶]. The remainder of this claim is rejected for the same rationale as claim 1.
Claims 3-4 and 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu and Fu in view of Zhang et al., “Towards Automatic and Agile AI/ML Accelerator Design with End-to-End Synthesis” hereinafter Zhang.
With respect to claim 3, the combination of Zhu and Fu teaches the system of claim 2. Zhang teaches wherein
the hardware composer generates an Operation Information Table for use by the compiler when compiling a model {Zhang [P.222 ¶2,4] “generate a custom TableGen file… TableGen file is then used at compilation” for “integer operations… arithmetic operations” implemented Figs 4-5, and Table 3 gives results for known models, a hardware composer is “hardware synthesis” [P.221 ¶1-3], [P.223 ¶3]}.
Zhang is directed to compilers for AI/ML accelerator design thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to generate tables using TableGen, a known file format extensively used in LLVM, per Zhang in combination as applying a known technique, being a standard file format, to a known device ready for improvement to yield predictable results and/or motivation [P.222 ¶2] “exploits modularity and support for importing external components…then integrate the specialized functional units” importing library of information to characterize operations to be integrated in a modular manner, i.e. [P.221 ¶2] “provides natural modularity and composability.”
With respect to claim 4, the combination of Zhu, Fu and Zhang teaches the system of claim 3, wherein
the Operation Information Table represents operational characteristics of a functional unit {Zhang [P.222 ¶2,4] “TableGen… integrate the specialized functional units” [P.221 Last2¶]}.
A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to integrate functional units with TableGen per Zhang in combination for a motivation being [P.221 Last2¶] “functional units also need to be characterized, at least in terms of performance (e.g. latency of the critical path) and area of the target technology of choice… we can further optimize the circuit design, applying techniques like chaining of functional units” and/or [P.222 ¶4] “speculative functional units… speculate on the result of the arithmetic operations (for example, not considering the carry bits in additions), thus potentially allowing to reach higher degrees of parallelism and higher frequencies (or reduced power).”
With respect to claim 12, the combination of Zhu and Fu teaches the method of claim 11, and further combination with Zhang teaches the limitation of claim 3. Therefore, the rejection of claim 3 with equal motivation is applied to claim 12.
With respect to claim 13, the combination of Zhu, Fu and Zhang teaches the method of claim 12, and further teaches the limitation of claim 4. Therefore, the rejection of claim 4 with equal motivation is applied to claim 13.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu, Fu and Zhang in view of Jin et al., “Compiling ONNX Neural Network Models Using MLIR” hereinafter Jin (arXiv: 2008.08272v2)
With respect to claim 5, the combination of Zhu, Fu and Zhang teaches the system of claim 4, wherein
the Operation Information Table comprises cost, {Zhang [P.219 Last¶] “cost, size and cooling requirements” listed among metrics for soda-compiler, the metrics regarding [P.222 ¶2] “TableGen file containing all the relevant metrics”, and prior to silicon is interpreted as a [P.219 ¶3] “pre-optimized LLVM IR” for the silicon-based accelerator introduced [Abst]}.
However, Zhang does not explicitly disclose skew which is disclosed by Jin.
Particularly, Jin at [P.6 Rt.Col] “skew” described for schedules that are composable and Listing 6 at top of page details TableGen definitions, such that “If users want to define custom declaration in the class, it can be done via the ‘extraClassDeclaration field (Line 7)”
Jin is directed to neural compilers using TableGen thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to include skew using the custom class declaration in TableGen per Jin in combination with Zhang to arrive at the invention as claimed because known work in one field may prompt variations of it in the same field based on design incentives if the variations are predictable the skilled artisan. Here, the skilled artisan is given a tool for customizing information to be included in TableGen files and gives a design incentive that points to composable schedules [P.6 Last2¶] and further notes the ability to carry out optimizations over transformations including skew [P.4 ¶3], [P.5 ¶2].
With respect to claim 14, the combination of Zhu, Fu and Zhang teaches the method of claim 13, and further combination with Jin teaches the limitation of claim 5. Therefore, the rejection of claim 5 with equal motivation is applied to claim 14.
Claims 6, 9, 15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu and Fu in view of Li et al., “Compiler-Driven Simulation of Reconfigurable Hardware Accelerators” hereinafter Li (arXiv: 2202.00739v1).
With respect to claim 6, the combination of Zhu and Fu teaches the system of claim 2. Li teaches wherein
the hardware composer generates a processor architecture by selectively adding additional resources to the processor architecture or reducing selected resources that are under utilized when a selected model is being compiled by the compiler {Li [P.2 SectII.A] “define the hardware resources that make up an accelerator” by “instantiate components like processing elements (PEs) and memories… declare components including processors, memories” via “add_comp to add” or [P.12 ¶1] “reduce the 16 processors to 4” see Fig 2 showing SRAM and PE structure declarations with EQueue program, further employs AI engine for deep learning computations [P.2 ¶2], [P.11-12] code ai_eng, and generates systolic array architecture using MLIR compiler API [P.7 Sect.B], and discloses [P.5 SectV.B] “avoid waste or increase computation utilization”}.
Li is directed to compilers for hardware accelerators thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify hardware resources according to the teachings of Li in combination to arrive at the invention as claimed for a motivation [P.8 ¶2] “Benefits. EQueue programs can modularize hardware components (e.g., SRAM interfaces and processors) and thereby study the individual effect of a components… allows a programmer to concentrate on architecture design” e.g. [P.2 ¶2-3] “designers can easily switch among different architectures for the same computation… EQueue’s flexibility can guide designers to improve their designs on a real-world reconfigurable architecture.”
With respect to claim 9, the combination of Zhu, Fu and Li teaches the system of claim 6, wherein
the hardware composer generates a processor architecture selected from a library {Li [P.6 ¶2] “library of components, such as SRAM memories and processors… users can specify arbitrary behavior for components in EQueue programs” for [P.7 ¶2] “generator that emits EQueue code to model systolic array architectures” similar at [P.1 ¶3] “generated architectures”}. Motivation for combination is applied similarly as in claim 6.
With respect to claim 15, the combination of Zhu and Fu teaches the method of claim 11, and further combination with Li teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 15.
With respect to claim 18, the combination of Zhu and Fu teaches the method of claim 11, and further combination with Li teaches the limitation of claim 9. Therefore, the rejection of claim 9 with equal motivation is applied to claim 18.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu, Fu and Li in view of Yang et al., “Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks” hereinafter Yang (arXiv: 2002.04116v1).
With respect to claim 7, the combination of Zhu, Fu and Li teaches the system of claim 6. Yang teaches wherein
the hardware composer generates a processor architecture for each layer of the model {Yang Fig 2 accelerator synthesis for NN layers, described [P.3 Sect.III Rt.Col] “we map network layers to sub-accelerators… A map function map(li,j) = aick is defined, which indicates the jth network layer li,j in the ith DNN Di to be mapped to the kth sub-accelerator aick. Based on the mapping, we determine the execution order of the layers on sub-accelerator aick following a schedule function sch(aick) …mapping and scheduling of each layer”}.
Yang is directed to co-exploration of neural networks with hardware design thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify layers mapped to processors per Yang to arrive at the invention as claimed for motivation [P.4 ¶5,8] “explore the best hardware for a previous identified neural architecture …for mapping and scheduling, we need to obtain the latency and energy of each layer” in other words the hardware performance (latency, energy) is based on DNN layer-by-layer evaluation.
With respect to claim 8, the combination of Zhu, Fu and Li teaches the system of claim 6, wherein
the processor architecture for each layer of the model is manufactured as a semiconductor processor for executing the model {Yang Fig 2 Synthesis yielding Resultant Accelerator with NN layers where synthesis corresponds to manufacture, the accelerator or sub-accelerator includes ASIC as a semiconductor for processing and DNN layers are mapped for scheduled execution, see [P.2-3 Sect.III], also Figs 3, 5}.
A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to synthesize per Yang to arrive at the invention as claimed for a motivation being [P.2 ¶4-5] “ASIC designs grant the maximum flexibility to designers to determine the hardware organization… enables the co-exploration of neural architectures and ASIC designs by incorporating hardware allocation” and such that the neural architectures of DNN layers provides resultant accelerator based on maximum accuracy of DNNs with corresponding hardware constraints e.g. latency and energy [P.3 ¶9].
Claims 16-17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu and Fu in view of Yang.
With respect to claim 16, the combination of Zhu and Fu teaches the method of claim 11, and further combination with Yang teaches the limitation of claim 7. Therefore, the rejection of claim 7 with equal motivation is applied to claim 16.
With respect to claim 17, the combination of Zhu and Fu teaches the method of claim 11, and further combination with Yang teaches the limitation of claim 8. Therefore, the rejection of claim 8 with equal motivation is applied to claim 17.
With respect to claim 19, the combination of Zhu, Fu and Zhang teaches the method of claim 11. Yang teaches:
the hardware composer generates a plurality of processor architectures where each processor architecture is adapted to executing a layer of a model {Yang Fig 2 hardware synthesis with NN layers, [P.3 Sect.III Rt.Col] “we map network layers to sub-accelerators and determine their execution orders” emphasis execution for DNN network layers, processor architectures comprising ASIC accelerators}. Motivation for combination is applied similar to claims 7-8.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Lin et al., “NAAS: Neural Accelerator Architecture Search” arXiv:2105.13258v1 Figs 1-2 discloses compiler mappings for co-design with joint search space
Ahn et al., “Glimpse: Mathematical Embedding of Hardware Specification for Neural Compilation” Figs 1-2, Alg. 1 vectorization of target hardware features, DNN compiler
Ghodrati et al., “Planaria: Dynamic Architecture Fusion for Spatial Multi-Tenant Acceleration of Deep Neural Networks” discloses layer-wise compiler with architecture awareness
Sharma et al., “Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks” arXiv: 1712.01507v2 matches accelerator arch. to layer bitwidth
Lie, Sean “Multi-Million Core Multi-Wafer AI Cluster” Cerebras WSE-2 w/ 40GB SRAM
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chase P Hinckley whose telephone number is (571)272-7935. The examiner can normally be reached M-F 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda M. Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHASE P. HINCKLEY/Examiner, Art Unit 2124
Prosecution Insights