DETAILED ACTION
Claims 1, 3, 5, 8, 10 and 12 have been amended.
Claims 2 and 9 have been cancelled.
Claims 1, 3 – 8 and 10 – 14 have been examined and are pending.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3, 8 and 10 are rejected under 35 U.S.C. 102(a)(1) as being unpatentable by US Patent Application US Patent Application Publication No. 2022/0343137 to Surendran et al. (hereinafter Surendran).
Claim 1, Surendran discloses (Fig. 3B) techniques to automatically generate a reduced number of compute kernels for performing operations of one or more neural networks, and further it discloses:
method comprising:
obtaining a logical netlist (Surendran discloses (Fig. 3A and ¶75) determining a neural network graph) comprising a plurality of nodes (Surendran discloses (¶79) one or more graph nodes) and node connections representing a network (Surendran discloses (¶135) nodes are connected by edges, representing tensors – multidimensional data arrays flowing between operations – the input/output tensors represent the data exchanged between nodes and thereby define the connectivity among graph nodes).
storing a library of compute kernels (Surendran discloses (¶73 and ¶80) generating optimized or reduced number of kernels from predetermined arithmetic and memory operations set of software kernel modules in memory code data storage 101 – this stored collection of kernel modules (i.e. 5 - 10 reduced down to 1, 2, 3 or 4) functionally and inherently meets the library limitation), each compute kernel configured to implement an operation (Surendran discloses (¶73) compute kernels of a neural network are used to perform functions).
storing a kernel definition associated with each of the compute kernels, the kernel definition mapping (i) a physical input source to a logical input of the compute kernel and (ii) a logical output of the compute kernel to a physical output (Surendran discloses (¶56 - ¶59) each kernel module is defined by code specifying how physical memory resources (global memory, ALUs, activation storage) provide inputs and receive outputs – thereby mapping physical sources to logical inputs/outputs of each kernel. Surendran discloses (¶73) compute kernels perform functions of various cells of the neural network – requiring loads of input tensors from global memory and repeated writes to global memory. Code and/or Data Storage 101 is to store forward and/or output weight and input/output data, control timing and/or order in which weight or other parameter information is to be loaded to configure logic.)
determining a set of kernel configurations (Surendran explicitly discloses (¶Fig. 3B) receiving a set of operations and automatically adjusting (¶564, ¶604) them to generate a reduced number of software kernel modules (i.e., multiple kernel configurations), each optimized according to performance or resource criteria (¶77). It further describes generating first, second and third software code (¶86) for different operation groups, which are compiled to produce an optimized number of compute kernels) each kernel configuration in the set satisfying a compilation condition (Surendran discloses (¶74) selecting a reduced number of compute kernels for optimizing a set of operations of neural network graph nodes based on satisfying adjustment criteria), and each kernel configuration in the set comprising (a) a subset of the compute kernels, such that each node is covered by the operation of at least one compute kernel (Surendran discloses (Fig. 3B and ¶80-¶86) compute kernels implementing algorithms, and a subset of said optimized and adjusted set of operations that can be executed before/after said core operation associated with one or more graph nodes), and (b) links between the compute kernels in the subset, wherein the links are defined based on the kernel definitions (Surendran discloses (¶56 - ¶59) the compiler selects a subset of kernels covering all graph nodes (each node’s operation is included in at least one kernel). The preceding/following operation linkage (¶86) defines links between kernels, derived from each kernel’s defined input/output relationships (its ‘kernel definitions’).
selecting one kernel configuration from the set (Surendran explicitly discloses (¶86) generate optimized operations compiled as the second, first and third batch of software code, and then a single compute kernel (¶87) is generated by compiling optimized versions batches of code and/or replacements of operations)
compiling the selected one kernel configuration to implement the network (Surendran explicitly discloses (¶72) performing compilation of the selected kernel configuration – producing executable compute kernels that implement the neural-network graph. A compute kernel includes a routine compiled for high throughput accelerators, and a reduced number of compute kernels are generated by compiling software code (¶73) corresponding to adjusted set of operations of neural network graph nodes).
Claim 3, Surendran discloses all the elements of claim 1. Further, it discloses:
wherein the compilation condition comprises a legality condition (Surendran explicitly discloses (¶74 and Fig. 3A) satisfying one or more adjustment criteria. The adjustment criteria correspond to legality conditions (e.g., data-dependency and shape compatibility checks) that must be legally satisfied before fusion or compilation to produce correct kernels.)
Claim 8, do not teach or further define over the limitations in Claim 1. Therefore, claim 8 is rejected for the same rationale of rejection as set forth in Claim 1.
Claim 10, do not teach or further define over the limitations in Claim 3. Therefore, claim 10 is rejected for the same rationale of rejection as set forth in Claim 3.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 – 7 and 11 - 14 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application US Patent Application Publication No. 2022/0343137 to Surendran and in view of US Patent Application Publication No. 2017/0147387 to Allen.
Claim 4, Surendran discloses all limitations of Claim 3 including explicitly teaching (¶74 and Fig. 3A) one or more adjustment criteria. The adjustment criteria correspond to legality conditions kernel configurations from a neural-network graph. However, Surendran does not explicitly disclose wherein the legality condition comprises one or more of: verifying that the kernel configuration is fully connected, verifying that the kernel configuration does not contain dangling edges, and verifying whether the physical output of a source compute kernel matches the physical input source of a connected destination compute kernel. However, in an analogous art, Allen teaches:
wherein the legality condition comprises one or more of: verifying that the kernel configuration is fully connected, verifying that the kernel configuration does not contain dangling edges, and verifying whether the physical output of a source compute kernel matches the physical input source of a connected destination compute kernel (Allen explicitly teaches (¶17) verifying actual pointers in the memory snapshot correspond to edges … validate that the edge is a valid edge, ensuring all reachable nodes are connected. Allen removes (¶48) nodes and/or edges corresponding to invalid traversal and edge which have illegal memory addresses i.e. dangling connections. Allen validates that pointer address and transformed pointer address are legal (¶47) and data fields appropriately memory-aligned i.e. correct correspondence of ports.)
It would have been obvious as of the effective filing date to one of ordinary skill in the art to combine the compilation condition comprises a legality condition, as disclosed by Surendran, and wherein the legality condition comprises one or more of: verifying that the kernel configuration is fully connected, verifying that the kernel configuration does not contain dangling edges, and verifying whether the physical output of a source compute kernel matches the physical input source of a connected destination compute kernel, as taught by Allen, for the purpose of (¶14) enabling a hypervisor to locate and identify kernel data structures associated with one or more virtual machine (VM) instances on a virtualized computer system using a combination of static analysis and dynamic analysis techniques.
Claim 5, Surendran discloses all the elements of claim 1. Further, it discloses: wherein selecting the one kernel configuration from the set of kernel configurations comprises:
selecting an optimization target, and selecting the kernel configuration optimizing the optimization target (Surendran explicitly discloses (¶73 and Fig. 3A) that a compiler generates multiple candidate kernel configurations and selects an optimized configuration for compilation satisfying one or more adjustment criteria (¶74). The adjustment criteria correspond to legality conditions (e.g., data-dependency and shape compatibility checks) that must be satisfied before fusion or compilation to produce correct kernels)
evaluating each kernel configuration against the optimization target (Allen explicitly teaches kernel data structure is evaluated (¶41) and verifying that a graph configuration is valid (no invalid edges/dangling nodes) before further processing (¶48). Allen, thus ensures that only legal kernel configurations are subject to any subsequent evaluation or optimization.)
The motivation to combine the references is similar to the reasons in Claim 4.
Claim 6, Surendran in view of Allen discloses all the elements of claim 5. Further, it discloses:
wherein the optimization target includes an optimization metric comprising one or more of: power consumed by the selected subset of compute kernels, latency of the selected subset of compute kernels, throughput of the selected subset of computed kernels, and area of the selected subset of compute kernels (Surendran explicitly discloses (¶73) reducing CPU overhead (i.e. reducing memory transactions) by reducing the number of kernels used to implement the neural network graphs. Surendran explicitly discloses (¶73) groupings of operations and computing resources (¶173-¶175) to increase computational efficiency and reduce execution time. Thus, Surendran discloses (Fig. 8) the compiler generating multiple configurations and compiling an optimized number of compute kernels which inherently reduces latency and power consumption by reducing the kernel count, minimizing data movement and memory access.)
The motivation to combine the references is similar to the reasons in Claim 5.
Claim 7, Surendran in view of Allen discloses all the elements of claim 6. Further, it discloses:
wherein the optimization target includes an optimization objective for each optimization metric, the optimization objective comprising one or more of: a minimization objective, a maximization objective, and a target threshold (Surendran explicitly discloses the compiler perform (¶73) reducing CPU overhead i.e. reducing memory transactions and improve performance of executing neural network graphs, where ‘reduce’ = minimization objective (power/latency). Also, the compiler determines the groupings of operations … to increase computational efficiency and reduce execution time, where ‘increase’ = maximization objective (throughput/efficiency). The grouping is selected to satisfy performance criteria, where ‘satisfy criteria’ = meeting target threshold objective.)
The motivation to combine the references is similar to the reasons in Claim 6.
Claim 11, do not teach or further define over the limitations in Claim 4. Therefore, claim 11 is rejected for the same rationale of rejection as set forth in Claim 4.
Claim 12, do not teach or further define over the limitations in Claim 5. Therefore, claim 12 is rejected for the same rationale of rejection as set forth in Claim 5.
Claim 13, do not teach or further define over the limitations in Claim 6. Therefore, claim 13 is rejected for the same rationale of rejection as set forth in Claim 6.
Claim 14, do not teach or further define over the limitations in Claim 7. Therefore, claim 14 is rejected for the same rationale of rejection as set forth in Claim 7.
Response to Arguments
Claim Rejections - 35 USC § 102
Applicant’s arguments and amendments, filed on 01/16/2026 with respect to the Claims 1, 3 – 8 and 10 – 14 have been fully considered and they are not persuasive. Hence, the 35 USC § 102 rejection is maintained.
In response to the applicant’s argument, (Pg. 7), “… the first, second and third batches of software code cover different portions of the operation, rather than being alternative options, each of which are capable of implementing the optimized operations as a whole … clearly then, the first, second, and third batches of software code of Surendran cannot be said to be analogous to the set of kernel configurations as recited in presently amended claims,” the Examiner notes Surendran clearly discloses analogous kernel configurations (in ¶73) and notes that optimized kernels for deep neural networks can be generated by automatically adjusting operations associated with a cell in said deep neural network to new operations that achieve a same result but in a reduced number of kernels (e.g., in a single kernel), thus improving performance and minimizing round trips for data transfer between a deep neural network and dynamic random-access memory (DRAM) or other memory. An automated compiler flow and/or a neural network graph can be updated to generate an optimized and efficient single kernel to achieve a same output but that can be performed on a same kernel. A reduced number of compute kernels (e.g., a single compute kernel) can be generated by compiling software code corresponding to prolog, software code corresponding to said core operation, and software code corresponding to epilog after operations of neural network graph nodes have been updated. Surendran’s paragraph [0086] describes generating multiple code batches, a prolog batch, a core operation batch, and an epilog batch, which are compiled into a reduced number of compute kernels. These batches are distinct portions of the optimized operations and, when compiled, form a configuration of kernels that covers all nodes of the graph. Surendran’s optimization process ([0077]–[0086]) inherently produces multiple possible kernel arrangements before final compilation. The first, second, and third code batches in [0086] represent different subsets of operations that can be combined in various ways depending on adjustment criteria, optimization choices, and resource constraints.
Further, the cited disclosure ([0077], [0080]–[0086]) describes a process in which the “processing logic” loops through graph nodes, applies various adjustment criteria (e.g., GEMV/GEMM replacement, sum reduction optimization, slice operation repositioning), and compiles an optimized set of operations into compute kernels.
Because these criteria are applied selectively, depending on node type, sequence, batch size, and constraints, the optimization process necessarily produces different possible arrangements of kernels for the same neural network graph.
Surendran discloses “adjustment criteria” (¶[0077]) are applied per node or sequence of nodes.
For each node, the process determines whether the criterion is satisfied, and if so, performs adjustments.
If not, it leaves the node unchanged. Given that different nodes may satisfy different criteria, and criteria may be applied in different combinations, the process inherently generates a set of candidate kernel configurations before selecting the final one.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HASSAN ABDUR-RAHMAN KHAN whose telephone number is (313)446-6574. The examiner can normally be reached TEAPP - (M-Sa) 9/30/17-9/30/18, 6am-10pm IFP.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Parry can be reached at (571) 272-8328. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H. A. K./
Examiner, Art Unit 2451
/Chris Parry/Supervisory Patent Examiner, Art Unit 2451
Prosecution Insights