DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to pending claims 1-20 filed 1/5/2024.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application CN202110772340.6 filed in China on 7/8/2021.1
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 12, 15, 16, 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The 35 U.S.C. 101 subject matter eligibility analysis first asks whether the claim is directed to one of the four statutory categories (Step 1). It next asks whether the claim is directed to an abstract idea (Step 2A), via Prong 1, whether an abstract idea (e.g., mathematical concept, mental process, certain methods of organizing human activity) is recited, and Prong 2, whether it is integrated into a practical application. It finally asks whether the claim as a whole includes additional elements that amount to significantly more than the judicial exception (Step 2B). See MPEP 2106.
STEP 1: The claims falls within one of the four statutory categories:
All claims are directed to methods and hence fall within one of the four statutory categories.
STEP 2A PRONG 1: The claims recite a judicial exception:
Claim 12 recites a method for receiving instructions and data, parsing instructions to acquire operations, and sequentially performing operations based on the instructions. As such, it is directed to a mental process, e.g., a multi-step algorithm to be performed on data that may be performed in the mind or with the aid of pen and paper (e.g., long division).
In particular: For claim 12: An acceleration method for a neural network computation (methods directed to neural network computation, including those that lead to increased efficiency, may be performed in the mind), comprising:
receiving an instruction sequence predetermined based on a size of a memory module (considering operations related to a memory module size may be performed in the mind) and data required for the neural network computation, and parsing the instruction sequence to acquire multiple types of operation instructions (receiving data, parsing instruction sequences to obtain instruction parts may be performed in the mind); and
sequentially performing corresponding operations for the neural network computation based on the multiple types of operation instructions (carrying out steps may be performed mentally, including operations associated with neural network computation).
For claim 15: The acceleration method according to claim 12,further comprising:
generating an end-of-execution tag after execution of an operation instruction of a corresponding type is completed (Generating tags, associations for data may be performed in the mind, including generation after operating instruction is completed); and
parsing the instruction sequence to acquire a dependency relationship between the plurality of functional modules, and generating operation instructions of corresponding types in an order based on the dependency relationship and the end-of-execution tag (planning out operations and generating instructions based on module dependency and end-of-execution associations may be performed mentally).
For claim 16: The acceleration method according to claim 12, further comprising:
controlling a working state corresponding to each type of operation instruction, the working state comprising at least an on state and an off state (associating a working state, including on or off states, for operations may be performed mentally).
For claim 20: The acceleration method according to claim 12, further comprising:
disassembling the neural network computation into a plurality of sub-computations based on the size of the memory module and the data required for the neural network computation (disassembling computations may be performed mentally, including computations based on consideration of memory size, data requirements);
determining the instruction sequence based on respective data required for the plurality of sub-computations resulting from the disassembling and a dependency relationship between the plurality of sub-computations (making determinations based on said considerations may be performed mentally);
parsing the instruction sequence to acquire operation instructions corresponding to the plurality of sub-computations (parsing or dividing instruction sequences may be performed mentally); and
generating the multiple types of operation instructions in an order based on the dependency relationship between the plurality of sub-computations (generating instructions based on considerations of dependency relationships may be performed mentally).
STEP 2A PRONG 2: The claims do not integrate the exception into a practical application:
There are no additional elements to consider for this step.
STEP 2B: The claim as a whole do not include additional elements that amount to significantly more than the abstract idea:
There are no additional elements to consider for this step.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7, 9-18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (US 11762631 B2) in view of Venkatesan ("Magnet: A modular accelerator generator for neural networks", published 2019).
Fo claim 1, Chen discloses: a hardware acceleration apparatus for a neural network computation, comprising a memory module, a parsing module and a plurality of functional modules, wherein
the parsing module (fig.1A:615, 0087, 0090: instructions including operation, address, computation topology is parsed for dispatch to additional units by the controller unit) is electrically connected to each of the functional modules and is configured to receive an instruction sequence (figs.1A, 1D shows electronic interconnection of the various components) predetermined based on data required for the neural network computation (fig.1C shows typical instruction sequences, the instructions being based on the various data needed), parse the instruction sequence to acquire multiple types of operation instructions, and issue to each functional module an operation instruction of a corresponding type among the multiple types of operation instructions (ibid: instruction set is parsed for each operation instruction and dispatched to relevant functional modules);
each of the functional modules is electrically connected to the memory module and the parsing module and is configured to perform a corresponding operation for the neural network computation in response to receiving the operation instruction of the corresponding type (figs.1A:613, 614, fig.1D:4-6, fig.1E-G contemplate various modules, connected to memory and configured to receive instructions from the parsing modules for performing operations); and
the memory module is electrically connected to each of the functional modules and is configured to cache the data required for the neural network computation (fig.1A:611, fig.1D:3; fig.1F:3, 52, 53; fig.1G: 3, 63, 64, contemplate various connections of the memory module for caching operations).
Chen does not disclose: wherein the based on is based on a size of the memory module.
Venkatesan disclose: wherein the based on is based on a size of the memory module (§V: mapper for generating mappings from task to instruction set, the mapping determined based on task attributes as well as efficiency and performance, see §V.A, §V.B).
It would have been obvious before the effective filing date to one of ordinary skill in the art to modify the apparatus of Chen by incorporating the mapping technique of Venkatesa. Both concern the art of neural network hardware accelerators, and the incorporation would have, according to Venkatesan, allow optimization of execution according to performance requirements (§V.B).
For claim 2, Chen modified by Venkatesan discloses the apparatus of claim 1, as described above. Chen further discloses: wherein the multiple types of operation instructions at least comprise a loading instruction, a computation instruction, and a storage instruction (various matrix convolution and matric multiplication instructions such as those in 0109, 0119, 0126 would all constitute loading, computation instructions; 0134 storage instructions); and the plurality of functional modules comprise:
a loading module electrically connected to an external storage, the memory module and the parsing module and configured to load the data required for the neural network computation from the external storage into the memory module in response to the loading instruction issued by the parsing module (figs. 1A:616, 1D:3, F:3, G:3, 0173: loading module for loading from external storage, e.g., fig.1A:611, 0089, for reading and writing to memory module via data access unit, hence, loading module connected to external storage in response to parsed commands) , wherein the data required for the neural network computation comprises parameter data and feature map data (fig.1C:Compute contemplates various data for loading including parameter data feature map data such as input data and parameter data such has kernel data);
a computation module electrically connected to the memory module and the parsing module and configured to perform, in response to the computation instruction issued by the parsing module, computation by reading the parameter data and the feature map data from the memory module and to return a computation result to the memory module (fig.1D:5, fig.1F:511-512, fig.1G:611-612: computation modules for performing computation when reading parameter and feature map data; see 0179, 0183-184: returning output to memory module); and
a storage module electrically connected to the external storage, the memory module and the parsing module and configured to store the computation result from the memory module back into the external storage in response to the storage instruction issued by the parsing module (fig.1H:S10, 0194-195; see also 0134: Matrix Store instruction stores matrix from scratchpad memory to external memory, hence, storage module for storing of computation result, see 0109, 0119, 0126: use of scratchpad memory for various matrix operations).
For claim 3, Chen modified by Venkatesan discloses the apparatus of claim 2, as described above. Chen further discloses: wherein the computation instruction comprises a first computation instruction and a second computation instruction, and the computation module comprises:
at least one first computation unit that comprises a plurality of multiply-accumulate units and is configured to receive the first computation instruction issued by the parsing module (fig.1G, corresponding to odes of fig.1E, fig.1D:4-6, the MAC units receiving instructions parsed by the controller unit according to topology created by the interconnect module, see 0087, 0092), perform convolutional computation and/or matrix multiplication computation by reading the parameter data and the feature map data from the memory module according to the first computation instruction (0109, 0119, 0127, 0168-169 disclose matrix multiplication and convolution instructions, with fig.1G, 0181 disclosing computation unit; the operations performed according to instructions parsed by the control unit including loading of feature map and parameter data as describe above) to acquire an intermediate computation result, and return the intermediate computation result to the memory module (fig.1G: 63-64: intermediate computation results are returned to the memory module or scratchpad memory, see 0109, 0119, 0127, 0135, 0168-169); and
at least one second computation unit that comprises a plurality of arithmetic computation units and logic computation units and is configured to receive the second computation instruction issued by the parsing module, perform activation computation and/or pooling computation by reading the intermediate computation result from the memory module according to the second computation instruction to acquire the computation result (pooling: 110-116, activation: 0106, 0194, 0167-168 i.e., reading activation from intermediate result for previous layer in order activation for subsequent layers), and return the computation result to the memory module (fig.1G: 63-64: intermediate computation results are returned to the memory module or scratchpad memory, see 0109, 0119, 0127, 0135, 0168-169).
For claim 4, Chen modified by Venkatesan discloses the apparatus of claim 1, as described above. Chen further discloses: each of the functional modules is further configured to send an end-of-execution tag to the parsing module after execution of the operation instruction of the corresponding type is completed (fig.1H:S10, 0195 contemplates ending operation via reading the IO instruction to write to external memory, hence, IO write to external memory constitutes an end-of-execution tag, the IO command being parsed by the control module as described above after the various compute operations); and
the parsing module is further configured to parse the instruction sequence to acquire a dependency relationship between the plurality of functional modules, and issue to each of the functional modules the operation instruction of the corresponding type in an order based on the dependency relationship (0174: interconnect module specifying interconnect topology constitutes dependency relationship among modules; 0178, 0182: data dependency modules discloses data dependency for avoiding consistency conflicts) and the end-of-execution tag as received (fig.1H:S10, 0195: end of operation tag associated with IO write command is issued to the functional modules based on order of operations).
For claim 5, Chen modified by Venkatesan discloses the apparatus of claim 1, as described above. Chen modified by Venkatesan further discloses: a control module electrically connected to each of the functional modules and configured to control a working state of each of the functional modules in the hardware acceleration apparatus (Chen fig.1A:615, 0087, 0090: controller unit includes control module for controlling interconnect module, functional units, etc. as described above), the working state comprising at least an on state and an off state (Venkatesan §V.B contemplates activation and non-activation of processing elements based on performance tradeoffs, hence the modules having a on / off or utilized / non-utilized state).
For claim 6, Chen modified by Venkatesan discloses the apparatus of claim 2, as described above. Chen further discloses: a data management module electrically connected to the memory module and the computation module and configured to move data cached in the memory module to the computation module and move output data from the computation module to the memory module (fig.1F:63-64, 3; fig.1G:53, 3 contemplates cache memory for future computation or storing data from computation to cache, see also 0109, 0118-120, 0121, 0134 contemplating use of scratchpad memory, external memory; hence, data management module for performing said operations).
For claim 7, Chen modified by Venkatesan discloses the apparatus of claim 3, as described above. Chen further discloses: a data interaction module electrically connected to a plurality of the first computation units and configured to enable data interaction between the first computation units (figs.1A, D, F-G:4: interconnect module connected to computation units to instantiate computational topology among computation units).
For claim 9, Chen modified by Venkatesan discloses the apparatus of claim 1, as described above. Chen further discloses: the instruction sequence is predetermined by disassembling the neural network computation into a plurality of sub-computations (fig.1C shows an overall COMPUTE command, such as received by controller unit 615, the command being passed to interconnect module to form computation topology (0092, 0094) for transmission to operation modules fig.1D:5-6, 1G-H, hence, neural network computation is disassembled into sub computations for operation modules) based on the size of the memory module (Chen 0092: preparing processing element topology and dissembling computations based on topology size metrics, see also Venkatesan §V.B) and the data required for the neural network computation (0087, 0090: computation topology based on instruction, interconnect module) and determining the instruction sequence based on respective data required for the plurality of sub-computations resulting from the disassembling and a dependency relationship between the plurality of sub-computations (0174, 0178, 0182 instructions sequence are determined based on topology, sequence dependency to avoid consistency conflicts); and
the parsing module is further configured to parse the instruction sequence to acquire operation instructions corresponding to the plurality of sub-computations, and issue to each of the functional modules the operation instruction of the corresponding type in an order based on the dependency relationship between the plurality of sub-computations (figs.1F-G, 511-512, 611-612: operations are acquired from initial instructions and issued to function modules based on an order as determined by data dependency determinations figs.1F-G: 152, 62).
For claim 10, Chen modified by Venkatesan discloses the apparatus of claim 1, as described above. Chen modified by Venkatesan further discloses: wherein the instruction sequence is predetermined further based on a storage space utilization rate of the memory module and a requirement for computation bandwidth (Venkatesan §V.B contemplates processing utilization tradeoffs, hence, determining mapped instruction sequence based on storage space utilization (i.e., of various processing elements, see Chen figs.1F-G:52, 63-64), and computation bandwidth based on number of PE’s activated for a task, hence, higher performance bandwidth requirements).
For claim 11, Chen discloses the apparatus of claim 1, as described above. Chen further discloses: a storage object of each storage space in the memory module is adjustably configured according to the instruction sequence (fig.1F-G: 53, 63-64: storage unit in processing modules are adjusted and configured according to instructions determining computation topology (figs.1F-G:4), data determinacy (figs.1F-G:52, 62)).
For claim 12, Chen discloses: an acceleration method for a neural network computation, comprising:
receiving an instruction sequence predetermined based on data required for the neural network computation, and parsing the instruction sequence to acquire multiple types of operation instructions (fig.1A:615, 0087, 0090: instructions including operation, address, computation topology is parsed for dispatch to additional units by the controller unit, with fig.1C showing typical instruction sequences, the instructions being based on the various data needed); and
sequentially performing corresponding operations for the neural network computation based on the multiple types of operation instructions (figs.1A:613, 614, fig.1D:4-6, fig.1E-G contemplate various modules, connected to memory and configured to receive instructions from the parsing modules for performing operations, the operations performed according the sequence, e.g., determined by data dependency (fig.1F-G:5-6) and computation topology (0087, 0092)).
Chen does not disclose: wherein the based on is based on a size of the memory module.
Venkatesan disclose: wherein the based on is based on a size of the memory module (§V: mapper for generating mappings from task to instruction set, the mapping determined based on task attributes as well as efficiency and performance, see §V.A, §V.B).
It would have been obvious before the effective filing date to one of ordinary skill in the art to modify the apparatus of Chen by incorporating the mapping technique of Venkatesa. Both concern the art of neural network hardware accelerators, and the incorporation would have, according to Venkatesan, allow optimization of execution according to performance requirements (§V.B).
Claim 13-18, 20 recite methods analogous to the above apparatuses and are hence rejected for the same reasons.
Claim(s) 8, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (US 11762631 B2) in view of Venkatesan ("Magnet: A modular accelerator generator for neural networks", published 2019) in view of Uhrenholt (US 20210216455 A1).
For claim 8, Chen modified by Venkatesan discloses the apparatus of claim 2, as described above. Chen modified by Venkatesan does not disclose: the loading module is further configured to perform decompression processing on compressed data to be loaded into the memory module; and
the storage module is further configured to perform compression processing on uncompressed data read from the memory module to be stored in the external storage.
Uhrenholt discloses: the loading module is further configured to perform decompression processing on compressed data to be loaded into the memory module (0003: loading data and decompressing for operation); and
the storage module is further configured to perform compression processing on uncompressed data read from the memory module to be stored in the external storage (ibid: compressing data before writing to memory).
It would have been obvious before the effective filing date to one of ordinary skill in the art to modify the apparatus of Chen modified by Venkatesan by incorporating the data compression technique of Uhrenholt. Both concern the art of hardware accelerators, and the incorporation would have, according to Uhrenholt, improve processing and power efficiency (0003).
Claim(s) 19 recite methods analogous to the above apparatuses and are hence rejected for the same reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Shao (US 20200293867 A1) discloses a reconfigurable hardware accelerator for a neural network.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LIANG LI whose telephone number is (303)297-4263. The examiner can normally be reached Mon-Fri 9-12p, 3-11p MT (11-2p, 5-1a ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. The examiner is available for interviews Mon-Fri 6-11a, 2-7p MT (8-1p, 4-9p ET).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Jennifer Welch can be reached on (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/LIANG LI/
Primary examiner AU 2143
Prosecution Insights