DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to claims filed on 01/29/2026.
Claims 1-6 and 8-21 are pending.
Claim Objections
Claim 9 is objected to because of the following informalities: The limitation: “deprioritise operations that have block identifier outside a range of other blocks;” should read “deprioritise operations that have a block identifier outside a range of other blocks;”. Further, the limitation: “do not deprioritise when block identifier numbers within a range of each other;” should read “do not deprioritise operations when block identifier numbers are within a range of each other;”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 4, 10-13, and 15 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 4 recite(s) the limitation(s): “wherein the priority value is initialised to the graph depth of a section of an operation” and claim 1, from which claim 4 depends, recites the limitation(s): “wherein the priority value is a block identifier representative of an iteration of depth of block position within the task data;”. However, upon review of the initial application filing, proper support in the written description for this limitation was not found. The specification states: “Referring to Figure 12, a starting priority for blocks can be graph depth … Graph depth may be used alone or combined with other information, such as the iteration depth of a block in the task’s operation space, or with other standard arbitration mechanisms, such as round robin or least-recently-granted or with a priority increment to indicate how frequently a new block is needed” in paragraph 213, and “As can be seen in Figure 14, in the column operation space step #, as we go down steps in operation space the block identification increments and is added to block depth. As seen from input reader priority and convolution engine priority which are run every cycle, the priority is incremented by a value 1” in paragraph 226. While the initial application filing includes prioritizing operations based on iteration depth and a priority value that is initialized to graph depth, this iteration depth is not the same as the priority value initialized to graph depth. The instant specification describes using a block identifier representing the iteration depth of a block section to increment this priority value, but it does not state that this block identifier itself is instialized to the graph depth of a section. Thus, there is a lack of support in the initial filing for this limitation of claim 4. If applicant disagrees that there is support within the written description, please provide citation to where the support can be found within the specification.
Claims 10-13 and 15 depend, directly or indirectly, from rejected claims and do not resolve the deficiencies thereof and are therefore rejected for at least the same reasons.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 10 and 14 recite the limitations “wherein the block command includes…” and “wherein the block command comprises…”. It is unclear which block command these limitations are referring to. Claim 1, from which the claims depend, states “the one or more data blocks with each data block being assigned a priority value and comprising a block command”. Given that there is a block command for each of the one or more data blocks, there may be multiple block commands. However, it is unclear which specific block command is being claimed in the limitations of claims 10 and 14. For the sake of compact prosecution, examiner will interpret these limitations to mean “wherein each block command includes…” and wherein each block command comprises…”.
Claim 11 recites the limitation “when the data block…”. It is unclear which data block this limitation is referring to. Claim 1, from which the claim depends, states “the one or more data blocks”. Given that there are one or more data blocks, there may be multiple data blocks. However, it is unclear which specific data block is being claimed in the limitation of claim 11. For the sake of compact prosecution, examiner will interpret this to mean “when a data block…”.
Claims 11-13 and 15 depend, directly or indirectly, from rejected claims and do not resolve the deficiencies thereof and are therefore rejected for at least the same reasons.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-21 are rejected under 35 U.S.C. 101 because the claimed invention recites a judicial exception, is directed to that judicial exception, an abstract idea, as it has not been integrated into practical application and the claims further do not recite significantly more than the judicial exception. Examiner has evaluated the claims under the framework provided in the 2019 Patent Eligibility Guidance published in the Federal Register 01/07/2019 and has provided such analysis below.
Step 1: Claims 1-18 are directed to a memory unit and fall within the statutory category of machine. Claim 19 is directed to a method and falls within the statutory category of process. Claims 20-21 are directed to a processor and fall within the statutory category of machine. Therefore, “Are the claims to a process, machine, manufacture or composition of matter?” Yes.
In order to evaluate the Step 2A inquiry “Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?” we must determine, at Step 2A Prong 1, whether the claim recites a law of nature, a natural phenomenon or an abstract idea and further whether the claim recites additional elements that integrate the judicial exception into a practical application.
Step 2A Prong 1:
Claims 1 and 19: The limitations of “wherein, the memory unit is configured to arbitrate between the one or more data blocks based upon the block identifier and block command of the respective one or more data blocks to prioritize the sequence of processing requests”, “arbitrating at the memory unit between the one or more data blocks based upon the block identifier and block command of the one or more data blocks, and prioritizing the sequence of processing requests”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe block identifiers and block commands of data blocks and, based on these observations, can mentally arbitrate between data blocks to prioritize a sequence of processing requests through mental comparison. This may also be done with pencil and paper.
Claim 20: The limitations of “and for each of a portion of the operation space: assign an order of priority and a block command to each of the one or more data blocks”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe one or more data blocks and, based on these observations, can mentally assign a priority and a block command to each data block. Further, the limitations of “and transform each portion of the operation space to generate respective operation-specific local spaces for each of the plurality of the operations of the acyclic graph according to the order of priority;”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe a portion of operation space and an order of priority of operations and, based on these observations, can mentally generate a respective operation-specific local space by mentally partitioning the space. This can be performed for each of a plurality of operations in an acyclic graph based on the order of priority. This may also be done with pencil and paper.
Therefore, Yes, claims 1, 19, and 20 recite a judicial exception.
Step 2A Prong 2:
Claims 1 and 19: The judicial exception is not integrated into a practical application. In particular, the claims recite additional element recitations of “A memory unit configured for handling task data,”, which are merely recitations of generic computing components (see MPEP § 2106.05(f)) which does not integrate a judicial exception into practical application. Further, the claims recite additional element recitations of “the task data describing a task to be executed in the form of a directed acyclic graph of operations, wherein each of the operations maps to a corresponding execution unit, and wherein each connection between operations in the acyclic graph maps to a corresponding storage element of the execution unit,”, “the task data further defining an operation space representing dimensions of a multi-dimensional arrangement of the connected operations to be executed represented by one or more data blocks;”, “wherein the priority value is a block identifier representative of an iteration of depth of block position within the task data;”, and “and wherein the processing requests include writing data to storage or reading data from storage”, which are merely recitations of technological environment/field of use (see MPEP § 2106.05(h)) which does not integrate a judicial exception into practical application. Further, the claims recite additional element recitations of “the memory unit configured to receive a sequence of processing requests comprising the one or more data blocks with each data block being assigned a priority value and comprising a block command;”, “the method including receiving at a memory unit a sequence of processing requests comprising the one or more data blocks with each data block being assigned a priority value and comprising a block command;”, and “and writing data to storage or reading data from storage”, which are merely recitations of data reception and storage which is insignificant extra solution activity (see MPEP §2106.05(g)) which does not integrate a judicial exception into practical application.
Claim 20: The judicial exception is not integrated into a practical application. In particular, the claims recite additional element recitations of “A processor for handling data, the processor comprising a handling unit configured to:”, which are merely recitations of generic computing components (see MPEP § 2106.05(f)) which does not integrate a judicial exception into practical application. Further, the claims recite additional element recitations of “obtain, from storage, task data that describes a task to be executed in the form of a directed acyclic graph of operations, wherein each of the operations maps to a corresponding execution unit of a connected processor, and wherein each connection between operations in the acyclic graph maps to a corresponding storage element of the processor,” and “and for each of the dimensions of the operation space associated with operations for which transformed local spaces have been generated, dispatch one or more data blocks to the one or more of a plurality of the execution units of the connected processor, based on the block identifier”, which are merely recitations of data storage and transmission which is insignificant extra solution activity (see MPEP §2106.05(g)) which does not integrate a judicial exception into practical application. Further, the claims recite additional element recitations of “the task data further defining an operation space representing dimensions of a multi-dimensional arrangement of the connected operations to be executed represented by one or more data blocks;” and “wherein the order of priority is a block identifier representative of an iteration depth of block position within the task data,”, which are merely recitations of technological environment/field of use (see MPEP § 2106.05(h)) which does not integrate a judicial exception into practical application.
Therefore, “Do the claims recite additional elements that integrate the judicial exception into a practical application? No, these additional elements do not integrate the abstract idea into a practical application and they do not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
After having evaluated the inquires set forth in Steps 2A Prong 1 and 2, it has been concluded that claims 1, 19, and 20 not only recite a judicial exception but that the claims are directed to the judicial exception as the judicial exception has not been integrated into practical application.
Step 2B:
Claims 1, 19, and 20: The claims do not include additional elements, alone or in combination, that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements amount to no more than generic computing components, technological environment/field of use, and insignificant extra solution activity which do not amount to significantly more than the abstract idea. Further, the insignificant extra solution activity is well-understood, routine, and conventional in the art. “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory” [MPEP§ 2106.05(d)(II)].
Therefore, “Do the claims recite additional elements that amount to significantly more than the judicial exception? No, these additional elements, alone or in combination, do not amount to significantly more than the judicial exception.
Having concluded analysis within the provided framework, Claims 1, 19, and 20 do not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 2, the claim recites additional element recitations of “wherein reading data from storage includes sending a read request to a memory system cache, reading data from the memory system cache, and writing the data to the storage element of the execution unit”, which are merely recitations of data transmission and storage which is insignificant extra solution activity (see MPEP §2106.05(g)) which does not integrate a judicial exception into practical application. Further, the insignificant extra solution activity is well-understood, routine, and conventional in the art. “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory” [MPEP§ 2106.05(d)(II)]. Further, claim 2 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 2 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 2 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 3, the claim recites additional element recitations of “wherein writing data to storage includes reading data from the storage element of the execution unit and sending a write request of the data to a memory system cache”, which are merely recitations of data storage and transmission which is insignificant extra solution activity (see MPEP §2106.05(g)) which does not integrate a judicial exception into practical application. Further, the insignificant extra solution activity is well-understood, routine, and conventional in the art. “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory” [MPEP§ 2106.05(d)(II)]. Further, claim 3 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 3 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 3 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 4, the claim recites additional abstract idea recitations of “wherein the priority value is initialised to the graph depth of a section of an operation”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe a graph depth of a section of an operation and, based on these observations, can mentally assign the priority value to be the graph depth. This may also be done with pencil and paper. Further, claim 4 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 4 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 4 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 5, the claim recites additional abstract idea recitations of “wherein a lower value of graph depth indicates a higher priority”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe a graph depth of a section of an operation and, based on these observations, can mentally assign the priority value to be the graph depth such that a lower value indicates a higher priority. This may also be done with pencil and paper. Further, claim 5 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 5 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 5 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 6, the claim recites additional abstract idea recitations of “wherein the memory unit uses graph depth to arbitrate between different blocks that are being processed by an execution unit at the same time”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe graph depth of different blocks and, based on these observations, can mentally arbitrate between the different blocks that are being processed by an execution unit at the same time through mental comparison. Further, claim 6 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 6 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 6 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 8, the claim recites additional abstract idea recitations of “wherein arbitration using graph depth is combined with the iteration depth of a block position within the task data;”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can perform mental arbitration using both graph depth and iteration depth of a block position within task data. Further, the claim recites additional abstract idea recitations of “and combined with other arbitration algorithms, comprising at least one of a round-robin algorithm or a least-recently-granted algorithm”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can perform mental arbitration using both graph depth and iteration depth of a block position within task data and another arbitration algorithm. Further, claim 8 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 8 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 8 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 9, the claim recites additional abstract idea recitations of “wherein a combination of graph depth and block identifier is used to arbitrate between different blocks according to one or more of the following conditions: a. deprioritise operations that have block identifier outside a range of other blocks; b. do not deprioritise when block identifier numbers within a range of each other; and c. for block identifiers within a range of each other, prioritise low graph depth”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can mentally arbitrate between different blocks by deprioritising operations that are outside a range of other blocks; not deprioritising when block identifier numbers are within a range of each other; and for block identifiers within a range of each other, prioritising low graph depth. This can be done through observation of graph depth and block identifier and mental comparison based on these observations. Further, claim 9 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 9 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 9 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 10, the claim recites additional element recitations of “wherein the block command includes a priority head value and a priority tail value initialised to the graph depth of the section”, which are merely recitations of technological environment/field of use (see MPEP § 2106.05(h)) which does not integrate a judicial exception into practical application. Further, claim 10 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 10 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 10 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 11, the claim recites additional abstract idea recitations of “wherein the priority head value is incremented when the data block is issued for writing data to storage”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe when a data block is issued for writing data to storage and, based on this observation, can mentally increment a priority head value. This may also be done with pencil and paper. Further, claim 11 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 11 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 11 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 12, the claim recites additional abstract idea recitations of “wherein the priority tail value is incremented wherein data is read from storage”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe when data is read from storage and, based on this observation, can mentally increment a priority tail value. This may also be done with pencil and paper. Further, claim 12 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 12 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 12 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 13, the claim recites additional abstract idea recitations of “wherein a minimum priority tail value is used to determine which processing request is the highest priority across the sequence of processing requests comprising the one or more data blocks being executed by memory unit”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe a minimum priority tail value and, based on this observation, can mentally determine which value of priority head value is the highest priority across the sequence of processing requests. Further, claim 13 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 13 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 13 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 14, the claim recites additional element recitations of “wherein the block command comprises a pointer, a section space for a block, a tensor descriptor with instructions for an address of a tensor which is being loaded or stored”, which are merely recitations of technological environment/field of use (see MPEP § 2106.05(h)) which does not integrate a judicial exception into practical application. Further, claim 14 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 14 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 14 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 15, the claim recites additional element recitations of “wherein data is written or read from storage as defined in the block command”, which are merely recitations of data storage which is insignificant extra solution activity (see MPEP §2106.05(g)) which does not integrate a judicial exception into practical application. Further, the insignificant extra solution activity is well-understood, routine, and conventional in the art. “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory” [MPEP§ 2106.05(d)(II)]. Further, claim 15 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 15 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 15 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 16, the claim recites additional element recitations of “comprising an input reader channel and output reader channel configured to be instantiated by the memory unit;”, which are merely recitations of technological environment/field of use (see MPEP § 2106.05(h)) which does not integrate a judicial exception into practical application. Further, the claim recites additional element recitations of “and comprises including a weight fetcher command to the memory unit to read compressed data and subsequently send the compressed data to a weight decoder to compress the data”, which are merely recitations of data transmission which is insignificant extra solution activity (see MPEP §2106.05(g)) which does not integrate a judicial exception into practical application. Further, the insignificant extra solution activity is well-understood, routine, and conventional in the art. “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory” [MPEP§ 2106.05(d)(II)]. Further, claim 16 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 16 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 16 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 17, the claim recites additional element recitations of “wherein the block command comprises a tag to indicate whether the block is participating in the arbitration”, which are merely recitations of technological environment/field of use (see MPEP § 2106.05(h)) which does not integrate a judicial exception into practical application. Further, claim 17 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 17 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 17 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 18, the claim recites additional abstract idea recitations of “wherein arbitration is determined by applying a round robin algorithm when priority value of the blocks for processing are equal”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe when priority value of blocks for processing are equal and, based on this observation, can perform mental arbitration by applying a round robin algorithm. Further, claim 18 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 18 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 18 does not recite patent eligible subject matter under 35 U.S.C. § 101.
With regard to claim 21, the claim recites additional abstract idea recitations of “wherein priority is assigned by first determining output availability for the respective operation-specific local space by assessing availability of an execution unit and memory to write output,”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe availability of an execution unit and memory to write output and, based on these observations, can mentally determine output availability. Further, the claim recites additional abstract idea recitations of “then for a plurality of respective operation-specific local spaces with output availability, serialize the section of the data blocks for transform by a synchronization unit”, as drafted, is a process that, but for the recitation of generic computing components, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a person can observe sections for a plurality of respective operation-specific local spaces with output availability and, based on these observations, can mentally serialize the sections by mentally translating them to a different format. This may also be done with pencil and paper. Further, claim 21 does not recite any further additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, claim 21 also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, Claim 21 does not recite patent eligible subject matter under 35 U.S.C. § 101.
Therefore, Claims 1-21 do not recite patent eligible subject matter under U.S.C. §101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-6, 8, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1).
With regard to claim 1, Bokam teaches:
A memory unit configured for handling task data, “Generally, the system can access a processor representation that defines the set of data-transfer resources of the multicore processor. More specifically, the system can access a representation of a set of DMA cores of the multicore processor configured to transfer data between memory locations in the memory hierarchy of the processor” [Bokam ¶ 29].
the task data describing a task to be executed in the form of a directed acyclic graph of operations, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: access a processor representation defining the compute resources and data-transfer resources of the multicore processor; access a network structure defining the layers and connectivity of the network; generate a directed acyclic graph (hereinafter "DAG") representing individual operations for execution of the network on the multicore processor and dependencies between these operations for each layer of the network; generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
wherein each of the operations maps to a corresponding execution unit, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: … generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
and wherein each connection between operations in the acyclic graph maps to a corresponding storage element of the execution unit, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: … generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
“a set of compute nodes representing a set of compute operations for the set of processor cores; a set of data transfer nodes representing a set of data transfer operations for the set of direct memory access cores; and a set of edges representing dependencies between the set of compute operations and the set of data transfer operations; and generating a selected schedule for the layer based on the selected graph, the selected schedule assigning the set of compute nodes to the set of processor cores and assigning the set of data transfer nodes to the set of direct memory access cores;” [Bokam Claim 1, Fig. 1 Examiner notes the connection between a compute node and a data transfer node is considered to map to the corresponding storage element of the direct memory access core assigned to the data transfer node].
the task data further defining an operation space representing dimensions of a multi-dimensional arrangement of the connected operations to be executed “Additionally, the system can access the network structure that defines properties of the network such as the number of layers, the type of each layer (e.g., convolutional, fully connected, pooling), dimensions of the input tensor for each layer (hereinafter "input tensor dimensions"), weight tensor dimensions of each layer. The system can further access or derive, based on the input tensor dimensions of a subsequent layer, the dimensions of the output tensors (hereinafter "output tensor dimensions"). Thus, the system can generate a layer-specific schedule defining compute operations and data-transfer operations that execute a particular layer on the multicore processor based on the properties of the layer accessed from the network structure” [Bokam ¶ 20].
represented by one or more data blocks; “Generally, the system partitions the input tensor into chunks that can be efficiently operated on by the processor. More specifically, the system can divide the 1D, 2D, 3D, or 4D array into chunks that can fit within the registers defined by the processor. For example, the system can partition an input tensor representing a 30x30 pixel image with a color depth of 3 into 60 3x5x3 partitions (data blocks) for a processor including 64-bit 2D registers” [Bokam ¶ 55].
the memory unit configured to receive a sequence of processing requests comprising the one or more data blocks “Generally, the system defines a set of operations for each layer of the network based on the type of layer, the input partitions and weight partitions of the layer, and the cost model for the processor More specifically, the system can generate graphs defining a set of data-transfer nodes representing a set of data-transfer operations for the set of DMA cores … Thus, the system can generate a schedule that specifies specific data-transfer operations to be executed by the DMA cores in the multicore processor” [Bokam ¶ 66].
with each data block being assigned a priority value “The node priority strategy parameter indicates the priority assignment strategy for operations of the same type. For example, this execution parameter can indicate that nodes with the highest out-degree are assigned highest priority, that nodes with the lowest out-degree are assigned highest priority, that nodes with the shortest approximate duration are assigned highest priority, that nodes with the longest approximate duration are assigned highest priority, that nodes with the lowest depth are assigned highest priority, or that nodes with highest depth are assigned highest priority” [Bokam ¶ 73].
and comprising a block command, “In one implementation further described below, the system can: partition the input tensor of a layer and the weight tensor of a layer to generate a set of input partitions (block commands) and a set of weight partitions; and generate a graph representing compute and data transfer operations for transforming these input partitions and weight partitions into output partitions according to calculations defined by the layer type” [Bokam ¶ 50].
wherein, the memory unit is configured to arbitrate between the one or more data blocks based upon the (priority value) block identifier and block command of the respective one or more data blocks to prioritize the sequence of processing requests “Additionally, as described above with respect to generating an individual schedule for a layer of the network, the system can also, for each candidate graph in the set of candidate graphs: calculate a node priority for each compute node in the set of compute nodes and for each data transfer node in the set of data transfer nodes; and execute a directed acyclic graph scheduling algorithm based on the node priority for each compute node in the set of compute nodes and for each data transfer node in the set of data transfer nodes to generate a candidate schedule in the set of candidate schedules” [Bokam ¶ 89].
and wherein the processing requests include writing data to storage or reading data from storage. “Upon generating the partitions of the input tensors and weights of each layer, the system can define a set of operations to execute each layer computation from input partitions and the weight partitions based on a cost model for each operation type of the processor. These operations can include shifting tensor data between storage locations via transfer busses, shifting data via shift registers, performing stencil operations on stored data, performing ALU operations on input partitions and weight partitions; and reducing ALU outputs to form layer outputs at the reducing unit” [Bokam ¶ 22]. “More specifically, the system can generate graphs defining a set of data-transfer nodes representing a set of data-transfer operations for the set of DMA cores, the set of data-transfer operations including: a data transfer from a main memory of the multicore processor to a shared cache of the multicore processor; and a data transfer from the shared cache of the multi core processor to an individual cache in the set of individual caches of the multicore processor; a data transfer from an individual cache in the set of individual caches of the multicore processor to the shared cache of the multicore processor; and a data transfer from the shared cache of the multi core processor to the main memory of the multicore processor. Thus, the system can generate a schedule that specifies specific data-transfer operations to be executed by the DMA cores in the multicore processor” [Bokam ¶ 66]. “Therefore, in one implementation, the system can store the output of a particular layer at a memory location that corresponds to the expected memory location for the input of a subsequent layer” [Bokam ¶ 52].
Bokam fails to teach wherein the priority value is a block identifier representative of an iteration of depth of block position within the task data; wherein, the memory unit is configured to arbitrate between the one or more data blocks based upon the block identifier and block command of the respective one or more data blocks to prioritize the sequence of processing requests.
However, Eichenberger teaches:
wherein the priority value is a block identifier representative of an iteration of depth of block position within the task data; “As discussed above, the AST is an encoded hierarchical ordered graph where each inner node corresponds to an iteration domain (block identifier) at a given depth in the loop nest structure. Each leaf node has also a list of statements that are enclosed by the loop nest. For a given node N at depth d in the AST, the node is associated with a domain DN, which is a polyhedral representation of the domain associated with the enclosed statements, and projected to reflect the depth d of the node in the AST” [Eichenberger ¶ 153]. “More precisely, each intermediate node of the program in the AST bears a polyhedron whose size is related to the depth d of the node. Each such polyhedron defines the nonredundant set of constraints needed to scan all the points in the corresponding transformed loop of depth d. The nesting of these polyhedra is directly translated into the nesting of the resulting loop nest. The leaves of the tree represent the polyhedral statements” [Eichenberger ¶ 97].
wherein, the memory unit is configured to arbitrate between the one or more data blocks based upon the block identifier and block command of the respective one or more data blocks to prioritize the sequence of processing requests “Each code generation optimization of the one or more code generation optimizations may receive a list of nodes in the program loop view for which the code generation optimization is to be applied and a propagation mode indicator that identifies a manner by which the list of nodes is to be processed by the code generation optimization. The list of nodes may be a list of prefix vectors for nodes in the program loop view of the source code. The prefix vector may be a vector of numbers which indicates a corresponding node's path from a root node to the corresponding node in the program loop view of the source code” [Eichenberger ¶ 23]. “The if-host/if-hoist-gentle code generation optimization walks the children of the given node and finds conditions on the current loop's depth and hoists them” [Eichenberger ¶ 124].
Eichenberger is considered to be analogous to the claimed invention because it is in the same field of arrangements for executing specific programs. The code optimization of Eichenberger can be combined with the priority scheduling of Bokam such that operations are prioritized based on an iteration depth within task data. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam to incorporate the teachings of Eichenberger and include the priority value is a block identifier representative of an iteration of depth of block position within the task data; wherein, the memory unit is configured to arbitrate between the one or more data blocks based upon the block identifier and block command of the respective one or more data blocks to prioritize the sequence of processing requests. Doing so would lower the control flow overhead. “Thereafter, the loops in the program loop view are optimized using code generation transformations that improve the control flow overhead of the program without modifying the program order of the statements in the program statement view of the program” [Eichenberger ¶ 80].
With regard to claim 4, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam further teaches wherein the priority value is initialised to the graph depth of a section of an operation. “In another example, the system can prioritize operations based on the depth of these operations in the graph” [Bokam ¶ 78].
With regard to claim 5, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 4, as referenced above. Bokam further teaches wherein a lower value of graph depth indicates a higher priority. “The node priority strategy parameter indicates the priority assignment strategy for operations of the same type. For example, this execution parameter can indicate that nodes with the highest out-degree are assigned highest priority, that nodes with the lowest out-degree are assigned highest priority, that nodes with the shortest approximate duration are assigned highest priority, that nodes with the longest approximate duration are assigned highest priority, that nodes with the lowest depth are assigned highest priority, or that nodes with highest depth are assigned highest priority” [Bokam ¶ 73].
With regard to claim 6, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 4, as referenced above. Bokam further teaches:
wherein the memory unit uses graph depth to arbitrate between different blocks “In another example, the system can prioritize operations based on the depth of these operations in the graph” [Bokam ¶ 78]. “Once the system has assigned a priority to each node in the graph, the system can schedule the root node of the graph and can insert all the children of the root node into a priority queue ordered by the assigned priority of each child node” [Bokam ¶ 79].
that are being processed by an execution unit at the same time. “The system can therefore assign operations in the DAG to parallel processing resources in the processor such that these operations can be executed in parallel” [Bokam ¶ 83].
With regard to claim 8, Bokam in view of Filoche teaches a memory unit as claimed in claim 1, as referenced above. Bokam further teaches:
wherein arbitration using graph depth… “In another example, the system can prioritize operations based on the depth of these operations in the graph” [Bokam ¶ 78]. “Once the system has assigned a priority to each node in the graph, the system can schedule the root node of the graph and can insert all the children of the root node into a priority queue ordered by the assigned priority of each child node” [Bokam ¶ 79].
…and combined with other arbitration algorithms, comprising at least one of a round-robin algorithm or a least-recently-granted algorithm. “The system can rank and schedule operations according to multiple DAG scheduling algorithms such as heterogeneous earliest finish time algorithm (HEFT), monte carlo algorithms (MCA), critical path on a processor algorithm (CPOPA), synthesized heuristic task scheduling algorithm (HCPPEFT), longest dynamic critical path algorithm (LDCP), highest level first with estimated time (HLFET), or any other static scheduling algorithm” [Bokam ¶ 83].
Bokam fails to teach is combined with the iteration depth of a block position within the task data.
However, Eichenberger teaches is combined with the iteration depth of a block position within the task data; “As discussed above, the AST is an encoded hierarchical ordered graph where each inner node corresponds to an iteration domain (block identifier) at a given depth in the loop nest structure. Each leaf node has also a list of statements that are enclosed by the loop nest. For a given node N at depth d in the AST, the node is associated with a domain DN, which is a polyhedral representation of the domain associated with the enclosed statements, and projected to reflect the depth d of the node in the AST” [Eichenberger ¶ 153]. “More precisely, each intermediate node of the program in the AST bears a polyhedron whose size is related to the depth d of the node. Each such polyhedron defines the nonredundant set of constraints needed to scan all the points in the corresponding transformed loop of depth d. The nesting of these polyhedra is directly translated into the nesting of the resulting loop nest. The leaves of the tree represent the polyhedral statements” [Eichenberger ¶ 97]. “The if-host/if-hoist-gentle code generation optimization walks the children of the given node and finds conditions on the current loop's depth and hoists them” [Eichenberger ¶ 124].
With regard to claim 19, Bokam teaches:
A computer implemented method of handling task data, the task data describing a task to be executed in the form of a directed acyclic graph of operations, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: access a processor representation defining the compute resources and data-transfer resources of the multicore processor; access a network structure defining the layers and connectivity of the network; generate a directed acyclic graph (hereinafter "DAG") representing individual operations for execution of the network on the multicore processor and dependencies between these operations for each layer of the network; generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
wherein each of the operations maps to a corresponding execution unit, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: … generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
and wherein each connection between operations in the acyclic graph maps to a corresponding storage element of the execution unit, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: … generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
“a set of compute nodes representing a set of compute operations for the set of processor cores; a set of data transfer nodes representing a set of data transfer operations for the set of direct memory access cores; and a set of edges representing dependencies between the set of compute operations and the set of data transfer operations; and generating a selected schedule for the layer based on the selected graph, the selected schedule assigning the set of compute nodes to the set of processor cores and assigning the set of data transfer nodes to the set of direct memory access cores;” [Bokam Claim 1, Fig. 1 Examiner notes the connection between a compute node and a data transfer node is considered to map to the corresponding storage element of the direct memory access core assigned to the data transfer node].
the task data further defining an operation space representing dimensions of a multi-dimensional arrangement of the connected operations to be executed “Additionally, the system can access the network structure that defines properties of the network such as the number of layers, the type of each layer (e.g., convolutional, fully connected, pooling), dimensions of the input tensor for each layer (hereinafter "input tensor dimensions"), weight tensor dimensions of each layer. The system can further access or derive, based on the input tensor dimensions of a subsequent layer, the dimensions of the output tensors (hereinafter "output tensor dimensions"). Thus, the system can generate a layer-specific schedule defining compute operations and data-transfer operations that execute a particular layer on the multicore processor based on the properties of the layer accessed from the network structure” [Bokam ¶ 20].
represented by one or more data blocks; “Generally, the system partitions the input tensor into chunks that can be efficiently operated on by the processor. More specifically, the system can divide the 1D, 2D, 3D, or 4D array into chunks that can fit within the registers defined by the processor. For example, the system can partition an input tensor representing a 30x30 pixel image with a color depth of 3 into 60 3x5x3 partitions (data blocks) for a processor including 64-bit 2D registers” [Bokam ¶ 55].
the method including receiving at a memory unit a sequence of processing requests comprising the one or more data blocks “Generally, the system defines a set of operations for each layer of the network based on the type of layer, the input partitions and weight partitions of the layer, and the cost model for the processor More specifically, the system can generate graphs defining a set of data-transfer nodes representing a set of data-transfer operations for the set of DMA cores … Thus, the system can generate a schedule that specifies specific data-transfer operations to be executed by the DMA cores in the multicore processor” [Bokam ¶ 66].
with each data block being assigned a priority value “The node priority strategy parameter indicates the priority assignment strategy for operations of the same type. For example, this execution parameter can indicate that nodes with the highest out-degree are assigned highest priority, that nodes with the lowest out-degree are assigned highest priority, that nodes with the shortest approximate duration are assigned highest priority, that nodes with the longest approximate duration are assigned highest priority, that nodes with the lowest depth are assigned highest priority, or that nodes with highest depth are assigned highest priority” [Bokam ¶ 73].
and comprising a block command; “In one implementation further described below, the system can: partition the input tensor of a layer and the weight tensor of a layer to generate a set of input partitions (block commands) and a set of weight partitions; and generate a graph representing compute and data transfer operations for transforming these input partitions and weight partitions into output partitions according to calculations defined by the layer type” [Bokam ¶ 50].
arbitrating at the memory unit between the one or more data blocks based upon the (priority value) block identifier and block command of the one or more data blocks, and prioritizing the sequence of processing requests “Additionally, as described above with respect to generating an individual schedule for a layer of the network, the system can also, for each candidate graph in the set of candidate graphs: calculate a node priority for each compute node in the set of compute nodes and for each data transfer node in the set of data transfer nodes; and execute a directed acyclic graph scheduling algorithm based on the node priority for each compute node in the set of compute nodes and for each data transfer node in the set of data transfer nodes to generate a candidate schedule in the set of candidate schedules” [Bokam ¶ 89].
and writing data to storage or reading data from storage. “Upon generating the partitions of the input tensors and weights of each layer, the system can define a set of operations to execute each layer computation from input partitions and the weight partitions based on a cost model for each operation type of the processor. These operations can include shifting tensor data between storage locations via transfer busses, shifting data via shift registers, performing stencil operations on stored data, performing ALU operations on input partitions and weight partitions; and reducing ALU outputs to form layer outputs at the reducing unit” [Bokam ¶ 22]. “More specifically, the system can generate graphs defining a set of data-transfer nodes representing a set of data-transfer operations for the set of DMA cores, the set of data-transfer operations including: a data transfer from a main memory of the multicore processor to a shared cache of the multicore processor; and a data transfer from the shared cache of the multi core processor to an individual cache in the set of individual caches of the multicore processor; a data transfer from an individual cache in the set of individual caches of the multicore processor to the shared cache of the multicore processor; and a data transfer from the shared cache of the multi core processor to the main memory of the multicore processor. Thus, the system can generate a schedule that specifies specific data-transfer operations to be executed by the DMA cores in the multicore processor” [Bokam ¶ 66]. “Therefore, in one implementation, the system can store the output of a particular layer at a memory location that corresponds to the expected memory location for the input of a subsequent layer” [Bokam ¶ 52].
Bokam fails to teach wherein the priority value is a block identifier representative of an iteration depth of block position within the task data; arbitrating at the memory unit between the one or more data blocks based upon the block identifier and block command of the one or more data blocks.
However, Eichenberger teaches:
wherein the priority value is a block identifier representative of an iteration depth of block position within the task data; “As discussed above, the AST is an encoded hierarchical ordered graph where each inner node corresponds to an iteration domain (block identifier) at a given depth in the loop nest structure. Each leaf node has also a list of statements that are enclosed by the loop nest. For a given node N at depth d in the AST, the node is associated with a domain DN, which is a polyhedral representation of the domain associated with the enclosed statements, and projected to reflect the depth d of the node in the AST” [Eichenberger ¶ 153]. “More precisely, each intermediate node of the program in the AST bears a polyhedron whose size is related to the depth d of the node. Each such polyhedron defines the nonredundant set of constraints needed to scan all the points in the corresponding transformed loop of depth d. The nesting of these polyhedra is directly translated into the nesting of the resulting loop nest. The leaves of the tree represent the polyhedral statements” [Eichenberger ¶ 97].
arbitrating at the memory unit between the one or more data blocks based upon the block identifier and block command of the one or more data blocks, “Each code generation optimization of the one or more code generation optimizations may receive a list of nodes in the program loop view for which the code generation optimization is to be applied and a propagation mode indicator that identifies a manner by which the list of nodes is to be processed by the code generation optimization. The list of nodes may be a list of prefix vectors for nodes in the program loop view of the source code. The prefix vector may be a vector of numbers which indicates a corresponding node's path from a root node to the corresponding node in the program loop view of the source code” [Eichenberger ¶ 23]. “The if-host/if-hoist-gentle code generation optimization walks the children of the given node and finds conditions on the current loop's depth and hoists them” [Eichenberger ¶ 124].
It would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam to incorporate the teachings of Eichenberger and include wherein the priority value is a block identifier representative of an iteration depth of block position within the task data; arbitrating at the memory unit between the one or more data blocks based upon the block identifier and block command of the one or more data blocks. Doing so would lower the control flow overhead. “Thereafter, the loops in the program loop view are optimized using code generation transformations that improve the control flow overhead of the program without modifying the program order of the statements in the program statement view of the program” [Eichenberger ¶ 80].
With regard to claim 20, Bokam teaches:
A processor for handling data, the processor comprising a handling unit configured to: “The computer-executable component can be a processor but any suitable dedicated hardware device can (alternatively or additionally) execute the instructions” [Bokam ¶ 99]. “Generally, as shown in FIG. 1, a computer system (hereinafter "the system"), which can include a single computational device (handling unit) or multiple computational devices (e.g., servers) connected over the internet, executes Blocks of the method S100 to generate a static schedule of an artificial neural network (hereinafter "the network") for execution on a multicore processor including both compute resources and data transfer resources” [Bokam ¶ 17].
obtain, from storage, task data that describes a task to be executed in the form of a directed acyclic graph of operations, “Additionally, the system can access the network structure that defines properties of the network such as the number of layers, the type of each layer (e.g., convolutional, fully connected, pooling), dimensions of the input tensor for each layer (hereinafter "input tensor dimensions"), weight tensor dimensions of each layer” [Bokam ¶ 20]. “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: access a processor representation defining the compute resources and data-transfer resources of the multicore processor; access a network structure defining the layers and connectivity of the network; generate a directed acyclic graph (hereinafter "DAG") representing individual operations for execution of the network on the multicore processor and dependencies between these operations for each layer of the network; generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
wherein each of the operations maps to a corresponding execution unit of a connected processor, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: … generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
and wherein each connection between operations in the acyclic graph maps to a corresponding storage element of the processor, “More specifically, in order to generate a static schedule for a network on a multicore processor with heterogenous resources, the system can: … generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
“a set of compute nodes representing a set of compute operations for the set of processor cores; a set of data transfer nodes representing a set of data transfer operations for the set of direct memory access cores; and a set of edges representing dependencies between the set of compute operations and the set of data transfer operations; and generating a selected schedule for the layer based on the selected graph, the selected schedule assigning the set of compute nodes to the set of processor cores and assigning the set of data transfer nodes to the set of direct memory access cores;” [Bokam Claim 1, Fig. 1 Examiner notes the connection between a compute node and a data transfer node is considered to map to the corresponding storage element of the direct memory access core assigned to the data transfer node].
the task data further defining an operation space representing dimensions of a multi-dimensional arrangement of the connected operations to be executed “Additionally, the system can access the network structure that defines properties of the network such as the number of layers, the type of each layer (e.g., convolutional, fully connected, pooling), dimensions of the input tensor for each layer (hereinafter "input tensor dimensions"), weight tensor dimensions of each layer. The system can further access or derive, based on the input tensor dimensions of a subsequent layer, the dimensions of the output tensors (hereinafter "output tensor dimensions"). Thus, the system can generate a layer-specific schedule defining compute operations and data-transfer operations that execute a particular layer on the multicore processor based on the properties of the layer accessed from the network structure” [Bokam ¶ 20].
represented by one or more data blocks; “Generally, the system partitions the input tensor into chunks that can be efficiently operated on by the processor. More specifically, the system can divide the 1D, 2D, 3D, or 4D array into chunks that can fit within the registers defined by the processor. For example, the system can partition an input tensor representing a 30x30 pixel image with a color depth of 3 into 60 3x5x3 partitions (data blocks) for a processor including 64-bit 2D registers” [Bokam ¶ 55].
and for each of a portion of the operation space: assign an order of priority “The node priority strategy parameter indicates the priority assignment strategy for operations of the same type. For example, this execution parameter can indicate that nodes with the highest out-degree are assigned highest priority, that nodes with the lowest out-degree are assigned highest priority, that nodes with the shortest approximate duration are assigned highest priority, that nodes with the longest approximate duration are assigned highest priority, that nodes with the lowest depth are assigned highest priority, or that nodes with highest depth are assigned highest priority” [Bokam ¶ 73].
and a block command to each of the one or more data blocks “In one implementation further described below, the system can: partition the input tensor of a layer and the weight tensor of a layer to generate a set of input partitions (block commands) and a set of weight partitions; and generate a graph representing compute and data transfer operations for transforming these input partitions and weight partitions into output partitions according to calculations defined by the layer type” [Bokam ¶ 50].
and transform each portion of the operation space to generate respective operation-specific local spaces for each of the plurality of the operations of the acyclic graph “In one implementation further described below, the system can: partition the input tensor of a layer and the weight tensor of a layer to generate a set of input partitions (operation-specific local spaces) and a set of weight partitions; and generate a graph representing compute and data transfer operations for transforming these input partitions and weight partitions into output partitions according to calculations defined by the layer type” [Bokam ¶ 50].
according to the order of priority; “Once the system has assigned a priority to each node in the graph, the system can schedule the root node of the graph and can insert all the children of the root node into a priority queue ordered by the assigned priority of each child node” [Bokam ¶ 79].
and for each of the dimensions of the operation space “More specifically, the X, Y, and Z input partition dimensions refer to the bit length of the input partition for each dimension of the input tensor. Generally, the system assumes regular partitions (as opposed to partitions of variable dimensions within the same layer). Likewise, the X, Y, and Z weight partition dimensions refer to the bit length of the weight partition for each dimension of the weight tensor” [Bokam ¶ 70].
associated with operations for which transformed local spaces have been generated, “In one implementation, the system can execute packing algorithms to efficiently generate input partitions based on the dimensions of the input tensor, the register dimensions and/or the receptive field dimensions (for convolutional layers)” [Bokam ¶ 59]. “In one implementation, the system can access a processor representation that defines the dimensions of each memory component in the multicore processor in order to generate a static schedule that specifies memory addresses for transfer operations assigned to the data-transfer resources of the processor. For example, the system can access a processor representation that defines a number of memory banks within the shared cache and the dimensions of the primary cache for each processor in order to partition the input tensors, weight tensors, and output tensors of the network such that these partitions can fit within each memory component along their data path during execution of the network on the multicore processor” [Bokam ¶ 28].
dispatch one or more data blocks to the one or more of a plurality of the execution units of the connected processor “Thus, the schedule of each layer defines a queue of instructions for each active compute resource and for each active data-transfer resource of the multicore processor that, when executed by the multicore processor, executes one layer of the network” [Bokam ¶ 77]. “Therefore, in one implementation, the system can store the output of a particular layer at a memory location that corresponds to the expected memory location for the input of a subsequent layer” [Bokam ¶ 52]. “The resource utilization strategy parameter indicates a priority assigned to particular resources of the multicore processor that determines the order with which operations are scheduled at these resources via the DAG scheduling algorithm” [Bokam ¶ 74].
Bokam fails to teach wherein the order of priority is a block identifier representative of an iteration depth of block position within the task data; dispatch one or more data blocks to the one or more of a plurality of the execution units of the connected processor, based on the block identifier.
However, Eichenberger teaches:
wherein the order of priority is a block identifier representative of an iteration depth of block position within the task data; “As discussed above, the AST is an encoded hierarchical ordered graph where each inner node corresponds to an iteration domain (block identifier) at a given depth in the loop nest structure. Each leaf node has also a list of statements that are enclosed by the loop nest. For a given node N at depth d in the AST, the node is associated with a domain DN, which is a polyhedral representation of the domain associated with the enclosed statements, and projected to reflect the depth d of the node in the AST” [Eichenberger ¶ 153]. “More precisely, each intermediate node of the program in the AST bears a polyhedron whose size is related to the depth d of the node. Each such polyhedron defines the nonredundant set of constraints needed to scan all the points in the corresponding transformed loop of depth d. The nesting of these polyhedra is directly translated into the nesting of the resulting loop nest. The leaves of the tree represent the polyhedral statements” [Eichenberger ¶ 97].
dispatch one or more data blocks to the one or more of a plurality of the execution units of the connected processor, based on the block identifier. “Each code generation optimization of the one or more code generation optimizations may receive a list of nodes in the program loop view for which the code generation optimization is to be applied and a propagation mode indicator that identifies a manner by which the list of nodes is to be processed by the code generation optimization. The list of nodes may be a list of prefix vectors for nodes in the program loop view of the source code. The prefix vector may be a vector of numbers which indicates a corresponding node's path from a root node to the corresponding node in the program loop view of the source code” [Eichenberger ¶ 23]. “The if-host/if-hoist-gentle code generation optimization walks the children of the given node and finds conditions on the current loop's depth and hoists them” [Eichenberger ¶ 124]. “A new execution order is then selected by using a reordering function, e.g., using a schedule, placement or chunking function. Then, in a code generation step, an AST or new source code is returned that implements the execution order implied by the reordering function” [Eichenberger ¶ 12].
It would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam to incorporate the teachings of Eichenberger and include wherein the order of priority is a block identifier representative of an iteration depth of block position within the task data; dispatch one or more data blocks to the one or more of a plurality of the execution units of the connected processor, based on the block identifier. Doing so would lower the control flow overhead. “Thereafter, the loops in the program loop view are optimized using code generation transformations that improve the control flow overhead of the program without modifying the program order of the statements in the program statement view of the program” [Eichenberger ¶ 80].
Claims 2 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Kadri (US 2023/0072082 A1).
With regard to claim 2, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam further teaches:
wherein reading data from storage includes sending a read request to a memory system cache, reading data from the memory system cache, “The memory hierarchy of the multicore processor can include a main memory (i.e., DDR SD RAM), a shared cache (i.e., L2 memory), and a set of primary caches for each processor core of the multicore processor … In one implementation, the system can access a processor representation that defines the dimensions of each memory component in the multicore processor in order to generate a static schedule that specifies memory addresses for transfer operations assigned to the data-transfer resources of the processor” [Bokam, ¶ 27-28 Fig. 1].
and writing the data to the storage element of the execution unit. “More specifically, the system can generate graphs defining a set of data-transfer nodes representing a set of data-transfer operations for the set of DMA cores, the set of data-transfer operations including: a data transfer from a main memory of the multicore processor to a shared cache of the multicore processor; and a data transfer from the shared cache of the multi core processor to an individual cache (storage element) in the set of individual caches of the multicore processor; a data transfer from an individual cache in the set of individual caches of the multicore processor to the shared cache of the multicore processor; and a data transfer from the shared cache of the multi core processor to the main memory of the multicore processor. Thus, the system can generate a schedule that specifies specific data-transfer operations to be executed by the DMA cores in the multicore processor” [Bokam ¶ 66].
Bokam in view of Eichenberger fails to explicitly teach wherein reading data from storage includes sending a read request.
However, Kadri teaches wherein reading data from storage includes sending a read request “For instance, the channel 530 may send read requests (i.e., requests to read data) to a memory through the memory interface 540. The channel 530 may also send write requests (i.e., requests to write data) to a memory through the memory interface 540” [Kadri ¶ 95].
Kadri is considered to be analogous to the claimed invention because it is in the same field of computing arrangements based on neural networks. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger to incorporate the teachings of Kadri and include that reading data from storage includes sending a read request. Doing so would facilitate data transfer to and from local memory. “Embodiments of the present disclosure may improve on at least some of the challenges and issues described above by providing a DNN accelerator that can facilitate compression and decompression of activation data transferred between a local memory of a compute block and an external memory” [Kadri ¶ 29].
With regard to claim 3, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam further teaches:
wherein writing data to storage includes reading data from the storage element of the execution unit “Therefore, in one implementation, the system can store the output of a particular layer at a memory location that corresponds to the expected memory location for the input of a subsequent layer” [Bokam ¶ 52].
to a memory system cache. “The memory hierarchy of the multicore processor can include a main memory (i.e., DDR SD RAM), a shared cache (i.e., L2 memory), and a set of primary caches for each processor core of the multicore processor … In one implementation, the system can access a processor representation that defines the dimensions of each memory component in the multicore processor in order to generate a static schedule that specifies memory addresses for transfer operations assigned to the data-transfer resources of the processor” [Bokam, ¶ 27-28 Fig. 1].
Bokam in view of Eichenberger fails to explicitly teach wherein writing data to storage includes reading data and sending a write request of the data.
However, Kadri teaches:
wherein writing data to storage includes reading data “The channel 530 may execute a data transfer task based on the task descriptor of the data transfer task. For instance, the data transfer channel may read the data block from the read address specified in the task descriptor and write the data block to the write address specified in the task descriptor” [Kadri ¶ 94].
and sending a write request of the data “For instance, the channel 530 may send read requests (i.e., requests to read data) to a memory through the memory interface 540. The channel 530 may also send write requests (i.e., requests to write data) to a memory through the memory interface 540” [Kadri ¶ 95].
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Moon (US 2022/0400258 A1).
With regard to claim 9, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam further teaches wherein a combination of graph depth and block identifier is used to arbitrate between different blocks “In another example, the system can prioritize operations based on the depth of these operations in the graph” [Bokam ¶ 78]. “Once the system has assigned a priority to each node in the graph, the system can schedule the root node of the graph and can insert all the children of the root node into a priority queue ordered by the assigned priority of each child node” [Bokam ¶ 79].
Bokam fails to teach wherein a combination of graph depth and block identifier is used to arbitrate between different blocks.
However, Eichenberger teaches wherein a combination of graph depth and block identifier is used to arbitrate between different blocks “As discussed above, the AST is an encoded hierarchical ordered graph where each inner node corresponds to an iteration domain (block identifier) at a given depth in the loop nest structure. Each leaf node has also a list of statements that are enclosed by the loop nest. For a given node N at depth d in the AST, the node is associated with a domain DN, which is a polyhedral representation of the domain associated with the enclosed statements, and projected to reflect the depth d of the node in the AST” [Eichenberger ¶ 153]. “More precisely, each intermediate node of the program in the AST bears a polyhedron whose size is related to the depth d of the node. Each such polyhedron defines the nonredundant set of constraints needed to scan all the points in the corresponding transformed loop of depth d. The nesting of these polyhedra is directly translated into the nesting of the resulting loop nest. The leaves of the tree represent the polyhedral statements” [Eichenberger ¶ 97]. “The if-host/if-hoist-gentle code generation optimization walks the children of the given node and finds conditions on the current loop's depth and hoists them” [Eichenberger ¶ 124].
Bokam in view of Eichenberger fails to explicitly teach according to one or more of the following conditions: a. deprioritise operations that have block identifier outside a range of other blocks; b. do not deprioritise when block identifier numbers within a certain range of each other; and c. for block identifiers within a range of each other, prioritise low graph depth.
However, Moon teaches according to one or more of the following conditions: a. deprioritise operations that have block identifier outside a range of other blocks; b. do not deprioritise when block identifier numbers within a certain range of each other; and c. for block identifiers within a range of each other, prioritise low graph depth. “According to the first method of the present invention, priorities may be determined on the basis of whether the current block is adjacent to the neighboring block and/or the position of the neighboring block. For example, a higher priority may be given to a neighboring block that is adjacent to the current block, and a lower priority may be given to a neighboring block that is not adjacent to the current block” [Moon ¶ 279].
Moon is considered to be analogous to the claimed invention because it is in the same field of indexing schemes relating to priority. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Filoche to incorporate the teachings of Moon and include according to one or more of the following conditions: a. deprioritise operations that have block identifier outside a range of other blocks; b. do not deprioritise when block identifier numbers within a certain range of each other; and c. for block identifiers within a range of each other, prioritise low graph depth. Doing so would allow for the order of the data blocks to be respected when assigning priority. “For example, by applying the second method and the third method simultaneously, the priority of the neighboring blocks for the configuring of the reference candidate list may be determined. FIG. 30 is a view illustrating a method of determining the priority of neighboring blocks considering both partition depth and partition order” [Moon ¶ 285].
Claims 10-12 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Zonca (US 2022/0374270 A1).
With regard to claim 10, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 4, as referenced above. Bokam in view of Eichenberger fails to teach wherein the block command includes a priority head value and a priority tail value initialised to the graph depth of the section.
However, Zonca teaches:
wherein the block command includes a priority head value “Returning to FIG. 1, the producer side (e.g., the producer threads 108a-n) can include a producer sequence-value 122 (priority head value). The producer sequence-value 122 can be an integer value, such as a 64-bit integer or a counter value, that represents an identifier of a particular write request 112 received by the producer threads 108a-n … The producer threads 108a-n may update (e.g., increment) the producer sequence-value 122 for each received write request, such that the current producer sequence-value corresponds to the most-recent write request 112 received by the producer threads 108a-n. The producer side can also include a producer buffer-value 120 indicating a current memory chunk (e.g., memory chunk 118c) for the producer threads 108a-n. The producer side can also include a producer buffer-value 120 indicating a current memory chunk (graph depth of the section) (e.g., memory chunk 118c) for the producer threads 108a-n. The producer buffer-value 120 may be a pointer to the current memory chunk where the producer threads 108a-n are to offer data items … The producer threads 108a-n can use the producer sequence-value 122 and the producer buffer-value 120 to coordinate among one another for storing data items in the data queue 102” [Zonca ¶ 17].
and a priority tail value initialised to the graph depth of the section. “In some examples, the consumer side (e.g., the consumer threads 114a-n) can include a consumer sequence-value 134 (priority tail value). The consumer sequence-value 134 can be an integer value, such as a 64-bit integer or a counter value, that represents an identifier of a particular read request 136 received by the consumer threads 114a-n … The consumer threads 114a-n may update the consumer sequence-value 134 for each received read request, such that the current consumer sequence-value corresponds to the most-recent read request 136 received by the consumer threads 114a-n. The consumer side can also include a consumer buffer-value 132 indicating a current memory chunk (graph depth of the section) (e.g., memory chunk 118b) for the consumer threads 114a-n. For example, the consumer buffer-value 132 may be a pointer to the current memory chunk where the consumer threads 114a-n are to read a next data item … The consumer threads 114a-n can use the consumer sequence-value 134 and the consumer buffer-value 132 to coordinate among one another for reading data items from the data queue 102” [Zonca ¶ 18].
Zonca is considered to be analogous to the claimed invention because it is in the same field of indexing schemes relating to allocation of resources. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger to incorporate the teachings of Zonca and include wherein the block command includes a priority head value and a priority tail value initialised to the graph depth of the section. Doing so would allow for reduced delay in the memory system. “Some aspects and features of the present disclosure can overcome one or more of the abovementioned problems by allowing the consumer threads to assist in the progressive chunking process. For example, if a producer thread is blocked from appending a memory chunk to the data queue due to a collision, a consumer thread can append the memory chunk to the data queue on behalf of the producer thread. This can allow the progressive chunking process to continue so that the producer thread can write its data items to the appended memory chunk with reduced delay, thereby improving the throughput of the system” [Zonca ¶ 9].
With regard to claim 11, Bokam in view of Eichenberger in view of Zonca teaches a memory unit as claimed in claim 10, as referenced above. Bokam in view of Eichenberger fails to teach wherein the priority head value is incremented when the data block is issued for writing data to storage.
However, Zonca teaches wherein the priority head value is incremented when the data block is issued for writing data to storage. “Returning to FIG. 1, the producer side (e.g., the producer threads 108a-n) can include a producer sequence-value 122 (priority head value). The producer sequence-value 122 can be an integer value, such as a 64-bit integer or a counter value, that represents an identifier of a particular write request 112 received by the producer threads 108a-n … The producer threads 108a-n may update (e.g., increment) the producer sequence-value 122 for each received write request, such that the current producer sequence-value corresponds to the most-recent write request 112 received by the producer threads 108a-n. The producer side can also include a producer buffer-value 120 indicating a current memory chunk (e.g., memory chunk 118c) for the producer threads 108a-n” [Zonca ¶ 17].
With regard to claim 12, Bokam in view of Eichenberger in view of Zonca teaches a memory unit as claimed in claim 10, as referenced above. Bokam in view of Eichenberger fails to teach wherein the priority tail value is incremented wherein data is read from storage.
However, Zonca teaches wherein the priority tail value is incremented wherein data is read from storage. “In some examples, the consumer side (e.g., the consumer threads 114a-n) can include a consumer sequence-value 134 (priority tail value). The consumer sequence-value 134 can be an integer value, such as a 64-bit integer or a counter value, that represents an identifier of a particular read request 136 received by the consumer threads 114a-n … The consumer threads 114a-n may update the consumer sequence-value 134 for each received read request, such that the current consumer sequence-value corresponds to the most-recent read request 136 received by the consumer threads 114a-n. The consumer side can also include a consumer buffer-value 132 indicating a current memory chunk (e.g., memory chunk 118b) for the consumer threads 114a-n.” [Zonca ¶ 18].
With regard to claim 15, Bokam in view of Eichenberger in view of Zonca teaches a memory unit as claimed in claim 10, as referenced above. Bokam further teaches wherein data is written or read from storage as defined in the block command. “Thus, the system can generate a layer-specific schedule defining compute operations and data-transfer operations that execute a particular layer on the multicore processor based on the properties of the layer accessed from the network structure” [Bokam ¶ 20]. “In particular, the system can generate a static schedule that includes distinct command queues for both the set of compute resources of the multicore processor and the set of data-transfer resources of the multicore processor” [Bokam ¶ 17]. “… generate a fixed schedule for each layer of the network assigning these operations to specific compute resources and data-transfer resources of the multicore processor; and aggregating these per-layer schedules into a complete schedule for the network executed on the multicore processor” [Bokam ¶ 18].
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Zonca (US 2022/0374270 A1) in view of Liu, Liqing (US 2020/0413428 A1).
With regard to claim 13, Bokam in view of Eichenberger in view of Zonca teaches a memory unit as claimed in claim 10, as referenced above. Bokam in view of Eichenberger in view of Zonca fails to teach wherein a minimum priority tail value is used to determine which processing request is the highest priority across the sequence of processing requests comprising the one or more data blocks being executed by memory unit.
However, Liu, Liqing teaches wherein a minimum priority tail value is used to determine which processing request is the highest priority across the sequence of processing requests comprising the one or more data blocks being executed by memory unit. “Here, #0, #1, and #2 are indices of the scheduling request configurations. For example, the SR #0 with the minimum index may have the highest priority” [Liu, Liqing ¶ 106].
Liu, Liqing is considered to be analogous to the claimed invention because it is in the same field of arrangements for dividing transmission path. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger in view of Zonca to incorporate the teachings of Liu, Liqing and include that a minimum priority tail value is used to determine which value of priority head value is the highest priority across the sequence of processing requests comprising the one or more data blocks being executed by memory unit. Doing so would allow for prioritizing operations between multiple requests when the requests are transmitted at the same time. “Furthermore, in a case that transmission of multiple scheduling requests to multiple scheduling request configurations is triggered in a certain time unit, the MAC layer may provide, to the physical layer, a notification/indication to signal the multiple scheduling requests” [Liu, Liqing ¶ 105].
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Zonca (US 2022/0374270 A1) in view of Bryan (2022/0229640 A1) in view of Liu, Shaoli (US 2021/0334105 A1).
With regard to claim 14, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam in view of Eichenberger fails to teach wherein the block command comprises a pointer.
However, Zonca teaches wherein the block command comprises a pointer, “Each of the processors 104a-n can include one or more cores for executing one or more processing threads, such as producer threads 108a-n and consumer threads 114a-n” [Zonca ¶ 13]. “The producer buffer-value 120 may be a pointer to the current memory chunk where the producer threads 108a-n are to offer data items. Some or all of the producer threads 108a-n may share the same producer buffer-value 120” [Zonca ¶ 17].
Bokam in view of Eichenberger in view of Zonca fails to explicitly teach a section space for a block.
However, Bryan teaches a section space for a block, “The model can include an operation representable by a matrix. The parameters can include the input and output ranges of the operation, the dimensions of the matrix, a noise value for the system, an overflow probability, a regularization parameter, and a desired number of accurate digits” [Bryan ¶ Abstract].
Bryan is considered to be analogous to the claimed invention because it is in the same field of computing arrangements based on neural networks. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger in view of Zonca to incorporate the teachings of Bryan and include a section space for a block. Doing so would allow for identifying constraints of the system inputs and outputs. “The method can further include selecting the datatype for the input or the output, based at least in part on the determined numerical range of the output, or the determined numerical range of the input” [Bryan ¶ 2]. “Devices configured to numerically solve EQ. 1 (e.g., using code deployed to the device, or the like), can be constrained in the range and precision of the numerical values that they can represent … Constraints may directly involve the datatype (e.g., word length, or the like), or may be implicated in the choice of datatype (e.g., latency, or the like)” [Bryan ¶ 34].
Bokam in view of Eichenberger in view of Zonca in view of Bryan fails to explicitly teach a tensor descriptor with instructions for an address of a tensor which is being loaded or stored.
However, Liu, Shaoli teaches a tensor descriptor with instructions for an address of a tensor which is being loaded or stored. “In some embodiments, after determining the descriptor of the tensor data to be synchronized, the first processor may store the content of the descriptor in the storage space of synchronized data, and generate and send the descriptor synchronization instruction according to the address of the content of the descriptor in the storage space of synchronized data” [Liu, Shaoli ¶ 76].
Liu, Shaoli is considered to be analogous to the claimed invention because it is in the same field of computing arrangements based on neural networks. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger in view of Zonca in view of Bryan to incorporate the teachings of Liu, Shaoli and include a tensor descriptor with instructions for an address of a tensor which is being loaded or stored. Doing so would allow for further transmission efficiency. “In this way, there is no need to transmit tensor data itself from the first processor to the second processor during synchronization, which reduces the amount of transmitted data and synchronization overhead and improves processing efficiency” [Liu, Shaoli ¶ 66].
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Kumar (US 2014/0177648 A1) in view of King (US 7,760,936 B1).
With regard to claim 16, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam in view of Eichenberger fails to teach comprising an input reader channel and output reader channel configured to be instantiated by the memory unit.
However, Kumar teaches comprising an input reader channel and output reader channel configured to be instantiated by the memory unit; “Aspects of the present application include a method, which may involve associating a tag with one or more packets of at least one input channel of a network on chip (NoC), the tag having information for one or more arbitration decisions; reading the information of the tag; and conducting an arbitration of the one or more packets for an output channel based on the reading of the information” [Kumar ¶ 28]. “The physical channels are time sliced into a number of independent logical channels called virtual channels (VCs)” [Kumar ¶ 14].
Kumar is considered to be analogous to the claimed invention because it is in the same field of arrangements for dividing transmission path. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger to incorporate the teachings of Kumar and include an input reader channel and output reader channel configured to be instantiated by the memory unit. Doing so would allow for transmission of different message types through virtual channels. “A NoC interconnect may contain multiple physical networks. Over each physical network, there may exist multiple virtual networks, wherein different message types are transmitted over different virtual networks. In this case, at each physical link or channel, there are multiple virtual channels; each virtual channel may have dedicated buffers at both end points” [Kumar ¶ 12].
Bokam in view of Eichenberger in view of Kumar fails to teach and comprises including a weight fetcher command to the memory unit to read compressed data and subsequently send the compressed data to a weight decoder to compress the data.
However, King teaches and comprises including a weight fetcher command to the memory unit to read compressed data and subsequently send the compressed data to a weight decoder to compress the data. “For example, compressed data may be read from a frame buffer, decompressed and used in some manner. Embodiments in accordance with the present invention can be used to recompress the data before it is read back into storage” [King Col. 7 Lines 56-60].
King is considered to be analogous to the claimed invention because it is in the same field of arrangements for program control considering the load. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger in view of Kumar to incorporate the teachings of King and include and comprises including a weight fetcher command to the memory unit to read compressed data and subsequently send the compressed data to a weight decoder to compress the data. Doing so would allow for further storage efficiency. “Texture data can be compressed (encoded) to reduce the amount of memory required to store textures and to reduce the bandwidth needed to read the textures” [King Col. 1 Lines 21-23].
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Kumar (US 2014/0177648 A1).
With regard to claim 17, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam in view of Eichenberger fails to teach wherein the block command comprises a tag to indicate whether the block is participating in the arbitration.
However, Kumar teaches wherein the block command comprises a tag to indicate whether the block is participating in the arbitration. “To implement the dual arbitration, the router's arbitration logic may need to only allow those input packets to participate in the arbitration whose tag value matches the phase of the packet's desired output channel” [Kumar ¶ 47].
It would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger to incorporate the teachings of Kumar and include wherein the block command comprises a tag to indicate whether the block is participating in the arbitration. Doing so would allow for transmission of different message types through virtual channels. “A NoC interconnect may contain multiple physical networks. Over each physical network, there may exist multiple virtual networks, wherein different message types are transmitted over different virtual networks. In this case, at each physical link or channel, there are multiple virtual channels; each virtual channel may have dedicated buffers at both end points” [Kumar ¶ 12].
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Kibardin (US 2023/0071424 A1).
With regard to claim 18, Bokam in view of Eichenberger teaches a memory unit as claimed in claim 1, as referenced above. Bokam in view of Eichenberger fails to teach wherein arbitration is determined by applying a round robin algorithm when priority value of the blocks for processing are equal
However, Kibardin teaches wherein arbitration is determined by applying a round robin algorithm when priority value of the blocks for processing are equal. “At each instruction scheduling time, the highest priority 'ready' task is selected to run next. If there are multiple tasks ready at the same priority level, then a round-robin arbitration is used to select the next task to run” [Kibardin ¶ 935].
Kibardin is considered to be analogous to the claimed invention because it is in the same field of indexing schemes relating to priority. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger to incorporate the teachings of Kibardin and include that arbitration is determined by applying a round robin algorithm when priority value of the blocks for processing are equal. Doing so would allow for arbitration resolution for operations with the same priority. “The round-robin arbitration is configurable to run at each (instruction processing) pipeline advance or only when the currently running task is unable to run any more. When the main task is configured to be the same priority as micro-threads, the main task is considered in the round-robin arbitration” [Kibardin ¶ 935].
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Bokam (US 2021/0191765 A1) in view of Eichenberger (US 2009/0083702 A1) in view of Eskisan (US 2023/0205838 A1).
With regard to claim 21, Bokam in view of Eichenberger teaches a processor as claimed in claim 20, as referenced above. Bokam further teaches wherein priority is assigned by first determining output availability for the respective operation-specific local space by assessing availability of an execution unit and memory to write output, “In another example, for a dequeued node representing a data-transfer operation between a shared cache and primary cache, the system can attempt to schedule the data-transfer operation at a shared-to-primary cache DMA core in the multicore processor. Thus, the system can schedule each operation at any available resource compatible with the operation represented by the node” [Bokam ¶ 80]. “When the system is not able to schedule the node (e.g., no resources are available), the system can reduce the priority of this node and reinsert the node into the queue in order to allow the resources that can operate on this node to free up from performing previously scheduled nodes” [Bokam ¶ 81].
then for a plurality of respective operation-specific local spaces with output availability, “The system can schedule the operation represented by the dequeued node at an available resource by adding this operation to a queue corresponding to the available resource and indicating an approximate duration of the operation (e.g., based on the cost model). The system can then add children of the highest priority node to the queue. The system can then dequeue a subsequent node from the priority queue” [Bokam ¶ 81].
Bokam in view of Eichenberger fails to teach serialize the section of the data blocks for transform by a synchronization unit.
However, Eskisan teaches serialize the section of the data blocks for transform by a synchronization unit. “In at least one implementation, the rank 3 demultiplexer 1804 is configured to route its input 1803 to each of the arrays of processing elements in a serial manner as will be described in further detail with reference to FIG. 19” [Eskisan ¶ 316]. “As shown in FIG. 23, the elements of each of the input tensors are propagated in a serial manner to the tensor contraction processing block” [Eskisan ¶ 340].
Eskisan is considered to be analogous to the claimed invention because it is in the same field of matrix computation. Therefore, it would be obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bokam in view of Eichenberger to incorporate the teachings of Eskisan and include serialize the section of the data blocks for transform by a synchronization unit. Doing so would allow for the storage and transmission of data in accordance with the system processing unit. “The tensors may be stored in the memory 112 in an 8-bit, a 16-bit, a 32-bit, or a 64-bit format, though it should be noted that other formats may be supported by the memory 112. The format can depend on the type of processing unit 114 used” [Eskisan ¶ 87].
Response to Arguments
Applicant's arguments filed 01/29/2026 have been fully considered but they are not persuasive. Applicant argues in substance:
I. Claims 1-19, and 21 were rejected under 35 U.S.C. 112(b) as allegedly being indefinite. Applicant respectfully asserts that the present amendments fully address these rejections and request that the rejection be withdrawn.
a) Examiner respectfully disagrees. The 112(b) rejections of claims 10, 11, and 14 have not been addressed by the present amendments.
II. STEP 2A, PRONG 1: The Claims Do Not Recite a Mental Process
The Examiner alleges that a human could "observe priority values" and "mentally arbitrate." This oversimplifies the claim to a high level of abstraction, which is prohibited. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299 (Fed. Cir. 2016).
The claims involve: 1. Task data defining an operation space for a Directed Acyclic Graph (DAG). 2. Mapping operations to hardware execution units and connections to storage elements. 3. Calculating a block identifier based on iteration depth within that multi-dimensional space.
A human mind cannot perform these steps because they are inextricably tied to the specific architecture of a neural accelerator (execution units, storage elements, cache lines). "Iteration depth" in this context is a variable derived from the specific way the hardware traverses a tensor; it is not a concept that exists in human thought. "Claims do not recite a mental process when they do not contain limitations that can practically be performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations." See SRI Int'l, Inc. v. Cisco Systems, Inc., 930 F.3d 1295, 1304 (Fed. Cir. 2019). Therefore, the claims are not directed to an abstract idea.
a) Examiner respectfully disagrees. In response to applicant's argument that the claims do not recite a mental process, it is noted that the features upon which applicant relies (i.e., “Calculating a block identifier based on iteration depth within that multi-dimensional space”, “the specific architecture of a neural accelerator”, and “"Iteration depth" in this context is a variable derived from the specific way the hardware traverses a tensor;”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Further, MPEP § 2106.04 (III) states: “The courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea … Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions”. The claims are determined to recite mental processes because they recite limitations which can practically be performed in the human mind. The recited arbitration is a process which, as claimed, can be performed in the human mind or with pencil and paper using mental processes of observation, judgement, and determination. The additional elements of the claims are not evaluated as mental processes but instead are determined to amount to no more than generic computing components, technological environment/filed of use, and insignificant extra solution activity which do not amount to significantly more than the abstract idea. The arguments have been considered but were not found to be persuasive.
III. 1. The Specification Identifies a Specific Technical Problem and Solution
The Desjardins guidance instructs that "if the specification sets forth an improvement in technology... the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement." (MPEP § 2106.04(d)(1)).
The Problem: The Specification explains that processing neural networks (DAGs) involves complex dependencies. Standard arbitration (like round-robin) causes processor stalls and inefficient memory usage because execution units may wait for data that is not yet ready, or the memory unit may fetch data too early, wasting cache bandwidth. (See Spec. [0210]).
The Solution: The Specification discloses a specific solution: utilizing a "Block ID" that represents the "iteration depth" of a block within the operation space. This allows the hardware to synchronize sections that have different dimensions (e.g., due to reduction or broadcast operations) and prioritize critical path data. (See Spec. [0211]-[0217]; [0236]).
2. The Claims Reflect the Disclosed Improvement
Amended Claim 1 explicitly recites the mechanism that provides this improvement: "the priority value is a block identifier representative of an iteration of depth of block position within the task data; wherein, the memory unit is configured to arbitrate... based upon the block identifier and block command of the respective one or more data blocks..."
This is not a generic "arbitration" or a mental choice. It is a specific hardware configuration that tracks the iteration depth of data blocks within a multi-dimensional operation space. This mechanism directly addresses the problem of cache bandwidth waste and execution stalls described in the Specification.
Under Desjardins, the claims need not explicitly recite the result (e.g., "thereby improving efficiency") so long as they recite the components or steps that provide the improvement. By reciting the block identifier based on iteration depth, Claim 1 captures the precise technical means disclosed for optimizing the neural engine.
a) Examiner respectfully disagrees. In response to applicant's argument that the claimed invention provides an improvement in the technology, it is noted that the features upon which applicant relies (i.e., “hardware to synchronize sections that have different dimensions (e.g., due to reduction or broadcast operations) and prioritize critical path data”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
The claims include arbitrating tasks based on “a block identifier representative of an iteration of depth of block position within the task data”; however, the claims do not include the synchronization of sections that have different dimensions as applicant alleges. Claim 1 does include a block identifier and arbitration but without further detail to how this arbitration is actually performed, the claim fails to implement the argued solution of the specification. Thus, upon evaluation of the claims, it is not clear that an improvement in the technology is reflected. The arguments have been considered but were not found to be persuasive.
IV. 3. The Hardware Elements Are Not "Generic" Under Desjardins
The Office argued that the "memory unit" and "processor" are generic components. However, the new MPEP § 2106.05(a) guidance states: "When evaluating a claim as a whole, examiners should not dismiss additional elements as mere 'generic computer components' without considering whether such elements confer a technological improvement to a technical problem..."
The claimed memory unit is not generic; it is a specialized component configured to parse DAGs and arbitrate memory requests based on iteration depth (Claim 1). This specific configuration allows the computer to execute non-linear neural network operations more efficiently than a generic computer using standard First-In-First-Out (FIFO) or round-robin logic. Just as the claims in Desjardins were eligible because they addressed "catastrophic forgetting" in machine learning models, the present claims are eligible because they address execution stalls and cache contention in neural processing units. This is an improvement to the computer's own capability to process data.
For at least the foregoing reasons, under Step 2A Prong 2, the claims are not directed to a judicial exception, at least because each of the claims as a whole integrates the alleged judicial exception into a practical application and therefore, they qualify as patentable subject matter under 35 U.S.C. § 101.
a) Examiner respectfully disagrees. In response to applicant's argument that the computing components of the claimed invention are not generic, it is noted that the features upon which applicant relies (i.e., “a specialized component configured to parse DAGs”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Further, the arbitration is considered a mental process. MPEP 2106.05(f) states “For claim limitations that do not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer, examiners should explain why they do not meaningfully limit the claim in an eligibility rejection. For example, an examiner could explain that implementing an abstract idea on a generic computer, does not integrate the abstract idea into a practical application in Step 2A Prong Two or add significantly more in Step 2B, similar to how the recitation of the computer in the claim in Alice amounted to mere instructions to apply the abstract idea of intermediated settlement on a generic computer.” The claimed memory unit is a generic computing component used to apply the abstract idea of arbitration. The arguments have been considered but were not found to be persuasive.
V. Filoche does not teach a block identifier "representative of an iteration depth of block position within the task data" used for arbitration during execution. Filoche's IDs relate to the structure of the neural network model itself (which chunk contains which layers), whereas the claimed invention relates to the runtime iteration through the data operation space (looping through data blocks).
In addition, one of ordinary skill in the art would not combine Bokam and Filoche because the references address fundamentally different technical problems using different mechanisms.
The skilled person would not look to Filoche (network download optimization) to modify Bokam (processor execution scheduling) to arrive at the specific claimed mechanism of using iteration depth for memory arbitration. The claimed mechanism solves the specific problem of synchronizing independent sections of a graph that are processing data loops at different rates-a problem neither cited reference addresses.
For at least the foregoing reasons, the cited references do not teach or suggest all of the requirements of independent amended claim 1 and therefore do not render claim 1 obvious. Independent claims 19 and 20 include limitations similar to those of independent claim 1 and should therefore be patentable for at least the reasons set forth above with respect to claim 1.
The addition of Zonca; Kadri; Moon; Liu, Liqing; Bryan; Liu, Shaoli; Kumar; King; or Kibardin do not appear to provide teaching or suggestion of the limitations absent from Bokam and Filoche as discussed above with respect to independent claim 1. Claims 2-6 and 8-18 each depend either directly or indirectly from and therefore include the limitations of claim 1; claim 21 depends from and therefore includes the limitations of amended independent claim 20. Accordingly, it is respectfully submitted that each of these claims is also patentable over the cited references for at least the reasons set forth above with respect to the independent claim from which they depend. Further, each of these dependent claims place additional limitations on their parent and intermediate claims which, when considered in light of their parent, further distinguish the claimed invention from the cited references.
a) Applicant’s arguments with respect to claim(s) 1-6 and 8-21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Examiner respectfully requests, in response to this Office action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line number(s) in the specification and/or drawing figure(s). This will assist Examiner in prosecuting the application.
When responding to this Office Action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections. See 37 CFR 1.111(c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARI F RIGGINS whose telephone number is (571)272-2772. The examiner can normally be reached Monday-Friday 7:00AM-4:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached at (571) 272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.F.R./Examiner, Art Unit 2197
/KENNETH TANG/Primary Examiner, Art Unit 2197
Prosecution Insights