DETAILED ACTION
1. This office action is in response to the Application No. 18622228 filed on 12/10/2025. Claims 5, 7, 16 and 18 has been cancelled, claims 1-4, 6, 8-15, 17, 19 and 20 are presented for examination and are currently pending. Applicant’s arguments has been carefully considered.
Response to Arguments
2. On page 12 of the remarks, the Applicant argued that “Applicant's claim 1, as amended, the command processor is receiving the context start signal from a register written by the host to indicate the task descriptor for a task of the neural engine. That is, in Khatamifard, the data used to update the task descriptor for a next task is received by the task manager from the neural engine performing a completed task, not received by the task manager from the compiler (e.g., host) that initially generates the task descriptors for the neural engine”.
It is noted that the Applicant argument has been considered but are moot because a new secondary reference has been applied to teach the newly added limitations.
On page 12 of the remarks, the Applicant argued that “The Office Action alleges that updating the task descriptor for a next task based on the data received from the neural engine after a first task is finished implies reusing the task descriptor for the first task for the task descriptor for the next task. Khatamifard merely discloses updating the task descriptor for the next task based on some data. Khatamifard does not further explain what the data includes. Khatamifard does not disclose, teach, or suggest the data includes the task descriptor for the completed first task, let alone updating the task descriptor for the next task to reuse the task descriptor for the first task”.
On page 12-13 of the remarks, the Applicant argued that “The Office action further alleges that Khatamifard teaches "determining whether a plurality of task descriptors for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for the current context of the neural network model based on a location of a register receiving the context start signal that includes … a second register for receiving the context start signal, the second register indicating that the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused as the plurality of task descriptors for the current context of the neural network model," as recited in claim 1 prior to the amendments in this response”.
On page 13 of the remarks, the Applicant argued that “In particular, the Office Action cites to paragraph [0093] in Khatamifard for this teaching. Khatamifard in [0093] discloses "Upon execution of the task, task manager controller 610 may also update one or more parameters in the corresponding task descriptor with data 636 read from register(s) of neural engine 314 (e.g., register(s) of NE control 418). Data 636 received by task manager controller 610 may further include information that the execution of the task is done. Task manager controller 610 may update the corresponding task descriptor in task memory 606 with an address 630 of partial tensor data for usage by a next task represented by a next task descriptor”.
On page 13 of the remarks, the Applicant argued that “Applicant respectfully submits Khatamifard is just reiterating that the task manager may update the task descriptor for a next task based on some data received from the neural engine after a first task is finished. The Office Action relies on this almost identical teaching as in [0090] to imply that the task descriptor for the first task cannot be reused for the task descriptor for the next task. Applicant fails to see how the Office Action could come to two opposite conclusions based on almost identical teachings between paragraphs [0090] and [0093]”.
The above arguments are not persuasive because Khatamifard states on [0093] that “Task manager controller 610 may update the corresponding task descriptor in task memory 606 with an address 630 of partial tensor data for usage by a next task represented by a next task descriptor”. This implies that there is more than one task descriptor, so the next task descriptor is different from the previous descriptor. In fact, Khatamifard discloses further in [0095] that “Task memory 606 may store a list of task descriptors, e.g., task descriptors 602(1), 602(2), . . . ”. As a result, the next task is performed by next task descriptor indicates that the previous task descriptor is not allowed to be reused. This reads on “the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused as the plurality of task descriptors for the current context of the neural network model”.
The argument that “the Office Action could come to two opposite conclusions based on almost identical teachings between paragraphs [0090] and [0093]” is not persuasive because [0090] and [0093] are two different paragraphs in Khatamifard’s teaching. Secondly, [0090] of Khatamifard discloses “Additionally or alternatively, task memory 606 may pass second data 622 to the one or more control registers with information about a particular task descriptor in task memory 606 for configuring neural engine 314 to perform a next task associated with the particular task descriptor”. This indicates that the control registers have information about a particular task descriptor that can perform a next task that corresponds to the particular task descriptor. This citation is clearly different from [0093] which discloses that “Upon execution of the task, task manager controller 610 may also update one or more parameters in the corresponding task descriptor with data 636 read from register(s) of neural engine 314 (e.g., register(s) of NE control 418). Data 636 received by task manager controller 610 may further include information that the execution of the task is done. Task manager controller 610 may update the corresponding task descriptor in task memory 606 with an address 630 of partial tensor data for usage by a next task represented by a next task descriptor”, here execution of a task is completed that is, the task descriptor associated with the completed task is done and will not be reused.
On page 13 of the remarks, the Applicant argued that “Furthermore, Khatamifard merely teaches the task manager receiving data from one or more control registers of the neural engine to update a corresponding task descriptor for a next task. Khatamifard is silent about updating the task descriptor for the next task based on an identity of the control registers from which the data is received. That is, Khatamifard is silent about the task manager determining whether to update the task descriptor for the next task based on whether the data is received from a first register or a second register of the neural engine”.
The above argument is not persuasive because [0090] of Khatamifard discloses “Additionally or alternatively, task memory 606 may pass second data 622 to the one or more control registers with information about a particular task descriptor in task memory 606 for configuring neural engine 314 to perform a next task associated with the particular task descriptor”. Upon execution of the task, task manager controller 610 may also update one or more parameters in the corresponding task descriptor with data 636 read from register(s) of neural engine 314 (e.g., register(s) of NE control 418). This indicates that the registers have information about a particular task descriptor that can perform a next task that corresponds to the particular task descriptor.
On page 14 of the remarks, the Applicant argued that “Furthermore, Khatamifard discloses a passive mechanism where the task manager merely "reads" status data from one or more registers of a neural engine. In contrast, Applicant's claimed invention involves an active hardware mechanism where "the command processor monitors the register to detect the context start signal being written by the host system," where "a location" of the register being monitored is used to determine "whether a plurality of task descriptors for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for a current context of the neural network model”.
It is noted that the Applicant argument has been considered but are moot because a new secondary reference has been applied to teach the newly added limitations.
On page 15 of the remarks, the Applicant argued that “The Office Action points to reusing the same set of data DMA descriptors for multiple neural network operations in Minkin for teaching "determining whether a plurality of task descriptors for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for the current context of the neural network model,' as recited in claim 1”, “However, Minkin does not disclose, teach, or suggest determining whether to reuse the DMA descriptors based on a location of a register receiving a context start signal for the DMA operations”.
The Examiner agrees that Minkin teaches “determining whether a plurality of task descriptors for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for the current context of the neural network model” and “based on a determination on whether the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused for the plurality of task descriptors for the current context of the neural network model” as detailed in the office action but Minkin does not teach determining whether to reuse the DMA descriptors based on a location of a register receiving a context start signal for the DMA operations.
However, Khatamifard is used to teach determining whether a plurality of task descriptors for a previous context of the neural network model (During each computational cycle (e.g., clock cycle), task DMA 604 may receive data 602 including at least one task descriptor. Each task descriptor may identify a respective set of convolution operations of one or more tensor layers in a CNN [0088]; determine which subsets of data are transmitted to neural engines 314 … based on the task commands associated with different subsets of data [0066]. The Examiner notes a determination is made whether data comprising task descriptors generated from task command should be transmitted; instant specification discloses “first task descriptor based on the at least one task generated from the one or more first commands” [0015]) are allowed to be reused for a plurality of task descriptors for a current context of the neural network model (Additionally or alternatively, task memory 606 may pass second data 622 to the one or more control registers with information about a particular task descriptor in task memory 606 for configuring neural engine 314 to perform a next task associated with the particular task descriptor [0090]. The Examiner notes that a particular task descriptor performing a next task implies the particular task descriptor is allowed to be reused) based on a location of a register receiving the context start signal includes a first register (… registers in neural engine 314 (e.g., register(s) in NE control 418) [0096]. The Examiner notes the registers include a first register)
On page 15 of the remarks, the Applicant argued that “However, just because there are a large number of DMA descriptors for the neural network operation does not imply Minkin discloses, teaches, or suggests that the task descriptors for a previous context or task of the neural network operation cannot be reused for the current context or task of the neural network operation”.
The above argument is not persuasive because Minkin teaches based on a determination on whether the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused for the plurality of task descriptors for the current context of the neural network model (As a neural network operation typically involves a large volume of input and output data, while the transfer data size of each DMA descriptor is limited, neural network processor 302 may need to create a large number of data DMA descriptors 320 and 322 to support the transfer of input data and output data for the neural network operation. The creation of the large number of the DMA descriptors 320 and 322, as well as the insertion of the large number of DMA descriptors into descriptor queues 304a and 304b, can add substantial delay to the neural network operation, col. 12, lines 37-46. The Examiner notes that “creation of the large number of the DMA descriptors 320 and 322” indicates that these newly created DMA descriptors will be used in the current context instead of the plurality of DMA descriptors for the previous context of the neural network model, which are not allowed to be reused),
On page 15-16 of the remarks, the Applicant argued that “Furthermore, Minkin merely discloses a DMA engine that fetches descriptors from a queue to support neural network operations. Minkin does not disclose, teach, or suggest a mechanism where the DMA engine or a processor monitors specific physical register locations to instantaneously trigger a reuse decision based on a host's write operation. In contrast, Applicant's claimed invention recites a mechanism where "the command processor monitors the register to detect the context start signal being written by the host system," where "a location" of the register being monitored is used to determine "whether a plurality of task descriptors for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for a current context of the neural network model.”
As a indicated above, a new secondary reference has been applied to map the newly added limitations.
On page 16 of the remarks, the Applicant argued that “Cui, Musleh, and Garfinkel do not cure the deficiencies of Khatamifard and Minkin. Applicant cannot find any disclosure, teaching, or suggestion in Cui, Musleh, or Garfinkel for "determining whether a plurality of task descriptors for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for the current context of the neural network model based on a location of a register receiving the context start signal," as recited in claim 1. Because the cited references, alone or in combination, do not teach all the limitations of amended claimed claim 1, Applicant respectfully submits that amended claim 1 is allowable over the cited references”.
It is noted that Khatamifard, Minkin and Mohapatra has now been applied to the amended independent claims. Cui, Musleh, and Garfinkel are used to these the dependent claims.
The Applicant also argued on page 16 that “Claim 12 as amended recites similar distinguishing features as in amended claim 1. Applicant respectfully submits that claim 12 is allowable for at least similar reasons to those for which claim 1 is allowable.”
The above argument is not persuasive because Khatamifard, Minkin and Mohapatra as now being rendered obvious the limitations of claim 1. The same reasoning applies to claim 12.
On page 16 of the remarks, the Applicant argued that “As each of the dependent claims depends from a base claim that is believed to be in condition for allowance, Applicant does not believe that it is necessary to argue the allowability of each of these claims individually. Applicant does not necessarily concur with the interpretation of these claims, or with the bases for rejections set forth in the Office Action. Applicant therefore reserves the right to address the patentability of these claims individually as necessary in the future. Accordingly, Applicant respectfully requests reconsideration and withdrawal of the rejections under 35 U.S.C. §103”.
The Examiner notes dependent claims 2-4, 6, 8-11, 13-15, 17, 19 and 20, which depend directly or indirectly from claims 1 and 12 are not allowable because the Applicant’s argument are not persuasive for similar reasons argued above regarding claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 1-4, 6, 8, 12-15, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Khatamifard et al. (US20240095541 filed 09/16/2022) in view of Minkin et al. (US11868872 filed 03/31/2020) and further in view of Mohapatra et al. (US20200134417)
Regarding claim 1, Khatamifard teaches an apparatus comprising: one or more neural processors configured to perform neural network model tasks (When compiler 240 is executed, for example, by CPU 280, a task schedule listing the task descriptors of tasks to be performed by neural engines in neural processor circuit 218 is generated [0048]. The Examiner notes neural processor circuit 218 is a neural processor);
a command processor configured to distribute neural network model tasks to the one or more neural processors (Neural task manager 310 manages the overall operation of neural processor circuit 218. Neural task manager 310 may receive a task list from a compiler executed by CPU 208 … Neural task manager 310 may receive a task list from a compiler executed by CPU 208 … Neural task manager 310 manages the overall operation of neural processor circuit 218 [0061]. The Examiner notes CPU 208 is the command processor); and
a shared memory shared by the one or more neural processors (Data of neural processor circuit 218 includes input data that is transmitted from another source such as system memory 230 [0061]; Fig. 2.; … software components executed on neural processor circuit 218, ISP 206, CPU 208 or GPU 220. Such software components may be stored in system memory 230 [0049]. The Examiner notes the system memory 230 is a shared memory between neural processor circuit 218, ISP 206, CPU 208 or GPU 220),
wherein the command processor (CPU 208 may be embodied using any suitable instruction set architecture, and may execute instructions defined in that instruction set architecture [0040]) is configured to cause: in response to receiving a context start signal indicating a start of a context of a neural network model from a host system ( Neural task manager 310 may receive a task list from a compiler executed by CPU 208 [0061]; FIG. 7 is a block diagram of compiler 240 that may be implemented at a processor, such as a system processor of a device, e.g., CPU 208 … of device 100 [0097]), directly accessing a memory in the host system to read neural network model data for the context of the neural network model (device 100 may include … a persistent storage (e.g., flash memory) 228 [0031]; Neural processor circuit 218 may receive the input data from … persistent storage 228 [0042]; Persistent storage 228 may also store one or more machine learning models, such as … artificial neural networks (ANNs) … The machine learning models may perform various tasks such as … context analysis [0036]. The Examiner notes persistent storage 228 is a memory in host device 100),
wherein the context start signal (Task manager controller 610 may be responsible for starting execution of a task, and for updating a corresponding task descriptor in task memory 606 after the task is executed [0092]) comprises an update index field (To start executing the task, task manager controller 610 may read an index (or task index) [0092]) and an update value field (Alternatively, some of the fields of task descriptor 602(n) may be grouped into a single field in task descriptor 602(n). Input size ID 604(n) may identify a size of a partial input tensor to be used for a next subset of convolution operations on a corresponding portion of the n-th layer [0096]. The Examiner notes Input size ID 604(n) is the update value field),
wherein the update index field indicates an information field to be updated … (Then, task manager controller 610 may generate, using index, a read address 628 for reading corresponding data 622 from the corresponding task descriptor in task memory 606 identified by index. Data 622 read from the corresponding task descriptor may be used to update the one or more control registers of neural engine 314 (e.g., register(s) of NE control 418) to prepare neural engine 314 for execution of the task [0092]) and the update value field indicates a value to be updated of the information field indicated by the update index field (Task descriptor 602(n) may include an input size identifier (ID) 604(n), … Task descriptor 602(n) may include some additional fields not shown in FIG. 6B. Alternatively, some of the fields of task descriptor 602(n) may be grouped into a single field in task descriptor 602(n). Input size ID 604(n) may identify a size of a partial input tensor to be used for a next subset of convolution operations on a corresponding portion of the n-th layer [0096]. The Examiner notes Input size ID 604(n) is the update value field);
determining whether a plurality of task descriptors for a previous context of the neural network model (During each computational cycle (e.g., clock cycle), task DMA 604 may receive data 602 including at least one task descriptor. Each task descriptor may identify a respective set of convolution operations of one or more tensor layers in a CNN [0088]; determine which subsets of data are transmitted to neural engines 314 … based on the task commands associated with different subsets of data [0066]. The Examiner notes a determination is made whether data comprising task descriptors generated from task command should be transmitted; instant specification discloses “first task descriptor based on the at least one task generated from the one or more first commands” [0015]) are allowed to be reused for a plurality of task descriptors for a current context of the neural network model (Additionally or alternatively, task memory 606 may pass second data 622 to the one or more control registers with information about a particular task descriptor in task memory 606 for configuring neural engine 314 to perform a next task associated with the particular task descriptor [0090]. The Examiner notes that a particular task descriptor performing a next task implies the particular task descriptor is allowed to be reused) based on a location of a register receiving the context start signal includes a first register (… registers in neural engine 314 (e.g., register(s) in NE control 418) [0096]. The Examiner notes the registers include a first register) for receiving the context start signal indicating that the plurality of task descriptors for the previous context of the neural network model are allowed to be reused (Task memory 606 may receive first data 622 from the one or more control registers, and use received first data 622 to update a corresponding task descriptor in task memory 606 after each task is finished [0090]) as the plurality of task descriptors for the current context of the neural network model (Additionally or alternatively, task memory 606 may pass second data 622 to the one or more control registers with information about a particular task descriptor in task memory 606 for configuring neural engine 314 to perform a next task associated with the particular task descriptor [0090]. The Examiner notes that a particular task descriptor performing a next task implies the particular task descriptor is allowed to be reused) and
second register (… registers in neural engine 314 (e.g., register(s) in NE control 418) [0096]. The Examiner notes the registers include a second register) for receiving the context start signal indicating that the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused as the plurality of task descriptors for the current context of the neural network model (Upon execution of the task, task manager controller 610 may also update one or more parameters in the corresponding task descriptor with data 636 read from register(s) of neural engine 314 (e.g., register(s) of NE control 418). Data 636 received by task manager controller 610 may further include information that the execution of the task is done. Task manager controller 610 may update the corresponding task descriptor in task memory 606 with an address 630 of partial tensor data for usage by a next task represented by a next task descriptor [0093]. The Examiner notes that the next task is performed by next task descriptor implies the previous task descriptor is not allowed to be reused);
based on a determination on whether the plurality of task descriptors for the previous context of the neural network model (During each computational cycle (e.g., clock cycle), task DMA 604 may receive data 602 including at least one task descriptor. Each task descriptor may identify a respective set of convolution operations of one or more tensor layers in a CNN [0088]; determine which subsets of data are transmitted to neural engines 314 … based on the task commands associated with different subsets of data. The Examiner notes a determination is made whether data comprising task descriptors generated from a task command should be transmitted; instant specification discloses “first task descriptor based on the at least one task generated from the one or more first commands” [0015]) are not allowed to be reused for the plurality of task descriptors for the current context of the neural network model (Upon execution of the task, task manager controller 610 may also update one or more parameters in the corresponding task descriptor with data 636 read from register(s) of neural engine 314 (e.g., register(s) of NE control 418). Data 636 received by task manager controller 610 may further include information that the execution of the task is done. Task manager controller 610 may update the corresponding task descriptor in task memory 606 with an address 630 of partial tensor data for usage by a next task represented by a next task descriptor [0093]. The Examiner notes that the next task performed by next task descriptor implies previous task descriptor is not allowed to be reused),
generating the plurality of task descriptors for the current context of the neural network model (In some embodiments, compiler 240 generates the list of task descriptors [0087]; … Each task descriptor may identify a respective set of convolution operations of one or more tensor layers in a CNN [0088]); and
distributing the plurality of task descriptors for the current context of the neural network model to the one or more neural processors so that the one or more neural processors perform tasks described by the plurality of task descriptors for the current context of the neural network model (Tasks descriptor generator 712 generates task descriptors for each task. During runtime, the tasks descriptor may be loaded to neural task manager 310 in order to configure components of neural processor circuit 218 to perform neural network operations in a streaming manner [0101]).
Khatamifard is silent about a primary context descriptor and a secondary context descriptor.
Minkin teaches primary context descriptor and a secondary context descriptor (The first subset of the data DMA descriptors can be configured to transfer data from the first memory, and each of the first subset of the data DMA descriptors may include an indirect address as a source address which can be translated into a physical address of the first memory. The second subset of the data DMA descriptors can be configured to transfer data to the first memory, and each of the second subset of the data DMA descriptors may include an indirect address as a destination address which can be replaced by a physical address of the first memory (col., 5, lines 4-14). The Examiner notes that first subset of the data DMA descriptors is the primary context descriptor and second subset of the data DMA descriptors is secondary context descriptor)
Minkin also teaches determining whether a plurality of task descriptors for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for the current context of the neural network model (After the first neural network operation completes and prior to a second neural network operation, the processor can be triggered by the pre-determined command in second instructions of the second neural network operation to program the address translation table with a second address mapping (col. 5, lines 59-64); Each of the data DMA descriptors can be statically stored in the data DMA descriptor queue and can be reused for DMA operations of a subsequent neural network operation after being fetched to the DMA engine (col. 5, lines 1-4); As a result, the same set of data DMA descriptors can be reused to support multiple neural network operations without the need to create and manage a different set of data DMA descriptors for a different neural network operation (col. 6, lines 4-7). The Examiner notes this implies a data DMA descriptor might be reused “without the need to create and manage a different set of data DMA descriptors”);
based on a determination on whether the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused for the plurality of task descriptors for the current context of the neural network model (As a neural network operation typically involves a large volume of input and output data, while the transfer data size of each DMA descriptor is limited, neural network processor 302 may need to create a large number of data DMA descriptors 320 and 322 to support the transfer of input data and output data for the neural network operation. The creation of the large number of the DMA descriptors 320 and 322, as well as the insertion of the large number of DMA descriptors into descriptor queues 304a and 304b, can add substantial delay to the neural network operation, col. 12, lines 37-46. The Examiner notes that “creation of the large number of the DMA descriptors 320 and 322” indicates that these newly created DMA descriptors will be used in the current context instead of the plurality of DMA descriptors for the previous context of the neural network model, which are not allowed to be reused),
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Khatamifard to incorporate the teachings of Minkin in order to avoid burdening the host processor and the neural network processor with memory access operations during an inferencing operation and a training operation that typically involves a large number of memory access operations (Minkin, col. 11, lines 1-6)
Modified Khatamifard does not explicitly teach wherein the command processor monitors the register to detect the context start signal being written by the host system, the first register being written by the host system to indicate to the command processor, the second register being written by the host system to indicate to the command processor,
Mohapatra teaches determining whether a plurality of task descriptors (As such, the configuration loader 122 is an example of means for determining and/or writing/loading descriptors [0026]) for a previous context of the neural network model are allowed to be reused for a plurality of task descriptors for a current context of the neural network model (In the illustrated example, the set of descriptor fields applied to the configuration registers 170 are programmed via the configuration loader 122 to implement a dataflow schedule, based on a tensor processing template, to process the IF and FL tensor data for a current layer (L) of the convolutional neural network being implemented [0056]) based on a location of a register receiving the context start signal (As shown in the illustrated example, respective ones of the PEs 105 a-i include example register file (RF) local storage 145 a-c to store IF, FL and OF tensor data, respectively, for that PE [0040]. The Examiner notes the register file (RF) local storage 145 a-c is a location of a register),
wherein the command processor monitors the register (As such, the configuration loader 122 is an example of means for determining and/or writing/loading descriptors into the configuration registers 120 … In some examples, the configuration loader 122 is implemented by one or more processors [0026]. The Examiner notes determining when to write/load descriptors into the registers indicates monitoring the register) to detect the context start signal being written by the host system (the example program 2600 of FIG. 26 begins execution at block 2605 at which the configuration loader 122 executes instructions (e.g., a compiler, software, etc.) to load input data (IF data) and filter data (FL data) corresponding to a convolutional neural network to be implemented by the configurable processor element array 100 [0086]; As mentioned above, the example process of FIG. 26 may be implemented using executable instructions (e.g., computer)[0083]. The Examiner notes detecting when to load input data (IF data) and filter data (FL data) at the beginning of execution is the context start signal detected, and the computer that implements process of Fig 26 is a host system),
wherein the register includes a first register for receiving the context start signal (he example PE 105 a of FIG. 6 includes RF local storage 145 a-c. The illustrated example includes three RFs 145 a-c for storing IF, FL and OF tensor data, respectively … In the illustrated example, the tensor data stored in IF and FL RFs 145 a-b are 8 bits wide [0059]. The Examiner notes RFs 145a in Fig. 6 is the first register for receiving the context start signal IF data), the first register being written by the host system (The TotalWrIFRF descriptor field indicates how many IF data points are to be written to a PE 105 a-i [0043]) to indicate to the command processor that the plurality of task descriptors for the previous context of the neural network model are allowed to be reused as the plurality of task descriptors for the current context of the neural network model (For example, the new set of descriptors can be the same as the prior set of descriptors applied to the configuration registers 120 [0025]),
wherein the register further includes a second register for receiving the context start signal (In the illustrated example, the tensor data stored in IF and FL RFs 145 a-b are 8 bits wide [0059]. The Examiner notes RF 145b is the second register for receiving the context start signal FL data),
the second register being written by the host system (The TotalWrIFRF descriptor field indicates how many IF data points are to be written to a PE 105 a-i [0043]) to indicate to the command processor that the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused as the plurality of task descriptors for the current context of the neural network model (For example, the new set of descriptors can be the same as the prior set of descriptors applied to the configuration registers 120 [0025]);
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Khatamifard to incorporate the teachings of Mohapatra for the benefit of developing DNN accelerators that exploit energy efficiency from data reuse (Mohapatra [0099])
Regarding claim 2, Modified Khatamifard teaches the apparatus of claim 1, Minkin teaches wherein generating the plurality of task descriptors for the current context of the neural network model comprises (To support the DMA operations to fetch the input data from the host memory to the local memory, the host processor that generates the request for a neural network operation can create a set of input data DMA descriptors each including a host input physical address of a portion of the input data in the host memory, a size of the portion of the input data, and a destination location of the portion of the input data in the local memory, col. 3, lines 21-29):
based on a determination that the plurality of task descriptors for the previous context of the neural network model are not allowed to be reused for the plurality of task descriptors for the current context of the neural network model (As a neural network operation typically involves a large volume of input and output data, while the transfer data size of each DMA descriptor is limited, neural network processor 302 may need to create a large number of data DMA descriptors 320 and 322 to support the transfer of input data and output data for the neural network operation. The creation of the large number of the DMA descriptors 320 and 322, as well as the insertion of the large number of DMA descriptors into descriptor queues 304a and 304b, can add substantial delay to the neural network operation (col. 12, lines 37-46). The Examiner notes that “creation of the large number of the DMA descriptors 320 and 322” indicates that these newly created DMA descriptors will be used in the current context instead of the plurality of DMA descriptors for the previous context of the neural network model, which are not allowed to be reused),
generating the plurality of task descriptors for the current context of the neural network model based on the neural network model data (Host processor 410 can generate address mapping DMA descriptors 522 each including a physical address where the address mapping data are stored, and store the address mapping DMA
descriptor queue 502. To support a first neural network operation, neural network processor 402 can execute a pre-determined instruction (e.g., instruction 536) which triggers the neural network processor to send a signal to DMA engine 406 to fetch an input address mapping DMA descriptor and an output address mapping DMA descriptor, col. 19, lines 54-63); and
based on a determination that the plurality of task descriptors for the previous context of the neural network model are allowed to be reused for the plurality of task descriptors for the current context of the neural network model (After the first neural network operation completes and prior to a second neural network operation, the processor can be triggered by the predetermined command in second instructions of the second neural network operation to program the address translation table with a second address mapping (col. 5, lines 59-64); As a result, the same set of data DMA descriptors can be reused to support multiple neural network operations without the need to create and manage a different set of data DMA descriptors for a different neural network operation, col. 6, lines 4-7. The Examiner notes this implies a data DMA descriptor might be reused “without the need to create and manage a different set of data DMA descriptors”),
reading the plurality of task descriptors for the previous context of the neural network model from the shared memory (In the example of FIG. 4B, address translation table 405 can be part of a data bus between DMA engine 406 and host memory 412. In such example, DMA engine 406 can fetch input data DMA descriptors 420 and output data DMA descriptors 422 and issue a memory access request (e.g., a read request, a write request, etc.) to host memory 412 and local memory 408 based on the fetched data DMA descriptors, col 14, lines 45-52) and
generating the plurality of task descriptors for the current context of the neural network model based on the plurality of task descriptors for the previous context of the neural network model (Each of the data DMA descriptors can be statically stored in the data DMA descriptor queue and can be reused for DMA operations of a subsequent neural network operation after being fetched to the DMA engine, col. 5, lines 1-4).
The same motivation to combine independent claim 1 applies here.
Regarding claim 3, Modified Khatamifard teaches the apparatus of claim 2, Minkin teaches wherein generating the plurality of task descriptors for the current context of the neural network model based on the neural network model data (Host processor 410 can generate address mapping DMA descriptors 522 each including a physical address where the address mapping data are stored, and store the address mapping DMA descriptor queue 502. To support a first neural network operation, neural network processor 402 can execute a pre-determined instruction (e.g., instruction 536) which triggers the neural network processor to send a signal to DMA engine 406 to fetch an input address mapping DMA descriptor and an output address mapping DMA descriptor, col. 19, lines 54-63) comprises:
storing the plurality of task descriptors for the current context of the neural network model in the shared memory for a next context of the neural network model (Based on each of input data DMA descriptors 320, DMA engine 306 can perform a DMA operation to fetch an input data portion (e.g., in0, in1, in2, etc.) from the host input physical addresses of host memory 312, and then store the input data portions at the local input addresses of local memory 308, in operations labelled "s2" in FIG. 3.(col. 12, lines 12-17); The DMA engine can then perform DMA operations to support the neural network operation, col. 5, lines 35-37)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Khatamifard to incorporate the teachings of Minkin in order to avoid burdening the host processor and the neural network processor with memory access operations during an inferencing operation and a training operation that typically involves a large number of memory access operations (Minkin, col. 11, lines 1-6)
Regarding claim 4, Modified Khatamifard teaches the apparatus of claim 2, Minkin teaches wherein generating the plurality of task descriptors for the current context of the neural network model based on the plurality of task descriptors for the previous context of the neural network model (Each of the data DMA descriptors can be statically stored in the data DMA descriptor queue and can be reused for DMA operations of a subsequent neural network operation after being fetched to the DMA engine (col. 5, lines 1-4) comprises:
setting the plurality of task descriptors for the current context of the neural network model equal to the plurality of task descriptors for the previous context of the neural network model (To support the second neural network operation, the processor has to control the DMA engine to fetch and execute the same set of data DMA descriptors, col. 5, lines 64-67).
The same motivation to combine dependent claim 2 applies here.
Regarding claim 6, Modified Khatamifard teaches the apparatus of claim 1, Minkin teaches wherein whether the plurality of task descriptors for the previous context of the neural network model are allowed to be reused for the plurality of task descriptors for the current context of the neural network model is determined based on an indication included in the context start signal (the processor can be triggered by a pre-determined command in first instructions of the first neural network operation to program the address translation table with a first address mapping for the first inferencing operation, col. 5, lines 48-52).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Khatamifard to incorporate the teachings of Minkin in order to avoid burdening the host processor and the neural network processor with memory access operations during an inferencing operation and a training operation that typically involves a large number of memory access operations (Minkin, col. 11, lines 1-6)
Regarding claim 8, Modified Khatamifard teaches the apparatus of claim 1, Minkin teaches wherein the command processor is further configured to cause: receiving, from the one or more neural processors (The host processor can be triggered to program the address translation table based on, for example, executing a pre-determined command in the instructions of the neural network operation, col. 5, lines 44-47),
task completion signals indicating completion of tasks described by the plurality of task descriptors for the current context of the neural network model (After the first neural network operation completes, neural network processor 402 can execute another one of the pre-determined instructions to fetch another input address mapping DMA descriptor and another output address mapping DMA descriptor, and perform DMA operations to fetch a second set of address mapping data (e.g., address mappings 542 and 544) from host memory 412, col. 20, lines 1-7); and
transmitting, to the host system, a context completion signal indicating completion of the current context of the neural network model in response to receiving the task completion signals (After the inferencing operation completes, the output data can be fetched back to the host memory and to the application, col. 1, lines 30-32).
The same motivation to combine independent claim 1 applies here.
Regarding claim 12, claim 12 is similar to claim 1. It is rejected in the manner and reasoning applying.
Regarding claim 13, claim 13 is similar to claim 2. It is rejected in the manner and reasoning applying.
Regarding claim 14, claim 14 is similar to claim 3. It is rejected in the manner and reasoning applying.
Regarding claim 15, claim 15 is similar to claim 4. It is rejected in the manner and reasoning applying.
Regarding claim 17, claim 17 is similar to claim 6. It is rejected in the manner and reasoning applying.
Regarding claim 19, claim 19 is similar to claim 8. It is rejected in the manner and reasoning applying.
4. Claims 9 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Khatamifard et al. (US20240095541 filed 09/16/2022) in view of Minkin et al. (US11868872 filed 03/31/2020) in view of Mohapatra et al. (US20200134417)
and further in view of Cui et al. (US20200403919 filed 09/03/2020)
Regarding claim 9, Modified Khatamifard teaches apparatus of claim 1, Minkin teaches wherein directly accessing a memory in the host system to read neural network model data for the context of the neural network model (The application that generates the inference/training request typically executes on a host processor and stores input data (e.g., image data) at a host memory. The input data can be fetched to the local memory of the neural network hardware accelerator as inputs to neural network computations, col. 3, lines 2-6) comprises:
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Khatamifard to incorporate the teachings of Minkin in order to avoid burdening the host processor and the neural network processor with memory access operations during an inferencing operation and a training operation that typically involves a large number of memory access operations (Minkin, col. 11, lines 1-6)
Modified Khatamifard does not explicitly teach in response to receiving the context start signal, directly accessing the memory in the host system to read one or more context descriptors; and directly accessing the memory in the host system based on the one or more context descriptors to read the neural network model data.
Cui teaches in response to receiving the context start signal, directly accessing the memory in the host system to read one or more context descriptors (For example, host interfaces 420 and 430 can perform direct memory access (DMA) operations to copy data, descriptors, context, or other metadata from network controller 402 to platform 450 or vice versa [0040]); and
directly accessing the memory in the host system based on the one or more context descriptors to read the neural network model data (logic within the processor, which when read by a machine, computing device or system causes the machine, computing device or system to fabricate logic to perform the techniques described herein [0078]; Multiple neural networks, processor cores, or graphics processing units can be made available for use by AI or ML models [0060]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Khatamifard to incorporate the teachings of Cui for the benefit of any number of factors, such as desired computational rate, power levels, heat tolerances, processing cycle budget, input data rates, output data rates, memory resources, data bus speeds and other design or performance constraints, as desired for a given implementation (Cui [0075])
Regarding claim 20, claim 20 is similar to claim 9. It is rejected in the manner and reasoning applying.
5. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Khatamifard et al. (US20240095541 filed 09/16/2022) in view of Minkin et al. (US11868872 filed 03/31/2020) in view of Mohapatra et al. (US20200134417)
in view of Cui et al. (US20200403919 filed 09/03/2020) and further in view of Musleh et al. (US20210359955 filed 07/23/2021)
Regarding claim 10, Modified Khatamifard teaches the apparatus of claim 9, Minkin teaches wherein directly accessing the memory in the host system to read the one or more context descriptors (The application that generates the inference/training request typically executes on a host processor and stores input data (e.g., image data) at a host memory. The input data can be fetched to the local memory of the neural network hardware accelerator as inputs to neural network computations, col. 3, lines 2-6) comprises:
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Khatamifard to incorporate the teachings of Minkin in order to avoid burdening the host processor and the neural network processor with memory access operations during an inferencing operation and a training operation that typically involves a large number of memory access operations (Minkin, col. 11, lines 1-6)
Cui teaches wherein directly accessing the memory in the host system based on the one or more context descriptors to read the neural network model data (… logic within the processor, which when read by a machine, computing device or system causes the machine, computing device or system to fabricate logic to perform the techniques described herein [0078]; Multiple neural networks, processor cores, or graphics processing units can be made available for use by AI or ML models [0060]) comprises:
The motivation to combine dependent claim 9 applies here.
Modified Khatamifard does not explicitly teach determining an address of the primary context descriptor based on the context start signal; directly accessing the memory in the host system based on the address of the primary context descriptor to read the primary context descriptor; determining an address of the secondary context descriptor based on the primary context descriptor; and directly accessing the memory in the host system based on the address of the secondary context descriptor to read the secondary context descriptor, directly accessing the memory in the host system based on the secondary context descriptor to read the neural network model data.
Musleh teaches determining an address of a primary context descriptor based on the context start signal; directly accessing the memory in the host system based on the address of the primary context descriptor to read the primary context descriptor (Descriptor queues 1120 can include descriptors that reference data or packets in transmit queue 1106 or receive queue 1108 [0061]. The Examiner notes descriptors that reference data or packets in transmit queue 1106 is a primary context descriptor);
determining an address of the secondary context descriptor based on the primary context descriptor (Descriptor queues 1120 can include descriptors that reference data or packets in transmit queue 1106 or receive queue 1108 [0061]. The Examiner notes descriptors that reference data or packets in transmit queue 1106 is a primary context descriptor and descriptors that reference data or packets in receive queue 1108 is the secondary descriptor); and
directly accessing the memory in the host system based on the address of the secondary context descriptor to read the secondary context descriptor (Direct memory access (DMA) engine 1152 can copy a packet header, packet payload, and/or descriptor directly from host memory to the network interface or vice versa [0060]), and
directly accessing the memory in the host system based on the secondary context descriptor to read the neural network model data (Direct memory access (DMA) engine 1152 can copy a packet header, packet payload, and/or descriptor directly from host memory to the network interface or vice versa [0060]; Multiple neural networks, processor cores, or graphics processing units can be made available for use by AI or ML models [0064]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Khatamifard to incorporate the teachings of Musleh for the benefit of allocating a region in a cache to store a context of a connection based on application-specified priority level (Musleh abstract)
6. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Khatamifard et al. (US20240095541 filed 09/16/2022) in view of Minkin et al. (US11868872 filed 03/31/2020) in view of Mohapatra et al. (US20200134417)
in view of Cui et al. (US20200403919 filed 09/03/2020) in view of Musleh et al. (US20210359955 filed 07/23/2021) and further in view of Gurfinkel et al. (US20230005096)
Regarding claim 11, Modified Khatamifard teaches the apparatus of claim 10, Minkin teaches wherein directly accessing the memory in the host system based on the secondary context descriptor to read the neural network model data (Direct memory access (DMA) engine 1152 can copy a packet header, packet payload, and/or descriptor directly from host memory to the network interface or vice versa [0060]; Multiple neural networks, processor cores, or graphics processing units can be made available for use by AI or ML models [0064]) comprises:
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Khatamifard to incorporate the teachings of Minkin in order to avoid burdening the host processor and the neural network processor with memory access operations during an inferencing operation and a training operation that typically involves a large number of memory access operations (Minkin, col. 11, lines 1-6)
Modified Khatamifard does not explicitly teach directly accessing the memory in the host system based on the secondary context descriptor to read parameter data for the neural network model and to store the parameter data into the shared memory; directly accessing the memory in the host system based on the secondary context descriptor to read input data for the neural network model and to store the input data into the shared memory; and directly accessing the memory in the host system based on the secondary context descriptor to read binary code data for the neural network model and to store the binary code data into the shared memory.
Gurfinkel teaches directly accessing the memory in the host system (In at least one embodiment, when said graph launches, said graph must also wait for events such that said graph has exclusive access to physical memory [0060])
based on the secondary context descriptor to read parameter data for the neural network model (A work descriptor (“WD”) 1984 contained in process element 1983 can be a single job requested by an application or may contain a pointer to a queue of jobs [0267]; In at least one embodiment, tensor cores are configured to perform deep learning matrix arithmetic, such as convolution operations for neural network training and inferencing [0371]) and
to store the parameter data into the shared memory (wherein the one or more processors are further to perform the API based at least in part on a set of parameter values indicating at least a size of the memory to be allocated [0517]); and
directly accessing the memory in the host system based on the secondary context descriptor to read input data for the neural network model (In at least one embodiment, a set of threads executing across a set of parallel processing engines is a thread group. In at least one embodiment, a thread group executes a program on different input data [0303]) and
to store the input data into the shared memory (In at least one embodiment, memory and cache interconnect 2268 is a crossbar interconnect that allows LSU 2266 to implement load and store operations between shared memory 2270 and register file 2258 [0312]); and
directly accessing the memory in the host system based on the secondary context descriptor to read binary code data for the neural network model and to store the binary code data into the shared memory (In at least one embodiment, host executable code 3502 and device executable code 3503 may be in any suitable format, such as binary code … In the case of CUDA, host executable code 3502 [0412]; In at least one embodiment, CUDA device executable code 3884 includes, without limitation, binary code [0446]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Khatamifard to incorporate the teachings of Gurfinkel for the benefit of accelerating High Performance Computing (“HPC”) (Gurfinkel [0349])
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MORIAM MOSUNMOLA GODO whose telephone number is (571)272-8670. The examiner can normally be reached Monday-Friday 8am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T Bechtold can be reached on (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.G./Examiner, Art Unit 2148
/MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148