Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment/Argument
Applicant’s arguments filed on March 18, 2026 regarding the rejection under 35 U.S.C. 103 has been fully consider but is not persuasive.
Applicant argues that Shafiee does not teach or suggest “the right subnetwork is configured to operate in response to a conditional trigger,” as recited in claim 1, and therefore contends that claim 1 is patentable over Shafiee.
Applicant’s arguments are not persuasive. Claim 1 does not require any particular hardware trigger mechanism, explicit trigger signal, or dedicated trigger circuitry. Instead, claim 1 broadly recites that “the right subnetwork is configured to operate in response to a conditional trigger.” Under the broadest reasonable interpretation, a “conditional trigger” encompasses any configuration in which operation of the right subnetwork occurs only when a condition is satisfied. Applicant’s arguments improperly attempt to narrow the claim by requiring a particular hardware trigger implementation that is not recited in the claim language.
As explained in the rejection, Shafiee teaches operation gating based on buffer content. Shafiee states that operations for the subsequent layer begin only when there is “enough information” in the buffer, and further teaches that each additional operation occurs only when new outputs from the preceding layer are appended to the buffer. Thus, the subsequent layer operates only when a condition is based on buffer contents is met.
Under the broadest reasonable interpretation, operation that is enabled only when a buffer-content condition is satisfied reasonably corresponds to operation “in response to a conditional trigger,” as recited. Accordingly, Applicant’s arguments do not overcome the rejection.
Applicant argues that claim 6 is patentable because claim 1 is patentable and further argues that the cited references do not teach or suggest that the “right subnetwork is configurable to dynamically operate at different frequencies.”
Applicant’s argument is not persuasive. Claim 6 does not require any particular dynamic frequency-scaling circuitry, runtime clock-control mechanism, or specific reconfiguration protocol. Instead, claim 6 broadly recites that “the right subnetwork is configurable to dynamically operate at different frequencies.” Under the broadest reasonable interpretation, operation at different frequencies based on configuration or operational conditions reasonably encompasses dynamic operation at different frequencies. Applicant’s arguments improperly attempt to narrow the claim by requiring a particular implementation that is not recited in the claim language.
As explained in the rejection, Timofejevs teaches that different functional portions of the neural network may operate at different activation frequencies, including implementations in which some blocks operate at one frequency while other blocks operate at another frequency. Timofejevs further teaches that activation frequencies may depend on the application or operational conditions. Thus, Timofejevs teaches subnetworks configurable to operate at different frequencies under differing operating conditions.
Under the broadest reasonable interpretation, operation at different frequencies based on configuration or operational conditions reasonably corresponds to a subnetwork configurable to “dynamically operate at different frequencies,” as recited. Accordingly, Applicant’s arguments do not overcome the rejection.
Applicant argues that claims 2-5 and 9-11 are patentable because these claims depend from patentable claim 1.
Applicant’s arguments are not persuasive for at least the reasons discussed above with respect to claim 1. As explained above, the rejection of claim 1 is maintained because the cited references teach or suggest the disputed limitations under the broadest reasonable interpretation of the claim language. Claims 2-5 and 9-11 depend from claim 1 and were separately rejection in the Office Action with additional findings specific to the respective dependent limitations. Applicant has not separately argued the additional limitations of claims 2-5 and 9-11. Accordingly, Applicant’s arguments do not overcome the rejection of claims 2-5 and 9-11.
Applicant argues that claim 7 and 8 are patentable because these claims depend from patentable claim 1 and further argues that Govea does not teach or suggest the allegedly missing features from Shafiee and Timofejevs.
Applicant’s arguments are not persuasive. As discussed above, claim 1 is not patentable over the combination of Shafiee and Timofejevs. Accordingly, Applicant’s arguments relying on the patentability of claim 1 are unpersuasive.
Further, Applicant does not specifically identify which limitations claims 7 and 8 are allegedly not taught or suggested by Govea or explain why the findings set forth in the rejection are deficient. The rejection of claims 7 and 8 relies on the combined teachings of Shafiee, Timofejevs, and Govea, not on Govea alone teaching all limitations independently. As explained in the rejection, Govea was cited for the additional limitations recited in claims 7 and 8, while Shafiee and Timofejevs teach the remaining limitations of the claims. Applicant has not persuasively addressed the articulated findings and rationale supporting the combination. Accordingly, Applicant’s arguments do not overcome the rejection of claims 7 and 8.
Applicant argues that claim 12 and 13 are patentable because these claims depend from patentable claim 1. Applicant further argues that claim 12 recites that “each of the interconnected neurons includes a respective operational amplifier and a respective resistor with a respective fixed resistance value,” and that claim 13 recites that “fixed resistance values of the resistors represent weights of the neural network implemented by the hardware apparatus.”
Applicant’s arguments are not persuasive. As discussed above, claim 1 is not patentable over the cited combination. Accordingly, Applicant’s arguments relying on the patentability of claim 1 are unpersuasive.
Further, as explained in the rejection, Timofejevs teaches an integrated circuit implementing an analog neural network including operational amplifiers and resistors corresponding to analog neurons and neural network connections. Specifically, Timofejevs teaches transforming a neural network topology into an equivalent analog network including “a plurality of operational amplifiers and a plurality of resistors,” where “[e]ach operational amplifier represents a respective analog neuron,” and further teaches generating a “resistance matrix” in which “[e]ach element of the resistance matrix corresponds to a respective weight of the weight matrix.” Under the broadest reasonable interpretation, these teachings correspond to the claimed interconnected neurons including respective operational amplifiers and respective resistors having respective fixed resistance values, where the resistance values represent weights of the implemented neural network.
Applicant has no persuasively explained why these teachings fail to satisfy the limitations of claims 12 and 13 under the broadest reasonable interpretation of the claim language. Accordingly, Applicant’s arguments do not overcome the rejection of claim 12 and 13.
Accordingly, for at least the reasons set forth above, the rejections of claims 1-13 under 35 U.S.C. 103 is maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6 and 9-13 are rejected under the 35 U.S.C. 103 as being unpatentable over Shafiee et al., (NPL: “ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars” (Published: 2016)) in view of Timofejevs et al., (Pub. No.: US 20210406661 A1 (Filed: 2021)).
Regarding claim 1, Shafiee teaches the following limitations:
A hardware apparatus implementing a neural network, the hardware apparatus comprising (Shafiee, [Abstract] “A number of recent efforts have attempted to design accelerators for popular machine learning algorithms, such as those involving convolutional and deep neural networks (CNNs and DNNs).”):
a network of interconnected neurons comprising a plurality of subnetworks , including a left subnetwork of the interconnected neurons and a right subnetwork of the interconnected neurons, wherein the left subnetwork and the right subnetwork are interconnected via a buffer (Shafiee, [Introduction] “The dominant data structures in CNNs and DNNs are the synaptic weight matrices that define each neuron layer These are distributed across several eDRAM banks on multiple tiles/nodes. The computations involving these weights are brought to the eDRAM banks and executed on adjacent NFUs, thus achieving near data processing. This requires moving the outputs of the previous neuron layer to the relevant tiles so they can merge with co-located synaptic weights to produce the outputs of the current layer. The outputs are then routed to appropriate eDRAM banks so they can serve as inputs to the next layer.” – describes neuron layers implemented across multiple hardware tiles (subnetworks), with the outputs of one tile group being routed into eDRAM buffers before being consumed by the next tile group. This directly corresponds to “a network of interconnected neurons” and “left and right subnetworks interconnected via a buffer”) and
(ii) the right subnetwork is configured to operate in response to a conditional trigger (Shafiee, [section IV] “To understand how results are passed from one stage to the next, consider the following example, also shown in Figure 3. Assume that in layer
i
, a 6×6 input feature map is being convolved with a 2×2 kernel to produce an output feature map of the same size. Assume that a single column in an IMA has the four synaptic weights used by the 2×2 kernel. The previous layer
i
-
1
produces outputs 0, 1, 2, ..., 6, 7, shown in blue in Figure 3a. All of these values are placed in the input buffer for layer
i
. At this point, we have enough information to start the operations for layer
i
. So inputs 0, 1, 6, 7 are fed to the IMA and they produce the first output for layer
i
. When the previous layer i−1 produces output 8, it gets placed in the input buffer for layer
i
. Value 0, shown in green in Figure 3b, is no longer required and can be removed from the input buffer. Thus, every new output produced by layer
i
-
1
allows layer
i
to advance the kernel by one step and perform a new operation of its own.” – teaches that operation of a subsequent layer (corresponding to the right subnetwork) is dependent on the availability of sufficient input data stored in an intermediate buffer. Specifically, the reference states that the operations begin only when there is “enough information” in the buffer, and further teaches that additional operations occur only as new outputs from a preceding layer are received and added to the buffer. Thus, the subsequent layer does not operate continuously, but instead operates only when a condition based on the content of the buffer is satisfied. Under the broadest reasonable interpretation, this corresponds to the claimed configuration in which the right subnetwork operates in response to a conditional trigger.)
However, Shafiee does not teach but Shafiee in view of Timofejevs teaches this limitation:
(i) the left subnetwork of neurons and the right subnetwork of neurons are configured to operate at different frequencies (Timofejevs, paragraph [0376] “In some implementations, the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency… In some implementations, this predetermined input signal frequency may be dependent on the application, such as Human Activity Recognition (HAR) or PPG. For example, the predetermined input signal frequency is 30-60 Hz for video processing, around 100 Hz for HAR and PPG, 16 KHz for sound processing, and around 1-3 Hz for battery management. Some implementations activate different signal delay blocks activate at different frequencies.” [0488] “Typically, all delay blocks are activated simultaneously with the same activation signal. Some blocks can be activated simultaneously on one frequency, and other blocks can be activated on another frequency. In some implementations, these frequencies have common multiplier, and signals are synchronized.” – disclose that different functional portions of the analog neural network operate at different activation frequencies (e.g., 100 Hz for HAR/PPG vs. 16 Hz for sound processing). Under the broadest reasonable interpretation, these distinct portions of the network – each dedicated to processing a specific signal type – correspond to the claimed “subnetworks.” Because Timofejevs teaches that one portion runs at a first frequency while another portion runs at a second, different frequency, the reference meets the claim requirement that the left and right subnetworks are “configured to operate at different frequencies.”
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having a combination of Shafiee and Timofejevs before them, to incorporate the frequency-based activation techniques of Timofejevs into the pipelined crossbar-based neural network architecture of Shafiee. One would have been motivated to make such a combination in order to optimize the timing, energy usage, and data-rate alignment of Shafiee’s hardware subnetworks (tiles), especially when processing heterogeneous sensor inputs or workloads that naturally occur at different update rates. This would allow each Shafiee subnetwork to operate only as fast as necessary for its assigned portion of the computation, thereby reducing unnecessary switching activity, improving synchronization between pipeline stages, and increasing overall processing efficiency.
Regarding claim 2, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein the neural network is a trained convolutional neural network (Shafiee, [page 16, section III] “The architecture is not used for in-the-field training; it is only used for inference, which is the dominant operation in several domains…After training has determined the weights for every neuron, the weights are appropriately loaded into memristor cells with a programming step…During inference, inputs are provided to ISAAC through an I/O interface and routed to the tiles implementing the first layer of the CNN.” – inference happens only after training is complete).
Regarding claim 3, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 2, therefore is rejected for the same reasons as those presented for claim 2, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein the network of interconnected neurons corresponds to a plurality of layers of the trained convolutional neural network, the trained convolutional neural network includes a first layer of neurons and a second layer of neurons, and communication of data between the first layer and the second layer in the convolutional neural network is implemented in the hardware apparatus by the buffer (Shafiee, [page 17, section III] “During inference, inputs are provided to ISAAC through an I/O interface and routed to the tiles implementing the first layer of the CNN…The dot-product operations involved in convolutional and classifier layers are performed on crossbar arrays; those results are sent to ADCs, and then aggregated in output registers after any necessary shift-and-adds. The aggregated result is then sent through the sigmoid operator and stored in the eDRAM banks of the tiles processing the next layer. The process continues until the final layer generates an output that is sent to the I/O interface.” – discloses that outputs produced by one CNN layer are aggregated and stored in eDRAM before being provided to the tiles processing the next layer. Shafiee states that results “are sent to the eDRAM banks of the tiles implementing the next layer,” which constitutes communication between a first layer and second layer via a buffer. Under BRI, this satisfies the requirement that communication between the claimed first and second layers “is implemented…by the buffer.”)
Regarding claim 4, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 3, therefore is rejected for the same reasons as those presented for claim 3, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein the first layer of neurons is implemented in the left subnetwork and the second layer of neurons is implemented in the right subnetwork (Shafiee, [page 17, section IV] “the tiles/IMAs of ISAAC have to be partitioned across the different CNN layers. For example, tiles 0-3 may be assigned to layer 0, tiles 4-11 may be assigned to layer 1, and so on. In this case, tiles 0-3 would store all weights for layer 0 and perform all layer 0 computations in parallel. The outputs of layer 0 are sent to some of tiles 4-11; once enough layer 0 outputs are buffered, tiles 4-11 perform the necessary layer 1 computations” – explains that tiles 0-3 implement all computations for layer 0 and tiles 4-11 implement all computations for layer 1, with data communicated between them via a buffer. Under BRI, tiles 0-3 constitute the claimed left subnetwork and tiles 4-11 constitute the claimed right subnetwork. Therefore, Shafiee meets the limitation that the first layer of neurons is implemented in the left subnetwork and the second layer is implemented in the second right subnetwork.).
Regarding claim 5, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein the buffer is a FIFO queue having a predetermined size (Shafiee, [page 17, section IV] “When the previous layer
i
-
1
produces output 8, it gets placed in the input buffer for layer
i
.
Value 0, shown in green in Figure 3b, is no longer required and can be removed from the input buffer.” – as the newest value (output 8) enters the back of the queue, the oldest value (Value 0) is removed from the front – First-In, First-Out. “The eDRAM buffer requirement between two layers is fixed. In general terms, the size of the buffer is:
(
(
N
x
×
(
K
y
-
1
)
)
+
K
x
)
×
N
i
f
” – the buffer size is determined based on CNN parameters.)
Regarding claim 6, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein the right subnetwork is configurable to dynamically operate at different frequencies (Timofejevs, paragraph [0376] “In some implementations, the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency… In some implementations, this predetermined input signal frequency may be dependent on the application, such as Human Activity Recognition (HAR) or PPG. For example, the predetermined input signal frequency is 30-60 Hz for video processing, around 100 Hz for HAR and PPG, 16 KHz for sound processing, and around 1-3 Hz for battery management. Some implementations activate different signal delay blocks activate at different frequencies.” [0488] “Typically, all delay blocks are activated simultaneously with the same activation signal. Some blocks can be activated simultaneously on one frequency, and other blocks can be activated on another frequency. In some implementations, these frequencies have common multiplier, and signals are synchronized.” – discloses that different functional portions of the neural network operate at different activation frequencies depending on the application or operational conditions (e.g., 100 Hz for HAR/PPG vs. 16 Khz for sound processing). Under the broadest reasonable interpretation, these distinct functional portions of the network correspond to the claimed “subnetworks.” The reference further teaches operation at different frequencies based on configuration or operating conditions, which inherently encompasses dynamic operation. Accordingly, Timofejevs teaches that right subnetwork is configurable to dynamically operate at different frequencies, as claimed.)
Regarding claim 9, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
the left subnetwork is a convolutional network that is configured to operate at a first frequency that is a first fraction of a predetermined frequency (Shafiee, [page 17, section IV] “the tiles/IMAs of ISAAC have to be partitioned across the different CNN layers. For example,
tiles 0-3 may be assigned to layer 0… tiles 0-3 would store all weights for layer 0 and perform all layer 0 computations in parallel.” – left subnetwork. Timofejevs, paragraph [0488] “Some blocks can be activated simultaneously on one frequency, and other blocks can be activated on another frequency…these frequencies have common multiplier, and signals are synchronized.” [0376] “the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency.” – Timofejevs teaches that different network blocks operate at different activation frequencies that are synchronized using a common multiplier relative to a predetermined input signal frequency. Under BRI, operation at a frequency defined by a common multiplier inherently includes operation at a fraction of a predetermined frequency.);
the right subnetwork comprises one or more elements configured to operate conditionally when a last value appended to the buffer causes buffer contents to exceed a predetermined threshold percentage of buffer capacity (Shafiee, [page 15, Introduction] “As soon as enough outputs are generated by a layer and aggregated in an eDRAM buffer, the next layer can start its operations.” [Page 20, section VI] “The mapping of layers to IMAs and the resulting pipeline have to be determined off-line and loaded into control registers that drive finite state machines. These state machines ensure that results are sent to appropriate destinations in every cycle. “[Page 17, section IV] “When the previous layer
i
-
1
produces output 8, it gets placed in the input buffer for layer
i
… Thus, every new output produced by layer
i
-
1
allows layer
i
to advance the kernel by one step and perform a new operation of its own.” and “The eDRAM buffer requirement between two layers is fixed.” [Page 18, section IV] “the previous layer
i
-
1
has to produce two values before layer
i
can perform its next step. This can cause an unbalanced pipeline where the IMAs of layer
i
-
1
are busy in every cycle, while the IMAs of layer
i
are busy in only every alternate cycle.”- Shafiee teaches that the downstream layer (the claimed right subnetwork) comprises specific elements (identified as control registers and finite stage machines) configured to operate conditionally. The operation does not begin until a sufficient amount of data has accumulated in the buffer, and the operation begins only after the buffer reaches a required fill level (“at this point, we have enough information”- page 17). Shafiee teaches that these pipeline requirements are “determined off-line” (predetermined) and “loaded into control registers.” Shafiee further teaches that each newly appended output causes the buffer to advance and conditionally enables the next operation (e.g., “produce two values” or “output 8”). Because the buffer capacity is fixed, this required fill level (stored in the control registers) constitutes a predetermined threshold percentage relative to buffer capacity. The arrival of the final required value is the “last value appended” that satisfies this threshold and triggers the conditional operation under the BRI.)
Regarding claim 10, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 9, therefore is rejected for the same reasons as those presented for claim 9, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein the threshold percentage is 50% (Shafiee, [page 18, section IV] “the previous layer
i
-
1
has to produce two values before layer
i
can perform its next step. This can cause an unbalanced pipeline where the IMAs of layer
i
-
1
are busy in every cycle, while the IMAs of layer
i
are busy in only every alternate cycle.” [Page 17, section IV] “The eDRAM buffer requirement between two layers is fixed.” [Page 20, section IV] “The mapping of layers to IMAs and the resulting pipeline have to be determined off-line and loaded into control registers that drive finite state machines. These state machines ensure that results are sent to appropriate destinations in every cycle.” – Shafiee discloses a system in which the downstream layer operates conditionally based on a predetermined buffer threshold that is fixed and loaded off-line into control registers. Shafiee further discloses a specific example in which this conditional execution causes the downstream layer to be “busy in only every alternate cycle.” Under the broadest reasonable interpretation, “alternate cycle” defines an operation ratio of one active cycle for every two total cycles (1/2), which corresponds to a 50% execution threshold. Because this behavior results from a fixed buffer capacity and a predetermined trigger condition (loaded into the control registers), Shafiee discloses a configuration wherein the threshold percentage is 50%.).
Regarding claim 11, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 9, therefore is rejected for the same reasons as those presented for claim 9, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
the network of interconnected neurons is configured to receive, at approximately 16 Hz, 1 channel of voice data (Timofejevs, paragraph [0451] “Referring next to FIG. 31M, the trained neural network is trained (3154), using training datasets containing speech commands (e.g., Google Speech Commands Dataset), for identifying voice commands (e.g., 10 short spoken keywords, including “yes”, “no”, “up”, “down”, “left”, “right”, “on”, “off”, “stop”, “go”)… the neural network topology is (3156) a Depthwise Separable Convolutional Neural Network (DS-CNN) layer with 1 neuron” Paragraph [0376] “the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency… For example, the predetermined input signal frequency is…16 KHz for sound processing…Some implementations activate different signal delay blocks activate at different frequencies.” – Timofejevs discloses a neural network configured to receive voice data, as shown by explicit training on “speech commands” (Fig. 31M). This satisfies the claimed data type (“voice data”). Timofejevs further discloses that the neural network topology includes a DS-CNN layer with 1 neuron. Under BRI, an input layer comprising a single neuron corresponds to receiving a single input data, such as mono voice input. With respect to the claimed frequency of “approximately 16 Hz,” Timofejevs teaches that sound signals are acquired at a raw input signal frequency of 16 kHz, but also expressly teaches that neural network operation is controlled by an activation frequency via signal delay blocks that are “synchronized to network input signal frequency” and may be activated at different frequencies. Under BRI, “configured to receive” does not require that the raw sensor sampling frequency equal the network processing frequency. Instead, it encompasses systems in which higher-rate raw input data is aggregated or framed and supplied to the neural network at lower effective input data. Accordingly, a neural network that process 16 kHz data but is configured – via activation timing and delay blocks – to received derived input frames at an approximately 16 Hz falls within the scope of the claimed limitation.);
the left subnetwork is configured to operate at a frequency that is approximately 200 Hz, receive and process 10 millisecond long input data sequences, shaped (160, 1), with output being shaped ( 1 ) (Timofejevs, paragraph [0376] “In some implementations, an external cycle timer activates the one or more signal delay blocks with a constant time period (e.g., 1, 5, or 10 time steps)…In some implementations, the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency…the predetermined input signal frequency is…16 KHz for sound processing…Some implementations activate different signal delay blocks activate at different frequencies.” Paragraph [0451] “the trained neural network is trained…training datasets containing speech commands…for identifying voice commands… the neural network topology is (3156) a Depthwise Separable Convolutional Neural Network (DS-CNN) layer with 1 neuron.” Paragraph [0516] “ while generating inferences using the integrated circuit, determining (3310) if a level of signal output of the plurality of operational amplifiers is equilibrated (e.g., using the signal monitoring module 268). Operational amplifiers go through a transient period (e.g., a period that lasts less than 1 millisecond from transient to plateau signal) after receiving inputs, after which the level of signal is equilibrated and does not change.” – Timofejevs teaches a voice processing system configured with the specific input parameters and timing capabilities to satisfy this limitation. First, regarding the operation frequency: Timofejevs teaches that execution timing is controlled by an “external cycle timer” with a selectable constant period and that delay blocks may run at “different frequencies.” Under BRI, “configured to operate at ~200 Hz” is satisfied by configuring the cycle timer to execute once every 5 ms (i.e., 200 executions/sec). This is technically supported by Timofejevs’s hardware-timing disclosure that the analog neurons settle in “less than 1 millisecond” after receiving inputs. Because the circuitry completes a stable compute step in <1ms, it is physically capable of operating at the claimed 5ms (200 Hz) execution period. Second, regarding the input shape (160,1): Timofejevs expressly teaches sound processing with a predetermined input signal frequency of “16 kHz.” A 10 ms frame of 16 kHz audio inherently contains 160 samples (16,000 samples/sec x 0.010 sec = 160). Timofejevs further teaches the topology is a DS-CNN layer with “1 neuron,” which under BRI corresponds to one input channel (mono). Therefore, a 10 ms frame in this system is necessarily shaped (160,1). Finally, because the disclosed DS-CNN layer is equipped “with 1 neuron,” its output is inherently a single scalar value, satisfying the output shape of (1).)
the buffer is a FIFO queue having size ( 40, 1) configured to update at a frequency that is approximately 200 Hz (Shafiee, [page 17, section IV] “When the previous layer
i
-
1
produces output 8, it gets placed in the input buffer for layer
i
. Value 0, shown in green in Figure 3b, is no longer required and can be removed from the input buffer. Thus, every new output produced by layer
i
-
1
allows layer
i
to advance the kernel by one step and perform a new operation of its own.” And “The eDRAM buffer requirement between two layers is fixed.” [Page 20, section IV] “The mapping of layers to IMAs and the resulting pipeline have to be determined off-line and loaded into control registers that drive finite state machines.” – Shafiee discloses an input buffer between neural network layers that operates by accepting newly produced outputs and removing older values once they are no longer needed. This behavior – adding new values while discarding the oldest values – constitutes a FIFO queue under BRI. Shafiee further teaches that the buffer is fixed, and the pipeline behavior is determined off-line and loaded into control registers, establishing that both buffer size and update behavior are predetermined configuration parameters. Under BRI, a fixed-capacity FIFO buffer configured for single-channel data corresponds to a size of (40,1), when the buffer is configured to store 40 sequential scalar outputs from the left subnetwork. Regarding the update frequency, Shafiee teaches that buffer updates synchronously with output production from the preceding layer. Therefore, configuring the left subnetwork to operate at approximately 200 Hz inherently configures the FIFO buffer to update at the same frequency, as each new output generated by the left subnetwork is appended to the buffer. Under BRI, this satisfies the limitation that the FIFO queue is “configured to update at a frequency that is approximately 200 Hz.”); and
the right subnetwork is configured to (i) operate when the last value appended to the buffer causes the buffer to exceed 50% of buffer capacity, (ii) process (40, 1) buffer values as an input data sequence, and (iii) output data having a shape (1) representing voice activity confidence level (Shafiee, [page 15, Introduction] “As soon as enough outputs are generated by a layer and aggregated in an eDRAM buffer, the next layer can start its operations.” [Page 18, section IV] “the previous layer
i
-
1
has to produce two values before layer
i
can perform its next step. This can cause an unbalanced pipeline where the IMAs of layer
i
-
1
are busy in every cycle, while the IMAs of layer
i
are busy in only every alternate cycle.” [Page 17, section IV] “The eDRAM buffer requirement between two layers is fixed.” [Page 20, section IV] “The mapping of layers to IMAs and the resulting pipeline have to be determined off-line and loaded into control registers that drive finite state machines.” Timofejevs, paragraph [0451] “ the trained neural network is trained (3154), using training datasets containing speech commands… the neural network topology is (3156) a Depthwise Separable Convolutional Neural Network (DS-CNN) layer with 1 neuron.” – Shafiee discloses a right subnetwork (the downstream layer) configured to operate conditionally based on buffer occupancy. Specifically, Shafiee teaches that execution begins only “as soon as enough outputs are generated” and aggregated in the buffer, explicitly nothing that the ”previous layer i-1 has to produce two values before layer i can perform its next step.” This creates a pipeline where the downstream resources are “busy in only every alternate cycle.” Under BRI, this “alternate cycle” behavior – operating once every two cycles – mathematically corresponds to a 50% operational threshold. Since Shafiee further establishes that the pipeline behavior is “determined off-line and loaded in control registers,” the subnetwork is clearly configurable to trigger execution precisely when the last appended value causes the buffer to exceed 50% of its capacity. Regarding the processing of specific data dimensions, Shafiee teaches that the buffer requirement between layers is “fixed” and programmable. As established in the analysis of the preceding limitation, the architecture is capable of being configured to a buffer depth of 40 scalar values. Consequently, when the buffer is configured to this specific dimension, the right subnetwork is inherently configured to ingest and process those “(40,1) buffer values” as its input data sequence upon the triggering of the 50% threshold. Finally, regarding the output characteristics, Timofejevs discloses a neural network trained on “speech commands” that utilizes a specific topology ending in a “DS-CNN layer with 1 neuron.” Under BRI, a single-neuron output layer produces a scalar output having a shape of (1). In the context of the disclosed voice recognition system, this scalar output reasonably represents a “voice activity confidence level” or probability score. Thus, the combination of Shafiee’s programmable pipeline control and Timofejevs’ specific voice-processing topology meets each element of the limitation.)
Regarding claim 12, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein each of the interconnected neurons includes a respective operational amplifier and a respective resistor with a respective fixed resistance value (Timofejevs, paragraph [0077] “In another aspect, an integrated circuit is provided, according to some implementations. The integrated circuit includes an analog network of analog components fabricated by a method that includes: (i) obtaining a neural network topology and weights of a trained neural network; (ii) transforming the neural network topology to an equivalent analog network of analog components including a plurality of operational amplifiers and a plurality of resistors. Each operational amplifier represents a respective analog neuron, and each resistor represents a respective connection between a respective first analog neuron and a respective second analog neuron; (iii) computing a weight matrix for the equivalent analog network based on the weights of the trained neural network. Each element of the weight matrix represents a respective connection; (iv) generating a resistance matrix for the weight matrix. Each element of the resistance matrix corresponds to a respective weight of the weight matrix; (v) generating one or more lithographic masks for fabricating a circuit implementing the equivalent analog network of analog components based on the resistance matrix; and (vi) fabricating the circuit based on the one or more lithographic masks using a lithographic process.” – Timofejevs teaches an analog neural network implementation including operational amplifiers and resistors corresponding to analog neurons and neural network connections. Specifically, each operational resistors corresponding to analog neurons, and the generated resistance matrix defines resistance values corresponding to neural network weights for fabrication of the circuits. Under the broadest reasonable interpretation, these fabricated analog neuron structures correspond to the claimed interconnected neurons including respective operational amplifiers and respective resistors having respective fixed resistance values.).
Regarding claim 13, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 12, therefore is rejected for the same reasons as those presented for claim 12, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
wherein fixed resistance values of the resistors represent weights of the neural network implemented by the hardware apparatus (Timofejevs, paragraph [0077] “In another aspect, an integrated circuit is provided, according to some implementations. The integrated circuit includes an analog network of analog components fabricated by a method that includes… Each operational amplifier represents a respective analog neuron, and each resistor represents a respective connection between a respective first analog neuron and a respective second analog neuron; (iii) computing a weight matrix for the equivalent analog network based on the weights of the trained neural network. Each element of the weight matrix represents a respective connection; (iv) generating a resistance matrix for the weight matrix. Each element of the resistance matrix corresponds to a respective weight of the weight matrix” – Timofejevs teaches an integrated circuit implementing an analog neural network in which a resistance matrix is generated from a weight matrix of the training neural network for fabrication of the circuit. Specifically, each element of the resistance matrix corresponds to a respective neural network weight. Under BRI, the resulting resistance values of the fabricated resistors therefore represents weights of the neural network implemented by the integrated circuit hardware apparatus, as claimed.).
Claims 7 and 8 are rejected under the 35 U.S.C. 103 as being unpatentable over Shafiee et al., (NPL: “ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars” (Published: 2016)) in view of Timofejevs et al., (Pub. No.: US 20210406661 A1 (Filed: 2021)) further in view of Govea et al., (Pub. No.: US 20200327397 A1 (Filed: 2019)).
Regarding claim 7, Shafiee in view of Timofejevs, as outlined above, teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
the left subnetwork is a convolutional network that is configured to operate at a first frequency that is a first fraction of a predetermined frequency (Shafiee, [page 17, section IV] “the tiles/IMAs of ISAAC have to be partitioned across the different CNN layers. For example, tiles 0-3 may be assigned to layer 0, tiles 4-11 may be assigned to layer 1, and so on. In this case, tiles 0-3 would store all weights for layer 0 and perform all layer 0 computations in parallel. The outputs of layer 0 are sent to some of tiles 4-11; once enough layer 0 outputs are buffered, tiles 4-11 perform the necessary layer 1 computations” – Under the broadest reasonable interpretation, tiles 0-3 form the claimed left subnetwork implementing the convolutional first layer. Timofejevs, paragraph [0376] “the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency… 30-60 Hz for video processing…100 Hz for HAR and PPG, 16 KHz for sound processing…” [0488] “Some blocks can be activated simultaneously on one frequency, and other blocks can be activated on another frequency. In some implementations, these frequencies have common multiplier, and signals are synchronized.” – under BRI, operation using a “common multiplier” encompasses fractional operating frequencies relative to a predetermined input-signal frequency, satisfying the claimed requirement that the left subnetwork operates a first frequency that is a first fraction of a predetermined frequency.)
the right subnetwork is configured to operate at a second frequency that is a fraction of the first frequency (Shafiee, [page 17, section IV] “the tiles/IMAs of ISAAC have to be partitioned across the different CNN layers. For example, tiles 0-3 may be assigned to layer 0, tiles 4-11 may be assigned to layer 1, and so on. In this case, tiles 0-3 would store all weights for layer 0 and perform all layer 0 computations in parallel. The outputs of layer 0 are sent to some of tiles 4-11; once enough layer 0 outputs are buffered, tiles 4-11 perform the necessary layer 1 computations” – teaches downstream group of tiles (e.g., tiles 4-11) that executes only after buffered outputs from an upstream layer are available, corresponding under the BRI to the claimed right subnetwork. Timofejevs, paragraph [0376] “the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency… 30-60 Hz for video processing…100 Hz for HAR and PPG, 16 KHz for sound processing…” [0488] “Some blocks can be activated simultaneously on one frequency, and other blocks can be activated on another frequency. In some implementations, these frequencies have common multiplier, and signals are synchronized.” – discloses functional portions of a neural network operate at different activation frequencies, and that these frequencies are synchronized using a common multiplier relative to a predetermined input-signal frequency. Under BRI, operating with a common multiplier inherently encompasses operation at a fraction of another frequency (e.g., down-sampled execution relative to an upstream rate). Accordingly, Timofejevs teaches configuring a downstream portion of the network to operate at a second frequency that is a fraction of a first frequency, satisfying the claimed limitation when applied to the downstream subnetwork of Shafiee.).
However, Shafiee in view of Timofejevs does not teach but Shafiee in view of Timofejevs further in view of Govea teaches the following limitation:
the right subnetwork comprises ResNet block elements and dense layers at its output (Shafiee, [page 17, section IV] “the tiles/IMAs of ISAAC have to be partitioned across the different CNN layers. For example, tiles 0-3 may be assigned to layer 0, tiles 4-11 may be assigned to layer 1, and so on. In this case, tiles 0-3 would store all weights for layer 0 and perform all layer 0 computations in parallel. The outputs of layer 0 are sent to some of tiles 4-11; once enough layer 0 outputs are buffered, tiles 4-11 perform the necessary layer 1 computations” – teaches downstream group tiles (e.g., tiles 4-11) performs subsequent layer computations after buffered outputs are received. Govea, paragraph [0045] “outputs of ResNet 224 and dense layers 248, 258, 260, 254, and 256 are provided to an 832-element concatenation layer…The output of dense layer 264 is provided to a fully connected layer, shown as SoftMax 266, that converts it into a probability distribution.” - discloses a neural network portion that includes ResNet block elements followed by dense (fully connected) layers at the output.); and
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having Shafiee, Timofejevs, and Govea before them, to incorporate ResNet block elements and dense layers at the output into the downstream hardware subnetwork of Shafiee. One would have been motivated to make such a combination because Shafiee teaches a buffered, tile-based hardware architecture for executing successive neural network layers, and Govea teaches that ResNet-based neural networks with dense output layers are effective for extracting higher-level features and producing final classification results from complex input data. Integrating ResNet blocks and dense layers into Shafiee’s downstream subnetwork would allow the buffered hardware architecture to support deeper feature refinement and robust classification, yielding a predictable improvement in inference performance while using known neural network structures within Shafiee’s existing hardware execution framework.
Regarding claim 8, Shafiee in view of Timofejevs further in view of Govea, as outlined above, teaches all the elements of claim 7, therefore is rejected for the same reasons as those presented for claim 7, mutatis mutandis. Shafiee in view of Timofejevs further teaches:
the network of interconnected neurons is configured to receive, at 25 Hz, PPG signals with 4 channels and accelerometer signals with 3 channels (Timofejevs paragraph [0452] “ using training datasets containing photoplethysmography (PPG) data, accelerometer data…Wrist-based sensor data contains PPG, 3-axis accelerometer, temperature and electrodermal response signals sampled from 4 to 64 Hz, and a reference heartrate data obtained from ECG sensor… for determining pulse rate during physical exercises (e.g., jogging, fitness exercises, climbing stairs) based on PPG sensor data and 3-axis accelerometer data.” [0454] “the trained neural network is trained (3170) to perform human activity type recognition (e.g., walking, running, sitting, climbing stairs, exercising, activity tracking), based on inertial sensor data (e.g., 3-axes accelerometers, magnetometers, or gyroscope data, from fitness tracking devices, smart watches or mobile phones; 3-axis accelerometer data as input, sampled at up to 96 Hz frequency… In some implementations, the neural network topology includes (3172) three channel-wise convolutional networks” [0498] “The multi-channel convolutional neural network can be used to distinguish between different types of human activities, such as walking, running, sitting, climbing stairs, exercising and can be used for activity tracking.” – Under the broadest reasonable interpretation, Timofejevs expressly discloses a neural network configured to receive PPG sensor data and 3-axis (3-channel) accelerometer data, satisfying the accelerometer portion of the limitation. Timofejevs further discloses that such sensor data is received over explicit frequency ranges (4-64 Hz and up to 96 Hz), which encompasses 25 Hz, thereby satisfying the claimed operating frequency. Although Timofejevs does not numerically specify “four” PPG channels, the reference expressly teaches a multi-channel neural network architecture (“channel-wise convolutional networks”) that processes PPG data, and the claim does not further limit the nature or structure of the PPG channels. Under BRI, a neural network expressly configured for multi-channel sensor inputs and disclosed as receiving PPG data reasonably encompasses PPG signals with four channels.)
the left subnetwork is configured to operate at a frequency that is approximately 2 Hz, receive I-second long input data sequences so that its input is shaped (25, 7) and output is shaped (1, 4) where 1 represents a time dimension and 4 represents a channel dimension (Timofejevs, paragraph [0452] “Wrist-based sensor data contains PPG, 3-axis accelerometer, temperature and electrodermal response signals sampled from 4 to 64 Hz, and a reference heartrate data obtained from ECG sensor with sampling around 2 Hz. Original data was split into sequences of 1000 timesteps (around 15 seconds), with a shift of 500 timesteps…for determining pulse rate… based on PPG sensor data and 3-axis accelerometer data” [0441] “the integrated circuit further includes an analog signal sampling module (3118) configured to process 1-dimensional or 2-dimensional analog inputs with a sampling frequency based on number of inferences of the integrated circuit” [0488] “the delay block has an external cycle timer (e.g., a digital timer) which activates the delay block with a constant period of time dt…Such activation frequency can, for instance, correspond to network input signal frequency…Some blocks can be activated simultaneously on one frequency, and other blocks can be activated on another frequency.” – Under BRI, Timofejevs teaches that a neural network process time-sensor inputs over fixed temporal windows (“split in sequences”) and that the rate at which subnetworks operate is controlled by an activation cycle (“external cycle timer”, “activation frequency”). Configuring the left subnetwork to operate at an approximately 2 Hz inference rate corresponds to executing every ~0.5 seconds, which is a reasonable and explicit subset of the disclosed frequency- controlled activation schemes. Further, a 1-second input window sampled at 25 Hz inherently yields 25 time steps, and combining PPG data with 3-axis accelerometer data yields 7-channel input shape (25, 7). The generation of a single aggregated vector per execution (“determining pulse rate”) reasonably corresponds to an output shape of (1,4), where 1 represents the time dimension and 4 represents channelized output features. Under BRI, Timofejevs therefore teaches the claimed operating frequency, input duration, and input/output dimensional structure of the left subnetwork.);
the buffer is a FIFO queue having size ( 40, 4) and configured to update at a frequency that is approximately 2 Hz (Shafiee, [page 17, section IV, Fig. 3] “All of these values are placed in the input buffer for layer
i
. At this point, we have enough information to start the operations for layer
i
. So inputs 0, 1, 6, 7 are fed to the IMA and they produce the first output for
l
a
y
e
r
i
.
When the previous layer
i
-
1
produces output 8, it gets placed in the input buffer for layer
i
. Value 0, shown in green in Figure 3b, is no longer required and can be removed from the input buffer. Thus, every new output produced by layer
i
-
1
allows layer
i
to advance the kernel by one step” – discloses an input buffer where newly produced outputs are appended while the oldest buffered values are removed once no longer needed. Under BRI, a buffer that admits new data and discards the oldest data in sequence operations as a First-In, First-Out queue. “The eDRAM buffer requirement between two layers is fixed. In general terms, the size of the buffer is:
(
(
N
x
×
(
K
y
-
1
)
)
+
K
x
)
×
N
i
f
“ – buffer is deterministically defined by convolution parameters and the number of input feature maps. Under BRI,
N
i
f
corresponds to the number of channels (here, 4 channels), and the remaining terms define the temporal depth of stored outputs. Selecting convolution parameters that yield 40 stored rows results in a buffer of size (40, 4). [Page 15, Introduction] “As soon as enough outputs are generated by a layer and aggregated in an eDRAM buffer, the next layer can start its operations.” – the buffer is updated each time the previous layer produces output, meaning the buffer update rate is directly tired to the execution frequency of the upstream (left subnetwork). Under BRI, when the left subnetwork is configured to operate at approximately 2 Hz, the buffer necessarily updates at approximately 2 Hz, because each execution produces a new buffered value and advances the FIFO.);
the right subnetwork is configured to process ( 40, 4) buffer values as an input data sequence and its output has a shape (1) representing heartrate (Shafiee, [page 17, section IV, Fig. 3] “All of these values are placed in the input buffer for layer
i
.” And “The outputs of layer 0 are sent to some of tiles 4-11; once enough layer 0 outputs are buffered, tiles 4-11 perform the necessary layer 1 computations” and “The eDRAM buffer requirement between two layers is fixed.” Timofejevs, paragraph [0452] “the trained neural network is trained…for determining pulse rate during physical exercises…based on PPG sensor data and 3-axis accelerometer data…. two dense layers with 16 neurons and 1 neuron, respectively” – Under BRI, Shafiee teaches a hardware architecture in which outputs of a first layer are accumulated in a buffer and then provided as a grouped set of buffered values to a subsequent layer for processing. This subsequent layer corresponds to the claimed right subnetwork, which processes buffered data as an input data sequence. Shafiee further teaches that the buffer size between layers is fixed and determined by network parameters, which under BRI reasonably encompasses a buffer organized as (40,4) values when configured for multi-channel sensor data. Timofejevs complements Shafiee by teaching that such buffered sensor data (PPG and 3-axis accelerometer) is processed to determine a pulse/heart rate, and that the network produces its result via a final dense layer having a single neuron, which under BRI corresponds to an output shape (1) representing heartrate.); and
the right subnetwork is configured to operate at a frequency that is approximately 1 Hz (Timofejevs, paragraph [0376] “the activation frequency of the one or more signal delay blocks is/are synchronized to network input signal frequency… For example, the predetermined input signal frequency is…around 1-3 Hz for battery management.” [0488] “Typically, all delay blocks are activated simultaneously with the same activation signal. Some blocks can be activated simultaneously on one frequency, and other blocks can be activated on another frequency.” – Under BRI, “configured to operate at a frequency” encompasses the rate at which a subnetwork is activated or executed, not merely a fixed hardware clock speed. Timofejevs teaches that different portions of a neural network are activated at different frequencies including low-frequency operation on the order of 1-3 Hz. A frequency approximately 1 Hz falls within this disclosed range. Because the right subnetwork corresponds to a downstream processing portion whose execution is controlled by activation timing, Timofejevs teaches that the right subnetwork may be configured to operate at approximately 1 Hz, satisfying the claimed limitation.).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daravanh Phakousonh whose telephone number is (571)272-6324. The examiner can normally be reached Mon - Thurs 7 AM - 5 PM, Every other Friday 7 AM - 4PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at 571-272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daravanh Phakousonh/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights