DETAILED ACTION
Claims 1-20 are pending. Applicant has amended claims 1, 3, 4 and 7.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Anderson et al. (US 2017/0220384 A1).
As to claim 1, Baum teaches a neural processing device comprising sequencer circuitry comprising (NN processing (core) 60; Fig. 5):
one or more processing engine clusters (clusters 66; Fig. 5), each of which includes one or more processing engine groups (subclusters), wherein each of the one or more processing groups includes one or more processing engines (The cluster comprises a plurality of M subclusters … a plurality of layers control circuits 256; paragraph [0187]);
a first memory shared by the one or more processing engine clusters (Layer 4 or L4 memory; paragraph [0108] and the NN processor includes shared memory; paragraph [0111]); and
an interconnection configured to exchange data between the one or more processing engine clusters and the first memory (cluster interconnect; paragraph [0192]),
wherein the sequencer circuitry is configured to cause:
providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, and the at least one processing engine (dynamic resource assignment; paragraph [0019] and dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]; The subcluster comprises a plurality of N Pes 182; paragraph [0175] and (Figs. 8-9 and associated text)).
Baum does not teach monitoring the one or more processing engine clusters, the one or more processing engine groups, and the one or more processing engines to check performance related to the one or more processing engine clusters, and providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, according to the performance, wherein the enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines when the performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines.
However, Baum teaches a processing engine perform calculation operation (the NN processor comprises a plurality of basis computation units doing the same or similar mathematical manipulations; paragraph [0098] and multiply/accumulate entity; [0157]).
Anderson teaches monitoring the at least one processing engine to check performance related to the at least one processing engine (The processor may detect degradation of performance and/or lifetime of the active or inactive processor cores. In an aspect, detecting such degradation may result from comparing collected operational data and/or self test data; paragraph [0118]), detecting a performance problem based on the performance related to the at least one processing engine (In determination block, the processor may determine whether the active or inactive processor core, for which degrading performance and/or lifetime is detected has failed; paragraph [0119], the processor may determine whether the active or inactive processor core is inefficient in determine block; paragraph [0120]), and when the performance problem is related to the at least one processing engine, enhancing performance of the at least one processing engine (see Fig. 16 and “The processor may detect degradation of performance of the active processor core”; paragraph [0018], “In response to determining that the active or inactive processor core is efficient (i.e., determination block 1606=“No”), the processor core may update performance, efficiency, or capability data, and/or the priority of the active or inactive processor core in block 1612. In an aspect, the performance, efficiency, or capability data, and/or the priority may be stored (e.g., in a database) for use by algorithms that schedule tasks to processor cores, and to manage use of the processor core in the system. For example, the performance, efficiency, or capability data, and/or the priority of the processor core exhibiting degradation of performance and/or lifetime may be used to indicate that the priority assigned to the processor core should capped at a certain priority so that it is used less often. Similarly, the data and/or the priority may be used to adjust the calculated priority for the processor core to reduce it by an amount corresponding to the severity of the degradation.”; paragraph [0121]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Anderson to the system of Baum because Anderson teaches a runtime optimization of multicore system for increasing operating life and maximized performance of the multicore of the computing device.
Claims 2-14 are rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Anderson et al. (US 2017/0220384 A1) further in view of Guenther et al. (US 2021/0287423 A1).
As to claim 2, Baum as modified by Anderson does not teach the neural processing device of claim 1, wherein the sequencer circuitry is configured to monitor at least one of a bandwidth, latency, supply power, or temperature of the one or more processing engine clusters, and wherein the sequencer circuitry is further configured to cause checking performance between the one or more processing engine clusters and the interconnection.
However, Anderson teaches monitoring data related to performance of the processing cores (paragraph [0055] and [0118]).
Guenther teaches wherein the processing circuitry is configured to monitor at least one of a bandwidth, latency, supply power, or temperature of the at least one processing engine, and the processing circuitry is configured to check performance between the at least one processing engine and the interconnection, and performance between the at least one processing engine (see Fig. 22 illustrates how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region; paragraph [0254]-[0255]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Guenther to the system of Baum and Anderson because Baum teaches the system can dynamically allocate resources/memory/bandwidth to the PEs at runtime, and when apply the teaching of Guenther, the system of Baum could enhance the performance of the processing core/engine by monitoring data performance of the cores/engines in real time.
As to claim 3, Baum as modified by Anderson and Guenther teaches wherein the sequencer circuitry is further configured to cause:
checking traffic between the interconnection and the one or more processing engine clusters to detect a performance problem based on the performance related to the one or more processing engine clusters (see Guenther: Fig. 22 illustrates how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region; paragraph [0254]-[0255]);
reducing traffic of the first memory or an off-chip memory exchanging data with the first memory when the performance problem is related to a bandwidth (see Baum: halt command; paragraph [0216]-[0217]); and
enhancing performance of the interconnection when the performance problem is related to the bandwidth (see Baum: dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]).
As to claim 4, Baum as modified by Anderson and Guenther teaches wherein the enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines includes generating a processor control signal for increasing at least one of supply power or frequency of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines (see Anderson: changing the frequency with which certain processor cores are used; paragraph [0054]).
As to claim 5, Baum as modified by Anderson and Guenther teaches wherein the reducing traffic of the off- chip memory includes generating a memory control signal for activating at least one of an operation of compressing traffic of the first memory or the off-chip memory or an operation of decompressing the traffic (halt command; paragraph [0216]-[0217]).
As to claim 6, Baum as modified by Anderson and Guenther teaches wherein enhancing performance of the interconnection includes generating an interconnection control signal for increasing a frequency of the interconnection (dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]).
As to claim 7, Baum teaches a first memory (Layer 4 or L4 memory; paragraph [0108] and the NN processor includes shared memory; paragraph [0111]), an interconnection (cluster interconnect; paragraph [0192]), one or more processing engine clusters (clusters 66; Fig. 5), each of which includes one or more processing engine groups (subclusters), wherein each of the one or more processing engine groups includes one or more processing engines (The cluster comprises a plurality of M subclusters … a plurality of layers control circuits 256; paragraph [0187]); and
providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, and the at least one processing engine (dynamic resource assignment; paragraph [0019] and dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]; The subcluster comprises a plurality of N Pes 182; paragraph [0175] and (Figs. 8-9 and associated text));
wherein the first memory is shared by the one or more processing engine clusters (Layer 4 or L4 memory; paragraph [0108] and the NN processor includes shared memory; paragraph [0111]), and
wherein the interconnection is configured to transmit data between the first memory and the one or more processing engine clusters (cluster interconnect; paragraph [0192]).
Baum does not teach monitoring a first memory, an interconnect, one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, and the at least one processing engine, detecting a performance problem through the monitoring; and enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines when the performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines.
However, Baum teaches the processing engine perform calculation (the NN processor comprises a plurality of basis computation units doing the same or similar mathematical manipulations; paragraph [0098] and multiply/accumulate entity; [0157]).
Anderson teaches enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines when the performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines (see Fig. 16 and “The processor may detect degradation of performance of the active processor core”; paragraph [0018], “In response to determining that the active or inactive processor core is efficient (i.e., determination block 1606=“No”), the processor core may update performance, efficiency, or capability data, and/or the priority of the active or inactive processor core in block 1612. In an aspect, the performance, efficiency, or capability data, and/or the priority may be stored (e.g., in a database) for use by algorithms that schedule tasks to processor cores, and to manage use of the processor core in the system. For example, the performance, efficiency, or capability data, and/or the priority of the processor core exhibiting degradation of performance and/or lifetime may be used to indicate that the priority assigned to the processor core should capped at a certain priority so that it is used less often. Similarly, the data and/or the priority may be used to adjust the calculated priority for the processor core to reduce it by an amount corresponding to the severity of the degradation.”; paragraph [0121]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Anderson to the system of Baum because Anderson teaches a runtime optimization of multicore system for increasing operating life and maximized performance of the multicore of the computing device.
Guenther teaches monitoring a neural core including processing a first memory, an interconnection, and a plurality of processing engine groups (Fig. 22 illustrates how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region; paragraph [0254]-[0255] and monitoring the progress of a workload on an engine; paragraph [0096]) wherein each of the plurality of processing engine groups includes at least one processing engine (a graphics processor having a graphics engine cluster having multiple instances of the graphics processing engine tiles; paragraph [0127]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Guenther to the system of Baum and Anderson because Baum teaches the system can dynamically allocate resources/memory/bandwidth to the PEs at runtime, and when apply the teaching of Guenther, the system of Baum could enhance the performance of the processing core/engine by monitoring data performance of the cores/engines in real time.
As to claim 8, Baum as modified by Anderson and Guenther teaches wherein each of the one or more processing engines comprises an array of a plurality of processing elements interconnected by a mesh style network, the processing elements being reconfigurable (The subcluster comprises a plurality of N Pes 182; paragraph [0175] and (Figs. 8-9 and associated text) and The PE comprises … allowing the order of calculations to be manipulated; paragraph [0160], i.e. reconfigurable; and each subcluster comprises flexible and programmable pathways for feeding input data and weights to the neurons in the Pes as well as steering intermediate results from the neurons to and from either L2 or L3 memory; paragraph [0182]).
As to claim 9, Baum as modified by Anderson and Guenther does not clearly teach determining whether the performance problem is related to an off-chip memory; and reducing traffic of the off-chip memory when the performance problem is related to the off-chip memory.
However, Baum teaches reducing traffic of the memory when the performance problem is related to the memory (halt command; paragraph [0216]-[0217]).
Guenther teaches how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region (paragraph [0254]-[0255]) and monitoring the progress of a workload on an engine (paragraph [0096]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Guenther to the system of Baum because Baum teaches a method to improve the performance of the system when there’s a problem with the memory of the system, and the system of Guenther could be modified to determine whether the performance problem is related to an off-chip memory.
As to claim 10, Baum as modified by Anderson and Guenther teaches wherein reducing traffic of the off-chip memory includes activating a compression engine of traffic of the off-chip memory (halt command is performed in response to the control signal; paragraph [0216]-[0217]).
As to claim 11, Baum as modified by Anderson and Guenther does not clearly teach determining whether the performance problem is related to the first memory; and reducing traffic of the first memory when the performance problem is related to the first memory.
See rejection of claim 8 above for similar teaching, except this is related to the first memory instead of the off-chip memory.
As to claim 12, see rejection of claim 10 above for rejection of “wherein reducing traffic of the first memory includes activating a compression engine of traffic of the first memory”.
As to claim 13, Baum as modified by Anderson and Guenther teaches enhancing performance of the interconnection when the performance problem is not related to the first memory (see Baum: dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]).
As to claim 14, Baum as modified by Anderson and Guenther does not teach wherein enhancing performance of the interconnection includes overdriving a frequency of the interconnection.
However, Baum teaches dynamic bus width and memory bit cell, balanced capability of runtime configuration modification (paragraph [0189]-[0190] and [0202]).
It would have been obvious to one of ordinary skill in the art that the system of Baum could be modified, based on the capability to reconfigure during runtime, to overdriving a frequency of the interconnection.
Claims 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Anderson et al. (US 2017/0220384 A1) and Guenther et al. (US 2021/0287423 A1) further in view of Heaton et al. (US 11,561,833 B1).
As to claim 15, Baum as modified by Anderson and Guenther does not teach limitations of claim 15.
However, Heaton teaches wherein a compiler configuring the at least one processing engine is configured to perform: (the compiler engine can allocate the memory and computation resources for the neural network processing operations; col. 1, line 59 – col. 2, line 8)
receiving a deep learning graph (a compiler can obtain information of a dependency graph representing a neural network model as well as neural network operations; col. 11, lines 46-50);
storing a calculation code through processing compilation in a compute library (inherent from the compiler can obtain neural network operations and analyze the operations; col. 11, lines 46-61. Thus, the operations/calculation code must be stored);
generating intermediate representation (IR) by optimizing the deep learning graph (order of operations, generate instruction file, reformatting the input data, etc.; col. 11, line 55 -43 and col. 16, line 28 – col. 17, line 25);
performing, according to the IR, scheduling of a task between the plurality of processing engine clusters (runtime engine may perform resource placement; col. 17, line 26 – col. 18, line 55); and
generating a binary code according to the compute library on a circuit (inherent from the codes must be converted to binary in order for the system to execute the neural network operations).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching Heaton to the system of Baum as modified by Anderson and Guenther because Heaton teaches a method to allocate and place resources of a computing environment for performing neural network processing operations of data.
As to claim 16, Baum as modified by Anderson, Guenther and Heaton teaches wherein storing the calculation code in the compute library comprises: determining a dimension of each of the one or more processing engines; and performing scheduling of a task related to the one or more processing engine clusters. (see Heaton: col. 14, line 45 – col. 16, line 63).
As to claim 17, Baum as modified by Anderson, Guenther and Heaton teaches wherein the determining the dimension of each of processing engines comprises determining a number of processing elements included in each of the one or more processing engines. (see Heaton: col. 14, lines 19-23).
As to claim 18, Baum as modified by Anderson, Guenther and Heaton does not teach wherein the one or more processing engine groups are optimized through an L2 level scheduling.
However, Baum teaches processing cores may share certain resources such as L1 and L2 cache (paragraph [0145]). Thus, by scheduling to share the L2 level cache, the performance of the operations would be improved by having data/instruction obtained/stored to the nearby cache instead of off-chip memory.
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Anderson et al. (US 2017/0220384 A1) further in view of Manousakis et al. (US 2021/0103458 A1).
As to claim 19, Baum as modified by Anderson does not teach the at least one processing engine which implements at least one virtual processor (VP), and scale a frequency of the at least one processing engine in real time according to a status indicating a correspondence between the at least one VP and the at least one processing engine.
However, Manousakis teaches the at least one processing engine which implements at least one virtual processor (VP) (CPU-MEM may include processing cores and memory that may be assigned to each VM being hosted by host 310 … VMs may be assigned to respective group of cores; paragraph [0030] and virtual CPU; paragraph [0023]), and scale a frequency of the at least one processing engine in real time according to a status indicating a correspondence between the at least one VP and the at least one processing engine (Each chassis manager … Per-VM power controller may use per-core dynamic voltage frequency scaling to cap the per-core running … throttling just these VMs may be enough … protect the user-facing VMs; paragraph [0031] and [0053]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Manousakis to the system of Baum as modified by Anderson because Manousakis teaches a method to fine-grained per-VM power capping system that protects the performance-critical workload, and criticality and utilization aware VM placement policy that distributes VMs across a cluster to reduce the number of expected capping events and their performance impact (paragraph [0017]).
As to claim 20, although Baum, Anderson and Manousakis does not teach wherein a number of the at least one processing engine is different from a number of the at least one VP, Manousakis teaches VMs may be assigned to respective group of cores (paragraph [0030]). Manousakis does not teach each VM can be assigned only one processing core. Therefore, it would have been obvious to one of ordinary skill in the art that a number of the at least one processing engine is different from a number of the at least one VP.
Response to Arguments
Applicant's arguments filed 1/28/2026 have been fully considered but they are not persuasive.
In response to Applicant’s arguments regarding Baum as modified by Anderson does not teach the limitation “wherein the sequencer circuitry is further configured to causing enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines when a performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines”, the rejection has been clarified to show that Baum as modified by Anderson teaches the claimed limitation (see rejection of claims 1 and 7 above).
Therefore, the rejection is maintained.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DIEM K CAO whose telephone number is (571)272-3760. The examiner can normally be reached Monday-Friday 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, April Blair can be reached at 571-270-1014. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DIEM K CAO/Primary Examiner, Art Unit 2196
DC
May 7, 2026
Prosecution Insights