DETAILED ACTION
Claims 1-3 and 5-20 are pending. Applicant has amended claims 1, 7 and canceled claim 4.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Sunwoo et al. (US 2022/0035679 A1) further in view of Kwon et al. (US 2023/0107333 A1).
As to claim 1, Baum teaches a neural processing device comprising sequencer circuitry comprising (NN processing (core) 60; Fig. 5):
one or more processing engine clusters (clusters 66; Fig. 5), each of which includes one or more processing engine groups (subclusters), wherein each of the one or more processing groups includes one or more processing engines (The cluster comprises a plurality of M subclusters … a plurality of layers control circuits 256; paragraph [0187]);
a first memory shared by the one or more processing engine clusters (Layer 4 or L4 memory; paragraph [0108] and the NN processor includes shared memory; paragraph [0111]); and
an interconnection configured to exchange data between the one or more processing engine clusters and the first memory (cluster interconnect; paragraph [0192]),
wherein the sequencer circuitry is configured to cause:
providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, and the at least one processing engine (dynamic resource assignment; paragraph [0019] and dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]; The subcluster comprises a plurality of N Pes 182; paragraph [0175] and (Figs. 8-9 and associated text)).
Baum does not teach monitoring the one or more processing engine clusters, the one or more processing engine groups, and the one or more processing engines to check performance related to the one or more processing engine clusters, and providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, according to the performance, wherein the sequencer circuitry is further configured to generate a processor control signal for increasing at least one of supply power or frequency of the at least one of the one or more processing engine causing enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines when the performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines.
However, Baum teaches a processing engine perform calculation operation (the NN processor comprises a plurality of basis computation units doing the same or similar mathematical manipulations; paragraph [0098] and multiply/accumulate entity; [0157]).
Sunwood teaches monitoring the one or more processing engine clusters, the one or more processing engine groups, and the one or more processing engines to check performance related to the one or more processing engine clusters, and providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, according to the performance (“a processor core may be provided with a number of different hardware resources of various types which help to support improved processing performance for instruction execution”; paragraph [0108], “each core may have performance monitoring circuitry 120 comprising various performance counters 122 for gathering the performance monitoring data used as input to the machine learning model. Each performance counter 122 may be configurable to count instances of a particular type of performance event, such as a TLB miss, cache miss, branch mispredictions, etc.”; paragraph [0116], and “using a machine learning model 140 to control setting of control information for configuring a multi-core system 2 to select an amount of inter-core borrowing of resource which is used when processing particular workloads. The multi-core system may support a range of different hardware resource configurations which vary in terms of the types of resource (and the amount of resource of a given type) which is borrowed from another core or lent to another core”; paragraph [0117]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Sunwoo to the system of Baum because Sunwoo teaches a method for controlling inter-core borrowing of resource between processor cores in a multi-core processing system, where resource is borrowed between respective cores, thus, the performance of the system is increased.
Kwon teaches generate a processor control signal for increasing at least one of supply power or frequency of the at least one of the one or more processing engine causing enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines when the performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines (“the processor may perform a compute bound operation”; paragraph [0113], and “The processor 300 may previously acquire voltages, operating frequencies, or power levels of the processing device 200 with respect to the power efficiency and divide a sum of performances obtained under conditions that the processing devices 200 exhibit determined optimized power efficiencies by a total sum of power, thereby searching for a setting condition to achieve the determined optimized power efficiency. The processor 300 may verify whether a mean time between failure (MTBF), which indicates a range of normal operation of the processing devices 200, is reduced. When the MTBF is reduced, the processor 300 may determine whether the MTBF is less than a predicted operating time of a current workload. If the MTBF is shorter than an execution time of the workload, the processor 300 may increase the MTBF by adjusting the temperature or increasing the voltage”; paragraphs [0092]-[0093]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Kwon to the system of Baum and Sunwoo because Kwon teaches a method to avoid the failure of processing a workload.
Claims 2, 3 and 5-14 are rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Sunwoo et al. (US 2022/0035679 A1) and Kwon et al. (US 2023/0107333 A1) further in view of Guenther et al. (US 2021/0287423 A1).
As to claim 2, Baum as modified by Sunwoo and Kwon teaches the neural processing device of claim 1, wherein the sequencer circuitry is configured to monitor at least one of a bandwidth, latency, supply power, or temperature of the one or more processing engine clusters (see Kwon: The processor 300 may previously acquire voltages, operating frequencies, or power levels of the processing device 200; paragraph [0092]).
Baum as modified by Sunwoo and Kwon does not teach wherein the sequencer circuitry is further configured to cause checking performance between the one or more processing engine clusters and the interconnection.
Guenther teaches checking performance between the at least one processing engine and the interconnection, and performance between the at least one processing engine (see Fig. 22 illustrates how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region; paragraph [0254]-[0255]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Guenther to the system of Baum as modified by Sunwoo and Kwon because Baum teaches the system can dynamically allocate resources/memory/bandwidth to the PEs at runtime, and when apply the teaching of Guenther, the system of Baum could enhance the performance of the processing core/engine by monitoring data performance of the cores/engines in real time.
As to claim 3, Baum as modified by Sunwoo, Kwon and Guenther teaches wherein the sequencer circuitry is further configured to cause:
checking traffic between the interconnection and the one or more processing engine clusters to detect a performance problem based on the performance related to the one or more processing engine clusters (see Guenther: Fig. 22 illustrates how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region; paragraph [0254]-[0255]);
reducing traffic of the first memory or an off-chip memory exchanging data with the first memory when the performance problem is related to a bandwidth (see Baum: halt command; paragraph [0216]-[0217]); and
enhancing performance of the interconnection when the performance problem is related to the bandwidth (see Baum: dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]).
As to claim 5, Baum as modified by Sunwoo, Kwon and Guenther teaches wherein the reducing traffic of the off- chip memory includes generating a memory control signal for activating at least one of an operation of compressing traffic of the first memory or the off-chip memory or an operation of decompressing the traffic (halt command; paragraph [0216]-[0217]).
As to claim 6, Baum as modified by Sunwoo, Kwon and Guenther teaches wherein enhancing performance of the interconnection includes generating an interconnection control signal for increasing a frequency of the interconnection (dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]).
As to claim 7, Baum teaches a first memory (Layer 4 or L4 memory; paragraph [0108] and the NN processor includes shared memory; paragraph [0111]), an interconnection (cluster interconnect; paragraph [0192]), one or more processing engine clusters (clusters 66; Fig. 5), each of which includes one or more processing engine groups (subclusters), wherein each of the one or more processing engine groups includes one or more processing engines (The cluster comprises a plurality of M subclusters … a plurality of layers control circuits 256; paragraph [0187]); and
providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, and the at least one processing engine (dynamic resource assignment; paragraph [0019] and dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]; The subcluster comprises a plurality of N Pes 182; paragraph [0175] and (Figs. 8-9 and associated text));
wherein the first memory is shared by the one or more processing engine clusters (Layer 4 or L4 memory; paragraph [0108] and the NN processor includes shared memory; paragraph [0111]), and
wherein the interconnection is configured to transmit data between the first memory and the one or more processing engine clusters (cluster interconnect; paragraph [0192]).
Baum does not teach monitoring a first memory, an interconnect, one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, and the at least one processing engine, detecting a performance problem through the monitoring; and enhancing performance of the at least one of the one or more processing engine clusters by generating a processor control signal for increasing at least one of supply power or frequency of the one or more processing engine groups, or the one or more processing engines when the performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines.
However, Baum teaches the processing engine perform calculation (the NN processor comprises a plurality of basis computation units doing the same or similar mathematical manipulations; paragraph [0098] and multiply/accumulate entity; [0157]).
Sunwood teaches monitoring the one or more processing engine clusters, the one or more processing engine groups, and the one or more processing engines to check performance related to the one or more processing engine clusters, and providing hardware resources to at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines, according to the performance (“a processor core may be provided with a number of different hardware resources of various types which help to support improved processing performance for instruction execution”; paragraph [0108], “each core may have performance monitoring circuitry 120 comprising various performance counters 122 for gathering the performance monitoring data used as input to the machine learning model. Each performance counter 122 may be configurable to count instances of a particular type of performance event, such as a TLB miss, cache miss, branch mispredictions, etc.”; paragraph [0116], and “using a machine learning model 140 to control setting of control information for configuring a multi-core system 2 to select an amount of inter-core borrowing of resource which is used when processing particular workloads. The multi-core system may support a range of different hardware resource configurations which vary in terms of the types of resource (and the amount of resource of a given type) which is borrowed from another core or lent to another core”; paragraph [0117]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Sunwoo to the system of Baum because Sunwoo teaches a method for controlling inter-core borrowing of resource between processor cores in a multi-core processing system, where resource is borrowed between respective cores, thus, the performance of the system is increased.
Kwon teaches generate a processor control signal for increasing at least one of supply power or frequency of the at least one of the one or more processing engine causing enhancing performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines when the performance problem is related to a calculation performance of the at least one of the one or more processing engine clusters, the one or more processing engine groups, or the one or more processing engines (“the processor may perform a compute bound operation”; paragraph [0113], and “The processor 300 may previously acquire voltages, operating frequencies, or power levels of the processing device 200 with respect to the power efficiency and divide a sum of performances obtained under conditions that the processing devices 200 exhibit determined optimized power efficiencies by a total sum of power, thereby searching for a setting condition to achieve the determined optimized power efficiency. The processor 300 may verify whether a mean time between failure (MTBF), which indicates a range of normal operation of the processing devices 200, is reduced. When the MTBF is reduced, the processor 300 may determine whether the MTBF is less than a predicted operating time of a current workload. If the MTBF is shorter than an execution time of the workload, the processor 300 may increase the MTBF by adjusting the temperature or increasing the voltage”; paragraphs [0092]-[0093]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Kwon to the system of Baum and Sunwoo because Kwon teaches a method to avoid the failure of processing a workload.
Guenther teaches monitoring a neural core including processing a first memory, an interconnection, and a plurality of processing engine groups (Fig. 22 illustrates how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region; paragraph [0254]-[0255] and monitoring the progress of a workload on an engine; paragraph [0096]) wherein each of the plurality of processing engine groups includes at least one processing engine (a graphics processor having a graphics engine cluster having multiple instances of the graphics processing engine tiles; paragraph [0127]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Guenther to the system of Baum and Anderson because Baum teaches the system can dynamically allocate resources/memory/bandwidth to the PEs at runtime, and when apply the teaching of Guenther, the system of Baum could enhance the performance of the processing core/engine by monitoring data performance of the cores/engines in real time.
As to claim 8, Baum as modified by Sunwoo, Kwon and Guenther teaches wherein each of the one or more processing engines comprises an array of a plurality of processing elements interconnected by a mesh style network, the processing elements being reconfigurable (see Baum: The subcluster comprises a plurality of N Pes 182; paragraph [0175] and (Figs. 8-9 and associated text) and The PE comprises … allowing the order of calculations to be manipulated; paragraph [0160], i.e. reconfigurable; and each subcluster comprises flexible and programmable pathways for feeding input data and weights to the neurons in the Pes as well as steering intermediate results from the neurons to and from either L2 or L3 memory; paragraph [0182]).
As to claim 9, Baum as modified by Sunwoo, Kwon and Guenther does not clearly teach determining whether the performance problem is related to an off-chip memory; and reducing traffic of the off-chip memory when the performance problem is related to the off-chip memory.
However, Baum teaches reducing traffic of the memory when the performance problem is related to the memory (halt command; paragraph [0216]-[0217]).
Guenther teaches how a monitor running on each respective node collects performance metric data including, but not limited to, the time consumed to transmit data over the network interface, the time consumed when denoising a region, and the time consumed rendering each region/ghost region (paragraph [0254]-[0255]) and monitoring the progress of a workload on an engine (paragraph [0096]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Guenther to the system of Baum because Baum teaches a method to improve the performance of the system when there’s a problem with the memory of the system, and the system of Guenther could be modified to determine whether the performance problem is related to an off-chip memory.
As to claim 10, Baum as modified by Sunwoo, Kwon and Guenther teaches wherein reducing traffic of the off-chip memory includes activating a compression engine of traffic of the off-chip memory (see Baum: halt command is performed in response to the control signal; paragraph [0216]-[0217]).
As to claim 11, Baum as modified by Sunwoo, Kwon and Guenther does not clearly teach determining whether the performance problem is related to the first memory; and reducing traffic of the first memory when the performance problem is related to the first memory.
See rejection of claim 8 above for similar teaching, except this is related to the first memory instead of the off-chip memory.
As to claim 12, see rejection of claim 10 above for rejection of “wherein reducing traffic of the first memory includes activating a compression engine of traffic of the first memory”.
As to claim 13, Baum as modified by Sunwoo, Kwon and Guenther teaches enhancing performance of the interconnection when the performance problem is not related to the first memory (see Baum: dynamic bus width and memory bit cell, balanced capability of runtime configuration modification; paragraph [0189]-[0190] and [0202]).
As to claim 14, Baum as modified by Sunwoo, Kwon and Guenther does not teach wherein enhancing performance of the interconnection includes overdriving a frequency of the interconnection.
However, Baum teaches dynamic bus width and memory bit cell, balanced capability of runtime configuration modification (paragraph [0189]-[0190] and [0202]).
It would have been obvious to one of ordinary skill in the art that the system of Baum could be modified, based on the capability to reconfigure during runtime, to overdriving a frequency of the interconnection.
Claims 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Sunwoo et al. (US 2022/0035679 A1), Kwon et al. (US 2023/0107333 A1) and Guenther et al. (US 2021/0287423 A1) further in view of Heaton et al. (US 11,561,833 B1).
As to claim 15, Baum as modified by Sunwoo, Kwon and Guenther does not teach limitations of claim 15.
However, Heaton teaches wherein a compiler configuring the at least one processing engine is configured to perform: (the compiler engine can allocate the memory and computation resources for the neural network processing operations; col. 1, line 59 – col. 2, line 8)
receiving a deep learning graph (a compiler can obtain information of a dependency graph representing a neural network model as well as neural network operations; col. 11, lines 46-50);
storing a calculation code through processing compilation in a compute library (inherent from the compiler can obtain neural network operations and analyze the operations; col. 11, lines 46-61. Thus, the operations/calculation code must be stored);
generating intermediate representation (IR) by optimizing the deep learning graph (order of operations, generate instruction file, reformatting the input data, etc.; col. 11, line 55 -43 and col. 16, line 28 – col. 17, line 25);
performing, according to the IR, scheduling of a task between the plurality of processing engine clusters (runtime engine may perform resource placement; col. 17, line 26 – col. 18, line 55); and
generating a binary code according to the compute library on a circuit (inherent from the codes must be converted to binary in order for the system to execute the neural network operations).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching Heaton to the system of Baum as modified by Sunwoo, Kwon and Guenther because Heaton teaches a method to allocate and place resources of a computing environment for performing neural network processing operations of data.
As to claim 16, Baum as modified by Sunwoo, Kwon, Guenther and Heaton teaches wherein storing the calculation code in the compute library comprises: determining a dimension of each of the one or more processing engines; and performing scheduling of a task related to the one or more processing engine clusters. (see Heaton: col. 14, line 45 – col. 16, line 63).
As to claim 17, Baum as modified by Sunwoo, Kwon, Guenther and Heaton teaches wherein the determining the dimension of each of processing engines comprises determining a number of processing elements included in each of the one or more processing engines. (see Heaton: col. 14, lines 19-23).
As to claim 18, Baum as modified by Sunwoo, Kwon, Guenther and Heaton does not teach wherein the one or more processing engine groups are optimized through an L2 level scheduling.
However, Baum teaches processing cores may share certain resources such as L1 and L2 cache (paragraph [0145]). Thus, by scheduling to share the L2 level cache, the performance of the operations would be improved by having data/instruction obtained/stored to the nearby cache instead of off-chip memory.
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Baum et al. (US 2018/0285254 A1) in view of Sunwoo et al. (US 2022/0035679 A1) and Kwon et al. (US 2023/0107333 A1) further in view of Manousakis et al. (US 2021/0103458 A1).
As to claim 19, Baum as modified by Sunwoo and Kwon does not teach the at least one processing engine which implements at least one virtual processor (VP), and scale a frequency of the at least one processing engine in real time according to a status indicating a correspondence between the at least one VP and the at least one processing engine.
However, Manousakis teaches the at least one processing engine which implements at least one virtual processor (VP) (CPU-MEM may include processing cores and memory that may be assigned to each VM being hosted by host 310 … VMs may be assigned to respective group of cores; paragraph [0030] and virtual CPU; paragraph [0023]), and scale a frequency of the at least one processing engine in real time according to a status indicating a correspondence between the at least one VP and the at least one processing engine (Each chassis manager … Per-VM power controller may use per-core dynamic voltage frequency scaling to cap the per-core running … throttling just these VMs may be enough … protect the user-facing VMs; paragraph [0031] and [0053]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teaching of Manousakis to the system of Baum as modified by Sunwoo and Kwon because Manousakis teaches a method to fine-grained per-VM power capping system that protects the performance-critical workload, and criticality and utilization aware VM placement policy that distributes VMs across a cluster to reduce the number of expected capping events and their performance impact (paragraph [0017]).
As to claim 20, although Baum and modified by Sunwoo, Kwon and Manousakis does not teach wherein a number of the at least one processing engine is different from a number of the at least one VP, Manousakis teaches VMs may be assigned to respective group of cores (paragraph [0030]). Manousakis does not teach each VM can be assigned only one processing core. Therefore, it would have been obvious to one of ordinary skill in the art that a number of the at least one processing engine is different from a number of the at least one VP.
Response to Arguments
Applicant’s arguments with respect to claims 1-3 and 5-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DIEM K CAO whose telephone number is (571)272-3760. The examiner can normally be reached Monday-Friday 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, April Blair can be reached at 571-270-1014. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DIEM K CAO/Primary Examiner, Art Unit 2196
DC
July 20, 2026
Prosecution Insights