DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-3, 5-8, 11, 13-17, 19, and 21 are pending.
The U.S.C. 101 rejections have been corrected and the rejections are withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 16, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US 20190250691) in view of Diril et al. (US 20190205358) and Bera et al. (US 20210287031).
Regarding claim 1, Lee teaches
A system-on-chip (Figs. 5 and 12) comprising:
processing circuitry configured to process input data to generate output data; and (Fig 1 (160 – processor) and (130 – context hub))
power management circuitry configured to control power management policy for at least a portion of the system-on-chip, wherein controlling the power management policy comprises restricting throughput of the processing circuitry depending on monitoring of power-intensive events; in which: (Fig. 6 [0078], “the context hub 220 determines a DVFS level corresponding to the determined context input pattern CIP. The context hub 220 refers to information stored in the context input pattern information block 230 for the purpose of determining the DVFS level corresponding to the context input pattern CIP.” And [0032-33], “speech data may require faster processing than text data or image data. As another example, audio and video data received simultaneously by two different sensors may require faster processing than only audio data received by one sensor. … the context hub 130 may adjust a frequency of a clock signal CLK and/or a level of a driving voltage VDD depending on the context input pattern CIP. Here, the context hub 130 identifies the context input pattern CIP of currently input context data CD1 to CDn and CDa to CDd and compares the identified context input pattern CIP with information stored in the context input pattern information block 140.” Where restricting throughput is interpreted as using DVFS based on context input patterns (metadata) to reduce frequency of operation and monitoring a power intensive event is interpreted as determining whether faster or slower processing is required for operation based on the context input pattern.)
the input data comprises input data for a machine learning workload to be performed by the processing circuitry, wherein the machine learning workload implements a convolutional neural network; ([0117], “the NPU 370 may perform the machine learning on the DVFS level corresponding to the context input pattern CIP by using various machine learning techniques such as a neural network algorithm, a decision tree, a convolutional neural network (CNN)”)
the power management circuitry is configured to control the power management policy depending on metadata indicative of a property of the input data to be processed by the processing circuitry; (Fig. 6, [0032], “Types and/or combinations of input context data may vary in terms of the type of source of the input context data, a characteristic measured by a source of the input context data, a relative importance of the input context data, a size of the input context data, a required processing speed for the input context data, and other aspects described herein. Examples of input context data may include raw or processed data from a sensor, and may vary based on, for example, a type of source of the input context, a type of communication mechanism by which the input context data is received and/or a format of the input context data. A type of input context data may therefore also be particular to one or a subset (i.e., but not all) of the different sensors described herein. A combination of input context data may refer, for example, to a combination of input context data from different sensors, different sources, different types of communication mechanisms, and/or different formats. As an example, the type or combination of input context data may reflect the relative urgency to process the input context data.” And [0073], “The mobile device 200 includes the context hub 220 which changes a DVFS level depending on a context input pattern CIP of context data provided from sensors. The power which is consumed when sensors monitor ambient conditions during the sleep mode of the mobile device 200 may be markedly reduced by adjusting a level of the driving voltage VDD or a frequency of the clock signal CLK depending on the context input pattern CIP with regard to the context hub 220.”)
Lee does not teach but Diril teaches
in which the power management circuitry is configured to determine, based on the metadata associated with the input data to be processed by the processing circuitry, a load/store overhead estimate indicative of an estimated overhead associated with loading the input data or storing the output data, and control the power management policy based on the load/store overhead estimate. ([0053], “density-aware logical-unit 304 may process incoming vectors or matrices to determine their density (e.g., how many non-zero elements they have, the percentage of non-zero elements they have, etc.). Density-aware logical unit 304 may determine density in a variety of ways. For example, density-aware logical unit 304 may evaluate all or a portion of an input matrix or vector, may read metadata or receive other information indicative of the density of an input matrix or vector, etc. … a reduced dot-product operation, which may be more efficient than a full dot-product operation, may bypass, skip, or eliminate multiplication and/or accumulation operations for one or more zero-value element pairs of the input vectors”, [0055], “sparsity-aware logical unit 306 may distinguish between zero-value and non-zero element pairs and may only enqueue non-zero element pairs (or may exclude one or more zero-value element pairs from enqueuing) in queues 276(1)-(3) for processing may processing elements 274(1)-(3)”, [0057], “The term “in-line,” in some examples, generally refers to any storage unit that caches and/or accumulates partial sums in a manner that saves hits on other memory devices (e.g., SRAM 220)” And [0007], “Caching and accumulating partial sums inline (e.g., within an output bus architecture of the sparsity-aware dot-product engine), instead of storing them in a static random access memory (SRAM) device, may save the overhead associated with reading and writing to SRAM to accumulate each partial sum.” And [0044], “each of processing elements 274(1)-(3) may send its partial sum to a bus to be written to SRAM 220. Because each processing element 274(1)-(3) may complete calculating partial sums asynchronously, accumulating the partial sums may involve repeated accesses to SRAM 220 as each processing element finished calculating its partial sum. To reduce or minimize accesses to SRAM 220, dot-product engine 272 and/or an output bus from dot-product engine 272 may use an inline cache 292 to accumulate partial sums until a complete sum is accumulated and ready for writing to SRAM 220. In this way, an inline cache may reduce reads and/or writes to SRAM 220, thereby preserving memory bandwidth and reduce power consumption.” where load/store overhead indicative an estimated overhead associated with loading the input data or storing the output data is interpreted as determining whether to skip, bypass, or cache results resulting reduced accesses to the SRAM and reduced power consumption)
in which the load/store overhead estimate is indicative of an estimated overhead incurred by at least one of: a load/store unit configured to control issuing of load/store requests to a memory system; an interconnect configured to control routing of memory access requests across the system-on-chip; access to on-chip memory storage circuitry; access to off-chip memory storage circuitry; and loss of processing efficiency at the processing circuitry due to load/store delays. ([0054-55], “a reduced dot-product operation, which may be more efficient than a full dot-product operation, may bypass, skip, or eliminate multiplication and/or accumulation operations for one or more zero-value element pairs of the input vectors. … dot-product engine 310 may direct one or more of processing elements 274(1)-(3) to eliminate (e.g., bypass or skip) any or all mathematical operations involving zero-value elements (e.g., operations in which a zero-value element is an operand). Additionally or alternatively, sparsity-aware logical unit 306 may distinguish between zero-value and non-zero element pairs and may only enqueue non-zero element pairs (or may exclude one or more zero-value element pairs from enqueuing) in queues 276(1)-(3) for processing may processing elements 274(1)-(3).” [0057], “dot-product engine 272 may cache the partial sums in a cache inline with an output bus of dot-product engine 272. The term “in-line,” in some examples, generally refers to any storage unit that caches and/or accumulates partial sums in a manner that saves hits on other memory devices (e.g., SRAM 220).”)
Lee and Diril are analogous art. Diril is cited to teach a similar concept of controlling power in processors based on input data. Diril teaches reducing power by using metadata to determine the overhead related to sparsity and memory accesses. Diril determines whether to skip or bypass vectors with a zero-value or to cache a partial result where both reduce accesses to the SRAM and “save the overhead associated with reading and writing to SRAM”, [0044] Based on Diril, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee to evaluate overhead associated input data and with loading and storing data to the SRAM and to manage power in the system based on this information. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “an inline cache may reduce reads and/or writes to SRAM 220, thereby preserving memory bandwidth and reduce power consumption.” , [0044] and to “enhance efficiency, the computing system may include an inline cache configured to store and accumulate partial sums of output from the plurality of processing units.”, [0011]
Lee and Diril do not teach metadata is indicative of a property of inter-value differences between successive data values.
Bera teaches
the metadata is indicative of a property of inter-value differences between successive data values of at least one matrix of the input data to be processed in the machine learning workload; ([0051], “ the delta may be determined based in large part (or wholly) by using codec metadata included with the frames. In some embodiments, the deltas may be calculated based on (or be the same as) values that are used to show the differences between the analyzed frame and the frame which that frame references (e.g., in video formats that use keyframes). For example, assuming Frame N.sub.1 is a keyframe and Frame N.sub.2 references that keyframe, then the codec metadata could include the entire value N.sub.1 as the delta for Frame N.sub.1 because it does not reference any other frame, and the codec metadata could include only the difference between N.sub.1 and N.sub.2, which is n.sub.2, (rather than the entire N.sub.2) as the delta for Frame N.sub.2.”)
Lee, Diril, and Bera are analogous art. Bera is cited to teach a similar concept of video processing using machine learning to reduce resource usage. Bera teaches using metadata to determine inter-value differences between successive data for video analysis. Based on Bera, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to using metadata to determine inter-value differences between successive data for video analysis and to manage power in the system based on this information to reduce resource usage. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “to the frames in the video may result in a significant decrease in the number of times that the object detection algorithm needs to be applied and may reduce overall resource usage correspondingly.”, [0051]
Regarding claim 2, Lee teaches in which the power management circuitry is configured to select a power management policy setting for a forthcoming period based on the metadata indicative of a property of the input data to be processed by the processing circuitry in the forthcoming period. (Fig. 4, “Referring to FIG. 4, a frequency of the clock signal CLK of the context hub 130 (refer to FIG. 1) may be dynamically adjusted according to the context input pattern CIP. Here, a frequency of the clock signal CLK is illustrated as an example of the DVFS level, but a level of the driving voltage VDD may also (or alternatively) be changed together with (or instead of) the frequency of the clock signal CLK.”)
Regarding claim 3, Diril teaches in which the metadata is indicative of at least one of: sparsity of the input data; a compression property associated with a compressed version of the input data; a range or distribution of numeric values within the input data; and a property of inter-value differences between successive data values of the input data. ([0053], “For example, density-aware logical unit 304 may evaluate all or a portion of an input matrix or vector, may read metadata or receive other information indicative of the density of an input matrix or vector, etc. If density-aware logical unit 304 determines that at least one of the first and second vectors or matrices are dense (e.g., have less than predefined number or threshold of zero-value elements), density-aware logical unit 304 may cause sparsity-aware logical unit 306 to be bypassed or disabled.”)
Regarding claim 16, Lee teaches in which the power management policy controlled depending on the metadata comprises dynamic voltage scaling. ([0073], “The mobile device 200 includes the context hub 220 which changes a DVFS level depending on a context input pattern CIP of context data provided from sensors. The power which is consumed when sensors monitor ambient conditions during the sleep mode of the mobile device 200 may be markedly reduced by adjusting a level of the driving voltage VDD or a frequency of the clock signal CLK depending on the context input pattern CIP with regard to the context hub 220.”)
Regarding claim 19, Lee teaches in which the processing circuitry comprises at least one of: a central processing unit (CPU); a graphics processing unit (GPU); a hardware accelerator; and a neural processing unit (NPU). (Fig. 1, “may include one or more of a central processing unit (CPU)”)
Claim(s) 5-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee and Diril in view of Croxford (US 20190258306)
Regarding claim 5, Lee and Diril teach metadata but not metadata indicative of a property of kernel weights.
Croxford teaches in which the metadata is indicative of a property of kernel weights provided as the input data for a convolutional neural network. ([0041], “inputs which may be received by the neural network accelerator 108 via the interface 124 include image data representative of an image to be classified using the neural network and kernel data representative of a kernel associated with the neural network after training. … The kernel data may be considered to correspond to weight data representative of weights to be applied to image data, as each element of a kernel may be considered to correspond to a weight, respectively. Each of these weights may be multiplied by a corresponding pixel value of an image patch, as part of the convolution of the kernel with the image patch.”)
Lee and Croxford are analogous art. Croxford is cited to teach a similar concept of controlling power in processors. Based on Croxford, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to use metadata is indicative of a property of kernel weights provided as the input data for processing image data. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “to further reduce power consumption of the data processing system, at least one parameter used to determine which of the single-bit storage devices of the second input register 130b to place into the power saving state may be calculated based on the image data to be classified. For example, as explained above, metadata may be associated with the image data.”, [0052]
Regarding claim 6, Lee and Diril teaches using metadata for power control but does not teach the metadata is indicative of a property of input neural network data.
Croxford teaches in which the metadata is indicative of a property of input neural network data or at least one input feature map provided as the input data for a convolutional neural network. ([0052], “the metadata may include a histogram of an image intensity …the dynamic range may be determined by calculating the difference between the maximum and minimum intensity, based on the histogram. In examples such as this, the image data size of the image may be determined by the control system 134 of the data processing system before the image data is transferred to the second input register 130b”)
Lee and Croxford are analogous art. Croxford is cited to teach a similar concept of controlling power in processors neural networks. Based on Croxford, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to use determine a power policy based on image classified image data with metadata assisting the classification. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “to further reduce power consumption of the data processing system, at least one parameter used to determine which of the single-bit storage devices of the second input register 130b to place into the power saving state may be calculated based on the image data to be classified. For example, as explained above, metadata may be associated with the image data.”, [0052]
Regarding claim 7, Lee and Diril does not teach but Croxford teaches in which the power management circuitry is configured to make separate power management policy decisions for processing of respective portions of the input data, based on portion metadata associated with the respective portions.([0052], “ For example, the metadata may indicate that the image has been compressed to generate the image data, and may indicate a value of the image data size of the image data or a range of values associated with image data representative of a region of an image. In other examples, the metadata may include a histogram of an image intensity, for example where the image has been generated using an image signal processor (ISP), such as an ISP of a digital camera. This histogram may therefore may be used to determine a dynamic range of the image (or a portion of the image, where the image data represents a portion of an image rather than an entire image). For example, the dynamic range may be determined by calculating the difference between the maximum and minimum intensity, based on the histogram. … In examples such as this, a plurality of image portions may be processed (for example by convolving each image portion with a plurality of kernels).”)”)
Lee and Croxford are analogous art. Croxford is cited to teach a similar concept of controlling power in processors. Based on Croxford, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to use determine a power policy based on image classified image data with metadata assisting the classification. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “to further reduce power consumption of the data processing system, at least one parameter used to determine which of the single-bit storage devices of the second input register 130b to place into the power saving state may be calculated based on the image data to be classified. For example, as explained above, metadata may be associated with the image data.”, [0052]
Claim(s) 8 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee and Diril in view of Heilper et al. (US 20190370086)
Regarding claim 8, Lee and Diril teaches power management circuitry but does not teach estimating a compute overhead associated with output data.
Heilper teaches.in which the power management circuitry is configured to determine, based on the metadata associated with the input data to be processed by the processing circuitry, a compute overhead estimate indicative of an estimated overhead associated with computation of the output data based on the input data, and control the power management policy based on the compute overhead. ([0043], “For example, an initial portion of the workload may be bandwidth intensive and involve much memory access, while a later portion of the workload may be computed intensive and involve one or more accelerators 232-236 and/or other processor 238 in the processor system 110. The power management processor 222 consolidates power information for one or more running workloads based on the workload information 210 and triggers the power table generator 224 to generate and/or update the power table 240 when workload(s) executing on the SoC 200 change, for example. The power management processor 222 reads a current power level from the power controller 260 and determines new power levels by consolidating workload requests and associated power characteristics.”, [0070], “allocating power in the apparatus 100 to execute the workload. For example, the power manager 220 receives the workload executable and associated meta-data from the compiler 130 and determines the power table to store in power table memory 240 using the meta-data associated with the workload for execution.”, [0040], “the analyzer 213 can determine dynamic voltage and frequency scaling (DVFS) transition/optimization points and associated frequency or work ratios from the code analysis. For example, by analyzing machine instructions, the analyzer 213 can determine which operations involve computation and which operations involve memory access” where the overhead estimate is interpreted as the memory accesses)
Lee and Heilper are analogous art. Lee is cited to teach a similar concept of controlling power in processors. Based on Heilper, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to use determine an overhead estimate based on memory access to balance compute activities with the memory accesses to balance performance and power usage. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “the analyzer 213 can determine an improved (e.g., optimal or other desirable) accelerator 232-236 frequency for a given memory 120 frequency to be involved in the workload layer.”
Regarding claim 21, Lee and Diril using metadata of input data for a power mitigation policy to reduce power in the system but does not teach determining whether to restrict throughput.
Heilper teaches in which the power management policy comprises a maximum power mitigation policy, where when the maximum power mitigation policy is active, the power management circuitry is configured to select, depending on monitoring of power-intensive events, whether to restrict throughput of the processing circuitry; and the power management circuitry is configured to select a setting for the maximum power mitigation policy depending on the metadata indicative of the property of the input data. ([0088], “ if the SoC 200 exceeds its maximum power allocation, then, at block 850, a lower index 241 is selected from the power table. For example, a lower index 241 corresponding to a lesser and/or different distribution of power can be selected by the selector 266 in the power table memory 240. At block 855, the index 241 is updated to reflect a new power allocation from the power table,” and [0070], “the example program 500 includes allocating power in the apparatus 100 to execute the workload. For example, the power manager 220 receives the workload executable and associated meta-data from the compiler 130 and determines the power table to store in power table memory 240 using the meta-data”)
Lee and Diril and Heilper are analogous art. Heilper is cited to teach a similar concept of controlling power in artificial intelligence accelerators by determining a maximum power allocation. Based on Heilper, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to determine whether to restrict throughput/throttle parts of the system when the maximum power allocation is exceeded. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “the analyzer 213 can determine an improved (e.g., optimal or other desirable) accelerator 232-236 frequency for a given memory 120 frequency to be involved in the workload layer.”
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over, Lee and Diril further in view of Appu et al. (US 20210103550).
Regarding claim 11, Lee and Diril teach using metadata but do not teach using predicates used form masking operations for power saving.
Appu teaches comprising predicate generating circuitry to generate, based on the metadata, predicates for predicated instructions to be supplied to the processing circuitry for processing the input data, the predicates indicating one or more inactive elements of the input data for which processing operations are to be masked. ([0404], “ the instruction 3500 includes a predicate field 3502, ... The predicate field 3502 can specify a predicate mask 3562, as in FIG. 35B. Turning to FIG. 35B, the predicate mask can be used to enable or disable an execution channel of the instruction, such output generation for a specific element is disabled.”)
Lee, Diril, and Appu are analogous art. Appu is cited to teach a similar concept of controlling power and efficiency in artificial intelligence accelerators by determining a predicate mask for the input data/metadata. Based on Appu, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to determine whether to a predicate instruction to mask operations/functional units based on the sparsity of the input data/metadata. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to use “techniques [are] applied in the art of deep neural networks (DNNs) to improve computational speed and/or efficiency while maintaining acceptable accuracy of the results … expansion for specific instructions in the set of instructions can be bypassed, for example, in the instance that zero skipping is performed for sparse input.”, [0398] and [0418]
Claim(s) 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee and Diril in view of Rotem et al. (US 20190187775).
Regarding claim 13, Lee and Diril teaches power management a processor but does not teach a setting for the maximum power mitigation policy.
Rotem teaches in which the setting for the maximum power mitigation policy comprises at least one of: a setting for controlling detection or weighting of the power-intensive events; at least one comparison criterion for determining, based on the monitoring data, whether to restrict the throughput; and a throughput limit indicative of a maximum throughput allowed when the power management circuitry determines that throughput of the processing circuitry should be restricted. ([0046], “Functional unit utilization may also be represented in contextual terms (e.g., via an indication of an optimal frequency or clock rate for a particular utilization workload, a voltage value or range needed for optimal or stable processing of a particular workload, a scaling value that may optimize or maximize one or more functional parameters such as throughput or accuracy, etc.).”
Lee, Diril and Rotem are analogous art. Rotem is cited to teach a similar concept of controlling power in artificial intelligence accelerators by adjusting parameters to . Based on Rotem, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to determine whether to restrict throughput/throttle parts of the system when the maximum power allocation is exceeded. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification because “the systems disclosed herein may modify a variety of power and/or performance characteristics of a functional unit or AI accelerator (including frequency, clock rate, voltage, etc.) in order to accomplish any number of performance and/or power-related goals and objectives, including to decrease total computation time, optimize energy consumption, reduce heat generation”, [0052]
Regarding claim 14, Lee and Diril teach power policies which control throughput based on input metadata but do not teach more strongly restricting throughput for a first condition than a second condition.
Rotem teaches the power management circuitry is configured to control the maximum power mitigation policy to favor more strongly restricting the throughput when the metadata indicates a first condition than when the metadata indicates a second condition of the input data. (Fig. 4, [0049], “How much power the MAC unit will use may then be predicted based on the sparsity (i.e., the amount of zero-value elements) within a weight matrix. In certain operations, such as multiply-accumulate operations performed by a MAC unit, zero-value elements will produce a value of zero when operated on. As such, sparse elements may not require as much processing power (e.g., computations) non-sparse elements, and therefore the sparsity of a weight matrix may be indicative of the power-usage requirement for a particular AI workload. Accordingly, in some examples the disclosed systems may predict the power-usage requirement for a particular instruction stream by (1) identifying at least one element of sparsity within the instruction stream and then (2) predicting a power-usage requirement for a functional unit and/or set of functional units (e.g., an entire AI accelerator) based at least in part on the identified element of sparsity.” And [0051], “the disclosed systems may account for the maximum power envelope of a functional unit or AI accelerator when scaling its frequency and/or voltage.”, where the less sparsity indicate more power usage and therefore more need to throttle the system below a maximum power)
in which in the first condition, the metadata indicates less sparse input data than in the second condition; in the first condition, the metadata indicates input data supporting a smaller level of compression than in the second condition; in the first condition, the metadata indicates that the input data has a wider range of numeric values than in the second condition; in the second condition, the metadata indicates that a distribution of numeric values within the input data is clustered more heavily around zero than in the first condition; or in the first condition, the metadata indicates that an average of inter-value differences between successive data values of the input data is greater than in the second condition. ([0049], “How much power the MAC unit will use may then be predicted based on the sparsity (i.e., the amount of zero-value elements) within a weight matrix. In certain operations, such as multiply-accumulate operations performed by a MAC unit, zero-value elements will produce a value of zero when operated on. As such, sparse elements may not require as much processing power (e.g., computations) non-sparse elements, and therefore the sparsity of a weight matrix may be indicative of the power-usage requirement for a particular AI workload. Accordingly, in some examples the disclosed systems may predict the power-usage requirement for a particular instruction stream by (1) identifying at least one element of sparsity within the instruction stream and then (2) predicting a power-usage requirement for a functional unit and/or set of functional units (e.g., an entire AI accelerator) based at least in part on the identified element of sparsity.” And [0051], “the disclosed systems may account for the maximum power envelope of a functional unit or AI accelerator when scaling its frequency and/or voltage.”, where the less sparsity indicate more power usage and therefore more need to throttle the system below a maximum power)
Lee, Diril and Rotem are analogous art. Rotem is cited to teach a similar concept of controlling power in artificial intelligence accelerators. Rotem teaches that when sparsity is low, computations increase with a risk of exceeding a power maximum and that frequency should be reduced at this point. Based on Rotem, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Diril to determine to decrease throughput/frequency when the sparsity is low. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification because “the systems disclosed herein may modify a variety of power and/or performance characteristics of a functional unit or AI accelerator (including frequency, clock rate, voltage, etc.) in order to accomplish any number of performance and/or power-related goals and objectives, including to decrease total computation time, optimize energy consumption, reduce heat generation”, [0052]
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Takase et al. (US 2007022093) and Bera et al. (US 20210287031).
Regarding claim 15, Lee teaches
processing circuitry to process input data to generate output data; and (Fig 1 (160 – processor) and (130 – context hub))
power management circuitry to control power management policy for at least a portion of the system-on-chip, in which: (Fig. 6 [0078], “the context hub 220 determines a DVFS level corresponding to the determined context input pattern CIP. The context hub 220 refers to information stored in the context input pattern information block 230 for the purpose of determining the DVFS level corresponding to the context input pattern CIP.”)
the input data comprises input data for a machine learning workload to be performed by the processing circuitry, wherein the machine learning workload implements a convolutional neural network; ([0117], “the NPU 370 may perform the machine learning on the DVFS level corresponding to the context input pattern CIP by using various machine learning techniques such as a neural network algorithm, a decision tree, a convolutional neural network (CNN)”)
the power management circuitry is configured to control the power management policy depending on metadata indicative of a property of the input data to be processed by the processing circuitry; (Fig. 6, [0032], “Types and/or combinations of input context data may vary in terms of the type of source of the input context data, a characteristic measured by a source of the input context data, a relative importance of the input context data, a size of the input context data, a required processing speed for the input context data, and other aspects described herein. Examples of input context data may include raw or processed data from a sensor, and may vary based on, for example, a type of source of the input context, a type of communication mechanism by which the input context data is received and/or a format of the input context data. A type of input context data may therefore also be particular to one or a subset (i.e., but not all) of the different sensors described herein. A combination of input context data may refer, for example, to a combination of input context data from different sensors, different sources, different types of communication mechanisms, and/or different formats. As an example, the type or combination of input context data may reflect the relative urgency to process the input context data.” And [0073], “The mobile device 200 includes the context hub 220 which changes a DVFS level depending on a context input pattern CIP of context data provided from sensors. The power which is consumed when sensors monitor ambient conditions during the sleep mode of the mobile device 200 may be markedly reduced by adjusting a level of the driving voltage VDD or a frequency of the clock signal CLK depending on the context input pattern CIP with regard to the context hub 220.”)
the power management circuitry is configured to control power management policy by causing the processing circuitry to perform one or more dummy operations, based on the metadata indicative of the property of the input data, whether to limit a rate of change in power ([0073], “The mobile device 200 includes the context hub 220 which changes a DVFS level depending on a context input pattern CIP of context data provided from sensors. The power which is consumed when sensors monitor ambient conditions during the sleep mode of the mobile device 200 may be markedly reduced by adjusting a level of the driving voltage VDD or a frequency of the clock signal CLK depending on the context input pattern CIP with regard to the context hub 220.”)
Lee teaches power control of a neural network using metadata indicative of input data but does not teach power control by limiting a rate of change by using dummy operations.
Takase teaches in which the power management circuitry is configured to control the power management policy by causing the processing circuitry to perform one or more dummy operations, based on the metadata indicative of the property of the input data, to limit a rate of change in power requirement. ([0010], “selection and execution of one or more dummy programs to reduce the magnitude of changes in the total current drawn by the processor cores, or to minimize changes in voltage across the integrated circuit chip on which the multiprocessor is constructed.”)
Lee and Takase are analogous art. Takase is cited to teach a similar concept of controlling power in a processing system. Takase teaches that one way to control power/current in a system is to perform dummy operations (i.e. a reduction in frequency). Lee teaches controlling power via controlling frequency and based on a property of input data . Dummy operations are specific way to reduce the frequency of operation in the system. Based on Takase, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee to use dummy cycles to control reduce the frequency of operation in the system when controlling power based on input data property/types . Furthermore, limiting the rate of change of the current improves on Lee by being able to be smaller and more cost effective. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “modes of software programs to improve power consumption associated with execution of the programs”, Abstract
Lee and Takase do not teach metadata is indicative of a property of inter-value differences between successive data values.
Bera teaches
the metadata is indicative of a property of inter-value differences between successive data values of at least one matrix of the input data to be processed in the machine learning workload; ([0051], “ the delta may be determined based in large part (or wholly) by using codec metadata included with the frames. In some embodiments, the deltas may be calculated based on (or be the same as) values that are used to show the differences between the analyzed frame and the frame which that frame references (e.g., in video formats that use keyframes). For example, assuming Frame N.sub.1 is a keyframe and Frame N.sub.2 references that keyframe, then the codec metadata could include the entire value N.sub.1 as the delta for Frame N.sub.1 because it does not reference any other frame, and the codec metadata could include only the difference between N.sub.1 and N.sub.2, which is n.sub.2, (rather than the entire N.sub.2) as the delta for Frame N.sub.2.”)
Lee, Takase, and Bera are analogous art. Bera is cited to teach a similar concept of video processing using machine learning to reduce resource usage. Bera teaches using metadata to determine inter-value differences between successive data for video analysis. Based on Bera, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Takase and Diril to using metadata to determine inter-value differences between successive data for video analysis and to manage power in the system based on this information to reduce resource usage. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “to the frames in the video may result in a significant decrease in the number of times that the object detection algorithm needs to be applied and may reduce overall resource usage correspondingly.”, [0051]
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Rotem et al. (US 20190187775) and Bera et al. (US 20210287031).
Regarding claim 17, Lee teaches
A system-on-chip (Figs. 5 and 12) comprising:
processing circuitry to process input data to generate output data; and (Fig 1 (160 – processor) and (130 – context hub))
power management circuitry to control power management policy for at least a portion of the system-on-chip, in which: (Fig. 6 [0078], “the context hub 220 determines a DVFS level corresponding to the determined context input pattern CIP. The context hub 220 refers to information stored in the context input pattern information block 230 for the purpose of determining the DVFS level corresponding to the context input pattern CIP.”)
the input data comprises input data for a machine learning workload to be performed by the processing circuitry, wherein the machine learning workload implements a convolutional neural network; ([0117], “the NPU 370 may perform the machine learning on the DVFS level corresponding to the context input pattern CIP by using various machine learning techniques such as a neural network algorithm, a decision tree, a convolutional neural network (CNN)”)
the power management circuitry is configured to control the power management policy depending on metadata indicative of a property of the input data to be processed by the processing circuitry; (Fig. 6, [0032], “Types and/or combinations of input context data may vary in terms of the type of source of the input context data, a characteristic measured by a source of the input context data, a relative importance of the input context data, a size of the input context data, a required processing speed for the input context data, and other aspects described herein. Examples of input context data may include raw or processed data from a sensor, and may vary based on, for example, a type of source of the input context, a type of communication mechanism by which the input context data is received and/or a format of the input context data. A type of input context data may therefore also be particular to one or a subset (i.e., but not all) of the different sensors described herein. A combination of input context data may refer, for example, to a combination of input context data from different sensors, different sources, different types of communication mechanisms, and/or different formats. As an example, the type or combination of input context data may reflect the relative urgency to process the input context data.” And [0073], “The mobile device 200 includes the context hub 220 which changes a DVFS level depending on a context input pattern CIP of context data provided from sensors. The power which is consumed when sensors monitor ambient conditions during the sleep mode of the mobile device 200 may be markedly reduced by adjusting a level of the driving voltage VDD or a frequency of the clock signal CLK depending on the context input pattern CIP with regard to the context hub 220.”)
Lee teaches lower power operation of a neural network using metadata by controlling frequency and voltage but does not teach determining activity level of processing elements.
Rotem teaches in which the processing circuitry comprises a plurality of execution engines (Fig. 3 (130 – Functional units)), and the power management policy controlled depending on the metadata indicative of input data comprises control of how many of the plurality of execution engines are active. ([0048-49], “ one or more of the functional units within the disclosed AI accelerator may be power gated so that they only draw power when in use. For example, hardware accelerator 200 may be designed to shut off blocks of current to functional units 130 when they are not in use. … the power-usage requirement for a functional unit and/or AI accelerator may be predicted based on the data that is to be processed. … the disclosed systems may predict the power-usage requirement for a particular instruction stream by … predicting a power-usage requirement for a functional unit and/or set of functional units (e.g., an entire AI accelerator) based at least in part on the identified element of sparsity.”)
Lee and Rotem are analogous art. Rotem is cited to teach a similar concept of controlling power in artificial intelligence accelerators. Rotem teaches that using sparsity of input data to determine power usage for a functional unit (execution engine) where by frequency can be reduced or functional blocks can be shut off. Based on Rotem, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to control the enabling/disabling of functional units and dependent on the type of input data (i.e. sparsity).To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification because “the systems disclosed herein may modify a variety of power and/or performance characteristics of a functional unit or AI accelerator (including frequency, clock rate, voltage, etc.) in order to accomplish any number of performance and/or power-related goals and objectives, including to decrease total computation time, optimize energy consumption, reduce heat generation”, [0052] and “hardware accelerator 200 may be designed to shut off blocks of current to functional units 130 when they are not in use. By doing so, hardware accelerator 200 may reduce or eliminate standby power consumption and/or power leakage.”, [0048]
Lee and Rotem do not teach metadata is indicative of a property of inter-value differences between successive data values.
Bera teaches
the metadata is indicative of a property of inter-value differences between successive data values of at least one matrix of the input data to be processed in the machine learning workload; ([0051], “ the delta may be determined based in large part (or wholly) by using codec metadata included with the frames. In some embodiments, the deltas may be calculated based on (or be the same as) values that are used to show the differences between the analyzed frame and the frame which that frame references (e.g., in video formats that use keyframes). For example, assuming Frame N.sub.1 is a keyframe and Frame N.sub.2 references that keyframe, then the codec metadata could include the entire value N.sub.1 as the delta for Frame N.sub.1 because it does not reference any other frame, and the codec metadata could include only the difference between N.sub.1 and N.sub.2, which is n.sub.2, (rather than the entire N.sub.2) as the delta for Frame N.sub.2.”)
Lee, Rotem, and Bera are analogous art. Bera is cited to teach a similar concept of video processing using machine learning to reduce resource usage. Bera teaches using metadata to determine inter-value differences between successive data for video analysis. Based on Bera, it would have been obvious before the effective filing date of the invention to a person having ordinary skill in the art to which said subject matter pertains to have modified Lee and Rotem to using metadata to determine inter-value differences between successive data for video analysis and to manage power in the system based on this information to reduce resource usage. To one of ordinary skill in the art before the effective filing data of the invention it would have been advantageous to make this modification to “to the frames in the video may result in a significant decrease in the number of times that the object detection algorithm needs to be applied and may reduce overall resource usage correspondingly.”, [0051]
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 15, and 17 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHERI L. HARRINGTON whose telephone number is (571)270-0468. The examiner can normally be reached Generally, M-F, 7:30a-4p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jaweed Abbaszadeh can be reached at 571-270-1640. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHERI L HARRINGTON/Examiner, Art Unit 2176 December 28, 2025
/PHIL K NGUYEN/Primary Examiner, Art Unit 2176