CTNF 18/133,616 CTNF 90197 DETAILED ACTION 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. This Office Action is sent in response to Applicant’s Communication received 4/12/2023 for application number 18/133,616. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, and claims. Claims 1 – 23 are presented for examination. Title The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. The examiner believes that the title of the invention is imprecise. A descriptive title indicative of the invention will help in proper indexing, classifying, searching, etc. See MPEP 606.01. However, the title of the invention should be limited to 500 characters. Drawings Examiner contends that the drawings filed 4/12/2023 are acceptable for examination proceedings. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15-aia AIA Claim(s) 1-2, 7-8, 15-17, and 20 is/are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Croxford et al. (hereinafter as Croxford ) PGPUB 2022/0129321 . As per claim 1 , Croxford teaches an apparatus comprising: an accelerator, wherein the accelerator is configurable to load and execute a neural network [0002, 00338, and 0055: (neural processing unit (accelerator) executes a neural network)] ; and circuitry [FIG. 2: (receiving interface of processing unit)] configured to: receive a profile of the neural network, the profile including information regarding a plurality of layers of the neural network [0059, 0069, and 0071: (neural processor driver 25 estimates size of first layer, and receiving circuitry of the processing unit 24 in the neural processing unit receives in advance the layer size and weights (profile, which includes size and weighting information for a plurality of layers)] ; and responsive to the profile and the information regarding the plurality of layers, adjust a power level to the accelerator while the accelerator executes the neural network [0072: (processor frequency is adjusted in dependence on the processor load, varying the processing time for the current layer to match the memory time for the same layer)] . As per claim 2 , Croxford teaches the apparatus of claim 1, wherein the circuitry is further configured to: determine, using the profile, whether a particular layer of the plurality of layers is a compute-intensive layer or a memory-intensive layer [0064, 0067-0068, 0071-0073, and 0116: (when the layer is memory intensive (i.e. high memory time, as determined by layer size) and is the limiting factor, frequency may be lowered to so that processing is not stalled)]. As per claim 7 , Croxford teach the apparatus of claim 1, wherein the profile includes a layer-by-layer analysis of the neural network, the layer-by-layer analysis including one or more statistics for one or more respective layers of the plurality of layers of the neural network [0059, 0064, and 0071-0072: (process is repeated for each layer of the neural network that is to be processed, and layer size, memory time, and processing time (statistics of a layer) is calculated once per layer in advance of any processing being performed)]. As per claim 8 , Croxford teach the apparatus of claim 7, wherein the one or more statistics include an amount of hardware efficiency at the one or more respective layers of the neural network [0071, 0079], an amount of hardware utilization at the one or more respective layers of the neural network [0064: throughput], a number of compute cycles required to execute the one or more respective layers of the neural network [0102: number of cycles], a number of data cycles required to read or write weights or activations from a memory cache at the one or more respective layers of the neural network [0064, 0066, and 0068: memory time]. Claim 15 is similar in scope to claim 7 as addressed above and is thus rejected under the same rationale. Claim 16 is similar in scope to claim 8 as addressed above and is thus rejected under the same rationale. Claim 17 is similar in scope to claim 1 as addressed above and is thus rejected under the same rationale. Claim 20 is similar in scope to claim 8 as addressed above and is thus rejected under the same rationale . 07-15-aia AIA Claim(s) 21-22 is/are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Heilper et al. (hereinafter as Heilper ) PGPUB 2019/0370086 . As per claim 21 , Heilper teach a non-transitory computer-readable medium with instructions stored thereon that configures operations of a compiler [0097-0098] , the operations to: receive data corresponding to a plurality of layers of a neural network [0077-0078: (workload for layers of neural network is inputted into a compiler to analyze information)] ; compile respective layers of the plurality of layers into an executable form of the neural network [0040 and 0077: (compile or generate an executable)] ; determine, during the compiling, whether one or more of the respective layers is a compute-intensive layer or a memory-intensive layer [0026, 0043, and 0071, and 0078: (compiler evaluates whether it is memory-intensive or compute-intensive and determines whether transition is needed)] ; and generate a profile of the compiled one or more respective layers [0069, 0078, 0107, and FIG. 2A: (compiler analyzes workload for each layer and outputs meta-data (profile)] . As per claim 22 , Heilper teach the non-transitory computer-readable medium of claim 21, the operations further to: transmit the profile to a device configured to execute the neural network [FIG. 2A, 0040, 0042, and 0069: (workload information which includes the metadata (profile) is provided to the SOC that executes the neural network)] . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim (s) 3, 6, 11, 14, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Croxford et al. (hereinafter as Croxford ) PGPUB 2022/0129321, and further in view of Heilper et al. (hereinafter as Heilper ) PGPUB 2019/0370086 . As per claim 3, Croxford teach the apparatus of claim 2. Croxford does not explicitly teach further comprising: a local power management unit (PMU) configured to adjust the power level to the accelerator, and wherein the circuitry causes the PMU to adjust the power level. Croxford does not appear to describe a power management circuitry. Heilper teaches circuitry to manage frequency and power of deep learning accelerator systems on a layer by layer basis. Heilper is thus similar to Croxford. Heilper further teaches a local power management unit (PMU) configured to adjust the power level to the accelerator, and wherein the circuitry causes the PMU to adjust the power level [FIG. 2A: (power controller 260) and 0046, 0048, 0051, 0055, and 0057: (power controller can control and adjust or allocate the amount of power or frequency going to an accelerator)]. The combination of Croxford with Heilper leads to the processor unit in Croxford containing a power controller to effect the frequency adjustments. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Heilper’s teachings of providing a power controller to make the frequency adjustments in Croxford. Heilper teaches the mechanism for making the frequency adjustments in Croxford. One of ordinary skill in the art would have been motivated to use a power controller to make the adjustments in Croxford because it allows a dedicated component to focus on controlling the power and making frequency adjustments to the accelerator, thereby freeing the main processor components to focus on the other computational tasks of the neural network. As per claim 6 , Croxford teach the apparatus of claim 1, wherein the profile is generated by a compiler [0050: (the driver 25 (which analyzes and estimates the size of the layer) may be a compiler]. Croxford does not explicitly teach wherein an amount of compute bandwidth and an amount of memory bandwidth is determined when the neural network is compiled. Although Croxford mentions the bandwidth for processing and memory [0075-0076, 0087-0088, and 0109], Croxford does not mention that the bandwidth are determined by the compiler. Heilper teaches circuitry to manage frequency and power of deep learning accelerator systems on a layer by layer basis. Heilper is thus similar to Croxford. Heilper further teaches wherein an amount of compute bandwidth and an amount of memory bandwidth is determined when the neural network is compiled [0025-0026, 0037-0038, and 0040: (compiler determines frequency ratio for bandwidth resources for computing and memory tasks)]. Heilper teaches a compiler determining bandwidth resources to be allocated to memory and computation tasks. The combination of Croxford with Heilper leads to the compiler in Croxford determining the amount of bandwidth resources to provide to memory and computing tasks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Heilper’s teachings of the compiler determining the bandwidth for memory and computing tasks in Croxford. One of ordinary skill in the art would have been motivated to have the compiler in Croxford make the determination because the compiler would be aware of the upcoming tasks to be performed and can thus optimize management of resources to improve performance and manage power. As per claim 11 , Croxford teaches a method for dynamic power management of a neural network, the method comprising: receiving a profile of the neural network, the profile including information regarding a plurality of layers of the neural network [0059, 0069, and 0071: (neural processor driver 25 estimates size of first layer, and receiving circuitry of the processing unit 24 in the neural processing unit receives in advance the layer size and weights (profile, which includes size and weighting information for a plurality of layers)] ; and responsive to the profile and the information regarding the plurality of layers, adjusting a power level to an accelerator configured to load and execute the neural network as the accelerator executes the neural network [0002, 00338, and 0055: (neural processing unit (accelerator) executes a neural network) and 0072: (processor frequency is adjusted in dependence on the processor load, varying the processing time for the current layer to match the memory time for the same layer)] . Croxford does not explicitly teach the accelerator included on a visual processing unit (VPU) being configured to load and execute the neural network. Croxford describes a variety of processing units that may be used but does not mention a VPU. Heilper teaches circuitry to manage frequency and power of deep learning accelerator systems on a layer by layer basis. Heilper is thus similar to Croxford. Heilper further teaches teach the accelerator included on a visual processing unit (VPU) being configured to load and execute the neural network [FIG. 2A and 0013-0014: (VPU may be implemented as an SOC, such as SOC 200)]. Heilper shows a SOC that may be a VPU including an accelerator that executes the neural network. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Heilper’s teachings of the processing device containing the accelerator to be a VPU in Croxford. One of ordinary skill in the art would have been motivated to use a VPU in Croxford because it is a particular field of use or AI application, and using a VPU would allow for better and more efficient processing of video and image data on a neural network. Claim 14 is similar in scope to claim 6 as addressed above and is thus rejected under the same rationale. Claim 19 is similar in scope to claim 6 and 7 combined as addressed above and is thus rejected under the same rationale . 07-21-aia AIA Claim (s) 4-5 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Croxford et al. (hereinafter as Croxford ) PGPUB 2022/0129321 in view of Heilper et al. (hereinafter as Heilper ) PGPUB 2019/0370086, and further in view of Chen et al. (hereinafter as Chen ) PGPUB 2020/0104167 . As per claim 4 , Croxford and Heilper teach the apparatus of claim 3, wherein the accelerator and the circuitry are implemented by a visual processing unit (VPU) [Heilper 0013-0014: (VPU may be implemented as an SOC, such as SOC 200)], wherein the PMU is located on the VPU [Heilper FIG. 2A: (power controller 260 (PMU) is located on SOC 200 (VPU)]. Croxford and Heilper do not explicitly teach wherein the neural network is adapted for processing of image or video data. Chen teaches adjusting frequency to an accelerator if it determines a layer is compute-intensive. Chen is thus similar to Croxford and Heilper. Chen further teaches wherein the neural network is adapted for processing of image or video data [0229, 0262, 0271, 0585 and 0610: (neural network takes as an input, an input picture, and thus the neural network is adapted to process an image)]. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Chen’s teachings of the neural network processing image or picture data in Croxford and Heilper. One of ordinary skill in the art would have been motivated to have the neural network in Croxford and Heilper process an input picture because it allows processing of a particular type of data thereby improving its versatility and applicability, and allows for performing advanced AI features such as facial recognition. As per claim 5 , Croxford, Heilper, and Chen teach the apparatus of claim 4, wherein to adjust the power level to the accelerator includes adjusting at least one of a voltage level or a frequency of a signal to the accelerator while the accelerator executes the particular layer of the neural network based on whether the particular layer is a compute-intensive layer or a memory-intensive layer [Croxford 0071-0072 and 0116: (frequency is adjusted when memory time is the limiting factor (memory intensive layer); or Chen 0193 or Heilper 0026 and 0043]. Claim 12 is similar in scope to claim 5 as addressed above and is thus rejected under the same rationale . 07-21-aia AIA Claim (s) 9-10, 13, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Croxford et al. (hereinafter as Croxford ) PGPUB 2022/0129321 in view of Heilper et al. (hereinafter as Heilper ) PGPUB 2019/0370086, and further in view of Uppalapati et al. (hereinafter as Uppalapati ) PGPUB 2023/0195590 . As per claim 9, Croxford teach the apparatus of claim 8. Croxford does not teach wherein to determine whether a particular layer of the plurality of layers is compute-intensive or memory-intensive includes determining whether the number of compute cycles required to execute the particular layer meets a criterion. Uppalapati teaches profiling neural network and determining execution performance for a layer of a neural network. Uppalapati is thus similar to Croxford because they analyze performance and workload of each layer on a neural network using a compiler. Uppalapati further teaches determine whether a particular layer of the plurality of layers is compute-intensive or memory-intensive includes determining whether the number of compute cycles required to execute the particular layer meets a criterion [0135: (identify a layer as compute bound in response to the ration of compute cycles per instruction (number of compute cycles) exceeding a threshold ratio (criterion)]. The combination of Croxford with Uppalapati yields determining a layer is compute bound based on whether or not the number of cycles exceeds a threshold. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Uppalapati’s teachings of determining a layer is compute bound based on whether the number of cycles it takes to perform an instruction exceeds a threshold in Croxford. It would have been obvious to one of ordinary skill in the art to use such a metric to determine if it is compute bound in Croxford because it is a simple and direct measurement of seeing how long it takes to perform an instruction since compute-intensive workloads take longer time to execute. As per claim 10 , Croxford and Uppalapati teach the apparatus of claim 9, wherein the criterion is based on at least one of a number of data cycles or a number of cached cycles [0135: (threshold ratio of compute cycles per instruction (i.e. a number of data cycles per instruction))]. Claim 13 is similar in scope to claim 10 as addressed above and is thus rejected under the same rationale. Claim 18 is similar in scope to claim 5 and 10 combined as addressed above and is thus rejected under the same rationale . 07-21-aia AIA Claim (s) 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Heilper et al. (hereinafter as Heilper ) PGPUB 2019/0370086, and further in view of Richmond et al. (hereinafter as Richmond ) PGPUB 2023/0004430 . As per claim 23, Heilper teach the non-transitory computer-readable medium of claim 21, wherein the profile includes a layer-by-layer analysis of the neural network [FIG. 2A, 0068, and 0078: (analysis of each layer is part of the workload information)]. Heilper does not explicitly teach the layer-by-layer analysis including one or more statistics for the one or more respective layers. Specifically, Heilper does not explicitly teach statistics for each layer. Richmond teaches a compiler generating a profile for a neural network for its layers and the compiler compiling neural network into executable code [0020]. Richmond is thus similar to Heilper. Richmond further teaches the layer-by-layer analysis including one or more statistics for the one or more respective layers [0023, 0029, 0031, and 0044: (a profile is generated that includes information of hardware efficiency factor (statistics) for a neural network model on a per layer basis)]. Richmond teaches providing a profile that provides statistical information of each layer. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Richmond’s teachings of providing a profile with statistical data on a layer by layer basis in Heilper. One of ordinary skill in the art would have been motivated to include statistical information on a layer by layer basis in the metadata/workload information in Heilper because it allows for quick determination of adjustments to make during execution of each layer without requiring additional analysis, thereby reducing computational load . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is reminded that in amending in response to a rejection of claims, the patentable novelty must be clearly shown in view of the state of the art disclosed by the references cited and the objections made. Applicant must also show how the amendments avoid such references and objections. See 37 CFR §1.111(c) . Hanebutte et al. (USPAT 12,443,399) teaches tracking the number of cycles for processing in a layer. Siraso et al. (PGPUB 2023/0153583) teaches determining the number of cycles between initiation and completion processing of data for a layer. Ahmadi et al. (PGPUB 2024/0143986) teaches neural network with layers and determining whether a layer is compute-bound and measuring the number of cycles, and determining a layer is computation bound (compute-intensive) if greater than the knee point. Lo et al. (hereinafter as 2020/0242474) teaches compiler transforming source code to executable code for a neural network [0058]. Jackobsson (PGPUB 2023/0065730) teaches reduction of clock frequency of hardware accelerator using information of each layer in neural network. Power et al. (PGPUB 2021/0319317) teaches compiler compiling a configuration of one more layers of a neural network. Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANNY CHAN whose telephone number is (571)270-5134. The examiner can normally be reached Monday - Friday 10-7 EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew J. Jung can be reached at 5712703779. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /DANNY CHAN/Primary Examiner, Art Unit 2175 Application/Control Number: 18/133,616 Page 2 Art Unit: 2175 Application/Control Number: 18/133,616 Page 3 Art Unit: 2175 Application/Control Number: 18/133,616 Page 4 Art Unit: 2175 Application/Control Number: 18/133,616 Page 5 Art Unit: 2175 Application/Control Number: 18/133,616 Page 6 Art Unit: 2175 Application/Control Number: 18/133,616 Page 7 Art Unit: 2175 Application/Control Number: 18/133,616 Page 8 Art Unit: 2175 Application/Control Number: 18/133,616 Page 9 Art Unit: 2175 Application/Control Number: 18/133,616 Page 10 Art Unit: 2175 Application/Control Number: 18/133,616 Page 11 Art Unit: 2175 Application/Control Number: 18/133,616 Page 12 Art Unit: 2175 Application/Control Number: 18/133,616 Page 14 Art Unit: 2175
Prosecution Insights