DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
Claims 4 and 7 recites “NPU scheduler configured to control…”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
For example, Paragraphs [0977]-[0983] discloses the methods in which the NPU scheduler is performing the controlling. Additionally, the NPU scheduler is disclosed as having an internal memory [0979] and residing in the NPU [0973]. For the purposes of examination, the NPU scheduler is assumed to be a general or specialized processing unit.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Objections
Claim 1 is objected to because of the following informalities: the claim recites the abbreviations “DRAM”, “NPU” and “ANN” which must be spelled out at least once within the claims. Appropriate correction is required.
Claims 2-5 depend on claim 1 and inherits the same objections above.
Claim 4 is objected to because of the following informalities: the claim recites the abbreviation “AXI” which must be spelled out at least once within the claims. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3 and 5-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yoon et al. (US 2022/0083847 A1) hereinafter Yoon et al. in view of Heaton et al. (US 11561833 B1).
Regarding claim 1, Yoon et al. teaches an electronic device comprising:
a DRAM in which at least a part of data of an ANN model is stored (the plurality of devices may include at least one of a volatile memory and non-volatile memory… The volatile memory may be dynamic RAM (DRAM) Paragraph [0031], wherein the plurality of memory devices may store target model data Paragraph [0033]);
a NPU configured to process inference of the ANN model and provide an inference result of the ANN model according to input data (the neural network processor may output inference data about the input data based on the target model data and instruction data, and may output result data corresponding to each layer Paragraphs [0046]-[0047]);
the NPU configured to output a data access request… including a read operation or a write operation with respect to the ANN model (the neural network processor may receive a command and request the target model data and instruction data from the memory device Paragraphs [0044]-[0045]); and
a memory controller, which is electrically connected to the NPU and the DRAM (see Fig. 1 which depicts controller device 11 connected to NPU and memory devices 13a-b), configured to receive the data access request from the NPU and to control the DRAM based on the data access request (memory controller device configured to: perform a read operation on a target page based on a read command, and output the target page Paragraph [0008]).
Yoon et al. does not appear to explicitly teach, however, Heaton et al. teaches the NPU configured to output a data access request according to a sequence of operations, including a read operation or a write operation with respect to the ANN model, in which the sequence of operations is determined at compile time for the ANN model (the compiler engine generates a set of instructions based on a neural network model to be used to process input data, and may correspond to a sequence of operations which include operations to be performed such as retrieval of input data and storage of output data Column 2, Lines 9-33).
The disclosures of Yoon et al. and Heaton et al., hereinafter YH, are analogous art to the claimed invention because they are in the same field of endeavor of performing neural network operations.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of YH before them, to modify the teachings of Yoon et al. to include the teachings of Heaton since both YH teach performing processing by a neural network processor. Therefore it is applying a known technique (outputting a sequence of operations at compile time for the model Column 2, Lines 9-33 of Heaton et al.) to a known device (performing neural network operations of Yoon et al.) ready for improvement to yield predictable results (data requests are sent out according to a sequence of operations determined at compile time of Heaton et al.), KSR, MPEP 2143.
Regarding claim 2, YH teaches all of the features with respect to claim 1 as outlined above.
Yoon et al. further teaches data of the ANN model includes weight parameters (a weight parameter of a neural network model Paragraph [0028]) and Heaton et al. teaches the data of the ANN model includes activation values (activation functions is applied on the output and the result of the post processing is stored back Column 14, Lines 40-44). Yoon et al. further teaches the DRAM is dedicated to the data of the ANN model (the plurality of devices may include at least one of a volatile memory and non-volatile memory… The volatile memory may be dynamic RAM (DRAM) Paragraph [0031]).
Regarding claim 3, YH teaches all of the features with respect to claim 1 as outlined above.
Heaton et al. further teaches the NPU comprising : a processing element (PE) array (neural network processor includes an array of processing elements Column 11, Lines 15-45); an NPU memory system configured to store at least a portion of data of an artificial neural network (ANN) model processed in the PE array (see Fig. 4B which depicts a memory device storing input data and output data of a model); and an NPU scheduler configured to control the PE array and the NPU memory system based on artificial neural network (ANN) model structure data (a dependency graph representing a neural network model associated with the neural network processing layers represented, for example, operations for processing layer 211 are to be performed after processing layer 209 Column 46-65), wherein the ANN model includes a plurality of layers, each layer of the plurality of layers including corresponding weight parameters and corresponding feature map parameters to configure the ANN model structure data (an input feature map associated with weights in filter Column 7, Lines 61-67 and weight matrix Column 10, Lines 24-30 is used and stored as part of the dependency graph), wherein the ANN model structure data includes sequence information configured to schedule a processing sequence from an input layer to an output layer of the ANN model, and wherein the PE array is configured to process the ANN model based on the ANN model structure data (a sequence of instructions can be provided to the array of processing elements to perform the computations for each neural layer sequentially, and based on the data dependency among the network layers (for example, 207 is performed first, followed by layer 209 and 211 Column 11, Lines 15-45).
Regarding claim 5, YH teaches all of the features with respect to claim 1 as outlined above.
Yoon et al. teaches further comprising: at least one of a CPU and a GPU (Fig. 1 depicts CPU 20).
Regarding claim 6, Yoon et al. teaches an electronic device comprising:
a NPU configured to acquire at least a part of data of an ANN model from a DRAM and process inference of the ANN model and provide an inference result of the ANN model according to an input data (the plurality of devices may include at least one of a volatile memory and non-volatile memory… The volatile memory may be dynamic RAM (DRAM) Paragraph [0031], wherein the plurality of memory devices may store target model data Paragraph [0033])the neural network processor may output inference data about the input data based on the target model data and instruction data, and may output result data corresponding to each layer Paragraphs [0046]-[0047]);
the NPU configured to output a data access request… including a read operation or a write operation with respect to the ANN model (the neural network processor may receive a command and request the target model data and instruction data from the memory device Paragraphs [0044]-[0045]); and
the NPU comprises a memory controller configured to acquire at least a part of data of the ANN model from the DRAM based on the data access request (see Fig. 1 which depicts controller device 11 connected to NPU and memory devices 13a-b, wherein memory controller device configured to: perform a read operation on a target page based on a read command, and output the target page Paragraph [0008]).
Yoon et al. does not appear to explicitly teach, however, Heaton et al. teaches the NPU configured to output a data access request according to a sequence of operations, including a read operation or a write operation with respect to the ANN model, in which the sequence of operations is determined at compilation for the ANN model (the compiler engine generates a set of instructions based on a neural network model to be used to process input data, and may correspond to a sequence of operations which include operations to be performed such as retrieval of input data and storage of output data Column 2, Lines 9-33).
The disclosures of Yoon et al. and Heaton et al., hereinafter YH, are analogous art to the claimed invention because they are in the same field of endeavor of performing neural network operations.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of YH before them, to modify the teachings of Yoon et al. to include the teachings of Heaton since both YH teach performing processing by a neural network processor. Therefore it is applying a known technique (outputting a sequence of operations at compilation for the model Column 2, Lines 9-33 of Heaton et al.) to a known device (performing neural network operations of Yoon et al.) ready for improvement to yield predictable results (data requests are sent out according to a sequence of operations determined at compilation of Heaton et al.), KSR, MPEP 2143.
Regarding claim 7, YH teaches all of the features with respect to claim 1 as outlined above.
Heaton et al. further teaches the NPU comprising : a processing element (PE) array (neural network processor includes an array of processing elements Column 11, Lines 15-45); an NPU memory system configured to store at least a portion of data of an artificial neural network (ANN) model processed in the PE array (see Fig. 4B which depicts a memory device storing input data and output data of a model); and an NPU scheduler configured to control the PE array and the NPU memory system based on artificial neural network (ANN) model structure data (a dependency graph representing a neural network model associated with the neural network processing layers represented, for example, operations for processing layer 211 are to be performed after processing layer 209 Column 46-65), wherein the ANN model includes a plurality of layers, each layer of the plurality of layers including corresponding weight parameters and corresponding feature map parameters to configure the ANN model structure data (an input feature map associated with weights in filter Column 7, Lines 61-67 and weight matrix Column 10, Lines 24-30 is used and stored as part of the dependency graph), wherein the ANN model structure data includes sequence information configured to schedule a processing sequence from an input layer to an output layer of the ANN model, and wherein the PE array is configured to process the ANN model based on the ANN model structure data (a sequence of instructions can be provided to the array of processing elements to perform the computations for each neural layer sequentially, and based on the data dependency among the network layers (for example, 207 is performed first, followed by layer 209 and 211 Column 11, Lines 15-45).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over YH in further view of He et al. (US 2020/0394507 A1) hereinafter He et al.
Regarding claim 4, YH teaches all of the features with respect to claim 1 as outlined above.
YH does not appear to explicitly teach, however, He et al. teaches the electronic device of the NPU and the DRAM are connected via an AXI interface (the electronic device having a memory 103 and neural network processor 200 are communicatively coupled to each other via bus 101 that can be an AXI (Advanced eXtensible Interface) bus protocol Paragraph [0064]).
The disclosures of YH and He et al., hereinafter YHH, are analogous art to the claimed invention because they are in the same field of endeavor of performing neural network operations.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of YHH before them, to modify the teachings of YH to include the teachings of He et al. since both YHH teach performing processing by a neural network processor. Therefore it is applying a known technique (connecting the components via an AXI interface of He et al.) to a known device (performing neural network operations of Yoon et al.) ready for improvement to yield predictable results (data is transmitted between the NPU and DRAM through the AXI interface), KSR, MPEP 2143.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ngo (US 2021/0406655 A1) teaches an inference processing apparatus which batch processes operations using a learned neural network model and infers a feature of the pieces of input data.
Kim (US 2020/0234147 A1) teaches obtaining a processing order of branches of a neural network model.
Vee (US 2021/0019184 A1) teaches a generating a schedule for processing a computation graph by executing operations represented by the computation graph.
Heaton et al. (US 11714992 B1) teaches a computational graph representing a sequence of operations to be performed for a neural network model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANE W BENNER whose telephone number is (571)270-0067. The examiner can normally be reached Mon - Thurs (8 AM - 5 PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, REGINALD BRAGDON can be reached at (571) 272-4204. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JANE W. BENNER
Primary Examiner
Art Unit 2131
/JANE W BENNER/Primary Examiner, Art Unit 2139
