Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21-27, 29-36, 38-40 are rejected under 35 U.S.C. 103 as being unpatentable over Collins, US 2023/0123811 A1, in view of Shen, “NIMBLE: EFFICIENTLY COMPILING DYNAMIC NEURAL NETWORKS FOR MODEL INFERENCE”, March 2021.
Regarding Claim 21, Collins teaches:
An apparatus, comprising: interface circuitry; and processor circuitry coupled to the interface circuitry and configured to (Figs. 19 A-F; paragraph 356):
determine whether a tensor shape of an input tensor of an object in a deep neural network (DNN) is dynamic and exists in a shape buffer pool (paragraph 134: “system 200 includes one or more memories (e.g., memory 208 before kernel launch instruction and memory 212 after kernel launch instruction while device 204 is performing instructions) to store a software kernel that includes one or more dimensions (e.g., of one or more tensor shapes) of one or more sets of data (e.g., tensors used in a graph such as representation of program 104) determined using one or more dimensional constraints of one or more sets of data”. The memories representative of the shape buffer pool. And, paragraph 60: “tensors used by representation of a computer program 104 are referred to as one or more sets of data. In at least one embodiment, one or more dimensions of tensor shapes are referred to as one or more dimensions of one or more sets of data. In at least one embodiment, inferring tensor shapes is referred to as determining tensor shapes, calculating tensor shapes, determining one or more dimensions of one or more sets of data, and/or some other suitable manner of referring to inferring tensor shapes”. And, paragraph 63: “shapes of tensors with dynamic shapes are represented with shape expressions that include one or more variables”),
the shape buffer pool being configured to store compilation results obtained by the compilation procedure for a set of predetermined tensor shapes and associated objects (paragraph 134: “processor 210 of device 204 includes one or more circuits to perform one or more instructions in a software kernel that includes one or more dimensions (e.g., of one or more tensor shapes) of one or more sets of data (e.g., tensors used in a graph such as representation of program 104) determined using one or more dimensional constraints of one or more sets of data, where dimensions have been determined and included in software kernel by a compiler (e.g., DL compiler 102 of FIG. 1). In at least one embodiment, system 200 includes one or more memories (e.g., memory 208 before kernel launch instruction and memory 212 after kernel launch instruction while device 204 is performing instructions) to store a software kernel that includes one or more dimensions (e.g., of one or more tensor shapes) of one or more sets of data (e.g., tensors used in a graph such as representation of program 104) determined using one or more dimensional constraints of one or more sets of data”);
and invoke the compilation procedure to perform Just-in-time (JIT) compilation for the object so as to get the compilation result for the object when it is determined that the tensor shape of the input tensor of the object is dynamic and does not exist in the shape buffer pool (paragraph 140: “for graphs where one or more dimensions of one or more tensor shapes and one or more ranks of one or more tensors are not determined (e.g., represented as s @ [x, 8] where rank is not determined) during graph construction 302, a compiler performs just-in-time (JIT) code generation 310 at runtime to generate a kernel based, at least in part, on using dimensions of tensor input data that are used at kernel launch”).
Collins may not have explicitly taught the following, however, Shen shows:
the input tensor being received via the interface circuitry from a higher network level in a compilation procedure for the DNN (subsection 2.2, p. 3: “Nimble takes a model in the format of mainstream deep learning frameworks, converts it into a unified intermediate representation (IR), then optimizes and compiles the IR into an executable”. The intermediate representation representative of the higher level),
run the object by use of a compilation result for the object stored in the shape buffer pool when it is determined that the tensor shape of the input tensor of the object is dynamic and exists in the shape buffer pool (subsection 3.1, p. 4: “In order to support dynamic data shapes, Nimble introduces a special dimension called Any to represent statically unknown dimensions. For example, we can represent a tensor type as Tensor[(1, 10, Any), float32], where the size of the third dimension in this tensor is unknown while the other two dimensions have concrete values”; And “To limit the loss of precision introduced by using Any dimensions, we introduce sub-shaping to the type system. Much like sub-typing used in popular programming languages (Liskov & Wing, 1994; Amadio & Cardelli, 1993), our type system extension enables values with concrete dimensions to be valid sub-types of tensor types with dynamic dimensions”. The concrete dimensions representative of known/existing tensor shapes).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Shen with that of Collins for receiving the input tensor from a higher network level in a compilation procedure for the DNN; and running the object by use of a compilation result for the object stored in the shape buffer pool when it is determined that the tensor shape of the input tensor of the object is dynamic and exists in the shape buffer pool.
The ordinary artisan would have been motivated to modify Collins in the manner set forth above for the purposes of optimizing, compiling and executing dynamic neural networks on multiple platforms [Shen: Abstract].
Regarding Claim 22, Collins further teaches:
The apparatus of claim 21, wherein the processor circuitry is further configured to: update the shape buffer pool by adding the compilation result for the object obtained by the JIT compilation for the object (paragraph 140: “for graphs where one or more dimensions of one or more tensor shapes and one or more ranks of one or more tensors are not determined (e.g., represented as s @ [x, 8] where rank is not determined) during graph construction 302, a compiler performs just-in-time (JIT) code generation 310 at runtime to generate a kernel based, at least in part, on using dimensions of tensor input data that are used at kernel launch”; And, paragraph 89: “in order to infer information about shapes within a program (e.g., representation of program 104), DL compiler 102 first generates a set of shape constraints for program. In at least one embodiment, DL compiler 102 generates set of shape constraints using an augmented unification algorithm. In at least one embodiment, DL compiler 102 uses augmented unification algorithm to generate not only a type substitution for a pair of types, but also a set of type constraints. In at least one embodiment, whenever two tensor types involving shapes are unified, DL compiler 102 generates shape constraints and adds them to a set”).
Regarding Claim 23, Collins further teaches:
The apparatus of claim 21, wherein the processor circuitry is further configured to: update the shape buffer pool by applying a least recently used (LRU) algorithm to remove a compilation result for an unpopular tensor shape (paragraph 94: “In at least one embodiment, DL compiler 102 uses a set of basic rules to remove constraints that are tautologies, to perform replacement of free shape variables that have been determined”).
Regarding Claim 24, Shen further teaches:
The apparatus of claim 21, wherein the processor circuitry is further configured to run the object by use of a static tensor shape based compilation result when it is determined that the tensor shape of the input tensor of the object is static (subsection 3.5, p. 7: “We observe that a good configuration for one shape usually performs well on other shapes. Based on this observation, we devise the following mechanism to tune the kernel for symbolic shapes. 1. First replace the symbolic dimensions by a large enough value (e.g., 64) so that the search space can cover most possibilities, and run the tuning algorithm on the static shape for a sufficient number of iterations”).
Regarding Claim 25, Shen further teaches:
The apparatus of claim 21, wherein the compilation procedure comprises an Intermediate Representation (IR) lowering procedure based on a multi-level IR architecture for the compilation procedure (subsection 2.2, p. 3: “Nimble takes a model in the format of mainstream deep learning frameworks, converts it into a unified intermediate representation (IR), then optimizes and compiles the IR into an executable”).
Regarding Claim 26, Shen further teaches:
The apparatus of claim 25, wherein the IR lowering procedure comprises a shape inference pass for generating a buffer dialect from a static tensor shape based high level IR, and the buffer dialect is configured to define representations of one or more types of tensors with either static or dynamic tensor shapes, operations associated with the tensors, and attributes associated with the operations (subsection 3.1, p. 4: “Deep learning compilers use type systems to represent, check and infer the data types and shapes of tensors”).
Regarding Claim 27, Shen further teaches:
The apparatus of claim 26, wherein the IR lowering procedure further comprises a buffer management pass configured to: set a tag for a tensor and an object associated with the tensor to indicate the tensor is dynamic and static compilation of the tensor and the associated object is not to be performed, when it is determined the tensor needs dynamic buffer according to the representation of the tensor in the buffer dialect (subsection 3.3, p. 5: “we present a more complex example in the Appendix A, which illustrates how to handle memory allocation when operators have dynamic shaped inputs. The key insight is to internalize a notion of memory allocation into the IR, enabling static optimization of both static and dynamic allocations in the presence of control and dynamic shapes”).
Regarding Claim 29, Collins further teaches:
The apparatus of claim 21, wherein the input tensor has a static rank (paragraph 76: “taxonomy of tensor shapes includes specific shapes. In at least one embodiment, specific shapes are shapes of statically known constant rank, with dimensions that are all statically known constants”).
Regarding Claim 39, Collins further teaches:
A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by processor circuitry, cause the processor circuitry to perform the method of claim 30 (paragraph 142).
Regarding Claim 40, Collins further teaches:
A device, comprising means for performing the method of claim 30 (paragraph 133).
Claims 28, 37 are rejected under 35 U.S.C. 103 as being unpatentable over Collins, US 2023/0123811 A1, in view of Shen, “NIMBLE: EFFICIENTLY COMPILING DYNAMIC NEURAL NETWORKS FOR MODEL INFERENCE”, March 2021, and further in view of Liu, US 2023/0195599 A1.
Regarding Claim 28, with Collins and Shen teaching those limitations of the claim as previously pointed out, neither Collins nor Shen may have explicitly taught the following, however, Liu shows:
The apparatus of claim 26, wherein the representations of the tensors in the buffer dialect are based on a Static Single Assignment (SSA) form calculated from tensor values of the tensors (paragraph 3: “A static single assignment (SSA) form is an efficient data flow analysis technique and may be applied to almost all modern compilers currently. The SSA form is a kind of intermediate representation (IR). Under the SSA form, each variable is assigned only once, which makes a use-define chain of the variable very explicit and helps simplify algorithms of compilers”). (Emphasis added).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Liu with that of Collins and Shen for having representations of the tensors in the buffer dialect based on a Static Single Assignment (SSA) form calculated from tensor values of the tensors.
The ordinary artisan would have been motivated to modify Collins and Shen in the manner set forth above for the purposes of simplifying algorithms of the compiler [Liu: paragraph 3].
Claims 30-38 are similar to Claims 21-29 and are rejected under the same rationale as stated above for those claims.
Examiner's Note:
The Examiner cites particular pages, sections, columns, line numbers, and/or paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in its entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner and the additional related prior arts made of record that are considered pertinent to applicant's disclosure to further show the general state of the art. The Examiner's interpretations in parenthesis are provided with the cited references to assist the applicants to better understand how the examiner interprets the prior art to read on the claims. Such comments are entirely consistent with the intent and spirit of compact prosecution.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892 for the relevant prior art where for example Brady, US 2019/0391796 A1, teaches using sets of data with dynamically configurable dimensions.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVE MISIR whose telephone number is (571)272-5243. The examiner can normally be reached M-R 8-5 pm, F some hours.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at 5712703169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVE MISIR/Primary Examiner, Art Unit 2127
Prosecution Insights