Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 2, 5, 9, 12, 16, and 19 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that use the word “means” or “step” but are nonetheless not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph because the claim limitation(s) recite(s) sufficient structure, materials, or acts to entirely perform the recited function. Such claim limitation(s) is/are:
"means for receiving a machine learning (ML) model" in claim 15.
"means for decomposing the weights of a weight matrix into a second format" in claim 15.
"means for linearizing the decomposed weight matrix" in claim 15.
"means for quantizing the weights of the decomposed weight matrix" in claim 15.
“means for receiving a prompt” in claim 20.
“means for generating one or more tokens based on the prompt” in claim 20.
“means for processing the one or more tokens” in claim 20.
Because this/these claim limitation(s) is/are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are not being interpreted to cover only the corresponding structure, material, or acts described in the specification as performing the claimed function, and equivalents thereof.
If applicant intends to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to remove the structure, materials, or acts that performs the claimed function; or (2) present a sufficient showing that the claim limitation(s) does/do not recite sufficient structure, materials, or acts to perform the claimed function.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 4, 8, 11, 15, and 18 is/are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Sather et al (US 12579430 B1, hereinafter Sather).
Regarding Claim 1:
Sather teaches
An apparatus comprising: at least one memory (Sather [Col 40 lines 52-57]: "The CPU 1110 includes closely coupled memory 1140 for instruction (ICCM) and data (DCCM) which can be used for booting up the IC (e.g., boot loaders and/or firmware are loaded out of ROM (not shown) into the ICCM and/or DCCM), as well as typical components for accessing the unified memory");
at least one processor coupled to the at least one memory, the at least one processor configured to (Sather [Col 40 lines 52-57]: "The CPU 1110 includes closely coupled memory 1140 for instruction (ICCM) and data (DCCM) which can be used for booting up the IC (e.g., boot loaders and/or firmware are loaded out of ROM (not shown) into the ICCM and/or DCCM), as well as typical components for accessing the unified memory")
receive a machine learning (ML) model having multiple layers and including weights in a first format arranged in the at least one memory in a first layout (Sather [Col 20, lines 15-24]: "As shown, the process 800 begins by receiving (at 805) a definition of a multi-layer network (e.g., a neural network) for training with initialized floating-point weights. The network of some embodiments is made up of numerous computation nodes, organized in layers, that produce output values based on one or more input values. This network structure includes the type of each layer (e.g., convolutional, element-wise addition or multiplication, pooling, fully-connected, etc.), as well as the structure of the input and output of the layer");
decompose the weights of a weight matrix into a second format (Sather [Col 13 lines 39-59]: "As noted, the weight matrices for a convolution can be approximated using a low-rank decomposition: {EQN. 6}. Here, r is the rank of the decomposition, B is an N.sub.out×r matrix, A is an r×V.sub.p matrix, and their product, X=BA, is the low-rank approximation for the weight matrix W. The matrix A operates on the “Toeplitz” input matrix I, and thus has the form of a convolution (in that the same matrix is applied to each patch of the input matrix). A has the same spatial dimensions as the original weight matrix W (i.e., represents a k×k, stride-1 convolution), but has a smaller number of rows (i.e., rr values. Thus, B represents a 1×1 convolution that decompresses the compressed representation produced by A");
linearize the decomposed weight matrix (Sather [Col 4 line 64- Col 5 line 1]: "When a layer is decomposed, the activation functions associated with the original layer are applied to the outputs of the linear functions of the second of the successive layers, in order to mimic the original activation functions");
such that each vector block of the weight matrix is contiguous in the at least one memory (Sather [Col 46 lines 27-30]: "The activation write bus 1320 also includes a right shift circuit for each core that is used to align the output values for the core, in order for the values to be stored in contiguous blocks within the core RAM"; (EN): while the details about memory limitations are not specified with regard to each step of the methods presented in Sather, it can be inferred that because the values are stored in contiguous blocks at the end of the methods presented, the values are also stored in contiguous blocks at previous steps in the process(es))
and quantize the weights of the decomposed weight matrix of the ML model based on a scaling factor (Sather [Col 2 lines 14-18]: "In some embodiments, at the end of the training of the network, each of the weights is restricted to a set of allowed quantized weight values (e.g., the set {0, 1, −1} or {0, α.sub.k, −α.sub.k}, where α.sub.k is a scale value that varies for different layers or filters).")
Regarding Claim 4:
Sather teaches
The apparatus of claim 1, in which the first format comprises a four-bit integer and weight values are packed in an eight-bit integer format and stored in the at least one memory (Sather [Col 19 lines 15-23]: "However, for input to the second replacement layer, these intermediate values 610 are quantized to produce quantized intermediate values 615 (e.g., a set of 4-bit or 8-bit values). These quantized values 615 may be generated from the values 610 by any sort of quantization process (e.g., truncating, taking the most significant bits, etc.). In some embodiments, the application of the inserted activation function (not shown in this figure) also quantizes the intermediate values").
Regarding Claim 8:
Due to claim language similar to that of Claim 1, Claim 8 is rejected for the same reasons as presented above in the rejection of Claim 1.
Regarding Claim 11:
Due to claim language similar to that of Claim 4, Claim 1 is rejected for the same reasons as presented above in the rejection of Claim 4.
Regarding Claim 15:
Due to claim language similar to that of Claims 1 and 8, Claim 15 is rejected for the same reasons as presented above in the rejection of Claims 1 and 8.
Regarding Claim 18:
Due to claim language similar to that of Claims 4 and 1, Claim 18 is rejected for the same reasons as presented above in the rejection of Claims 4 and 11.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 10, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sather as applied to claims 1, 8, and 15 above, and further in view of Kernert et al (US 20170168990 A1, hereinafter Kernert).
Regarding Claim 3:
Sather does not distinctly disclose
The apparatus of claim 1, in which the first layout comprises a z-ordering layout.
However, Kernert teaches
The apparatus of claim 1, in which the first layout comprises a z-ordering layout (Kernert [0032]: "FIG. 4A represents a raw input matrix A, FIG. 4B represents Z-curve ordering of matrix A and logical atomic blocks").
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the systems and methods for improving structural sparsity of a machine-trained network of Sather with the systems for adaptive tile matrix representations and/or a multiplication operations of Kernert in order to provide a method for utilizing a two-dimensional density map of a matrix by counting matrix elements that are contained in logical two-dimensional block cells in order to create a Z-curve/Z-ordered matrix (Kernert [0038]: “At S620, Z-curve ordering of matrix A is performed to create a two-dimensional density map of matrix A by counting matrix elements that are contained in logical two-dimensional block cells of a given size. Note that a maximum adaptive matrix tile size may be selected based at least in part on a size of a last level cache and a width of accumulator arrays used in a sparse matrix multiplication kernel. Moreover, the Z-curve ordering of matrix A may comprise a locality-aware element re-ordering”).
Regarding Claim 10:
Due to claim language similar to that of Claim 3, Claim 10 is rejected for the same reasons as presented above in the rejection of Claim 3.
Regarding Claim 17:
Due to claim language similar to that of Claims 3 and 10, Claim 17 is rejected for the same reasons as presented above in the rejection of Claims 3 and 10.
Claim Rejections - 35 USC § 103
Claim(s) 6, 13, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sather as applied to claims 1, 8, and 15 above, and further in view of Liu et al (Liu, Y., He, H., Han, T., Zhang, X., Liu, M., Tian, J., … Ge, B. (2024). Understanding LLMs: A Comprehensive Overview from Training to Inference. arXiv [Cs.CL]. Retrieved from http://arxiv.org/abs/2401.02038, hereinafter Liu).
Regarding Claim 6:
Sather does not distinctly disclose
The apparatus of claim 1, in which the ML model comprises a large language model (LLM) and the at least one processor is further configured to: receive, by the LLM, a prompt; generate, by the LLM, one or more tokens based on the prompt;
process, by the LLM, the one or more tokens to generate an inference based on the quantized weights.
However, Liu teaches
The apparatus of claim 1, in which the ML model comprises a large language model (LLM) and the at least one processor is further configured to: receive, by the LLM, a prompt (Liu [Page 4, section 2.2, par. 1]: "By supplying appropriate prompts, researchers and practitioners can customize the model’s behavior, making it more suitable for specific domains or task requirements. In short, prompt learning is a machine learning approach that, builds upon pre-trained language models, and guides the model to perform various tasks through the design of prompt statements"); generate, by the LLM, one or more tokens based on the prompt (Liu [Page 3, Section 2.1.4, par. 1]: " When each token in a sentence passes through the Transformer’s Encoder/Decoder stack, the model itself lacks any sense of position/order for each token (permutation invariance). Therefore, a method is still needed to incorporate the sequential information of tokens into the model. To enable the model to perceive the input sequence, positional information about the location of each token in the sentence can be added, and this technique is known as positional embedding");
process, by the LLM, the one or more tokens to generate an inference based on the quantized weights (Liu [Page 10, section 3.3, par. 1]: " Large Language Models (LLMs) typically learn rich language representations through a pre-training process. During pre-training, these models leverage extensive corpora, such as text data from the internet, and undergo training through self-supervised learning methods. Language modeling is one common form of self-supervised learning task in which the model is tasked with predicting the next word in a given context.").
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the systems and methods for improving structural sparsity of a machine-trained network of Sather with the systems for large language models of Liu in order to provide a method for utilizing a LLM to generate tokens based on a given prompt and to generate an inference/prediction based on the tokens (Liu [Abstract]: “This paper reviews the evolution of large language model training techniques and inference deployment technologies aligned with this emerging trend. The discussion on training includes various aspects, including data preprocessing, training architecture, pre-training tasks, parallel training, and relevant content related to model fine tuning. On the inference side, the paper covers topics such as model compression, parallel computation, memory scheduling, and structural optimization.”).
Regarding Claim 13:
Due to claim language similar to that of Claim 6, Claim 13 is rejected for the same reasons as presented above in the rejection of Claim 6.
Regarding Claim 20:
Due to claim language similar to that of Claims 6 and 13, Claim 20 is rejected for the same reasons as presented above in the rejection of Claims 6 and 13.
Claim Rejections - 35 USC § 103
Claim(s) 7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sather as applied to claims 1, 8, and 15 above, and further in view of Li et al (M. Li et al., "The Deep Learning Compiler: A Comprehensive Survey," in IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 3, pp. 708-727, 1 March 2021, doi: 10.1109/TPDS.2020.3030548., hereinafter Li).
Regarding Claim 7:
Sather does not distinctly disclose
The apparatus of claim 1, the at least one processor being further configured to represent the ML model computation using polyhedral loops.
However, Li teaches
The apparatus of claim 1, the at least one processor being further configured to represent the ML model computation using polyhedral loops (Li [Page 713, Section 3.2.1, par. 5]: "TC has its unique design in low-level IR, which combines the Halide and polyhedral model. It uses Halide-based IR to represent the computation and adopts the polyhedral-based IR to represent the loop structures.").
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the systems and methods for improving structural sparsity of a machine-trained network of Sather with the comprehensive survey of deep learning compilers of Li in order to provide a method for utilizing a polyhedral loop structure within a machine learning model (Li [Page 713, section 3.2.1, par. 4]: “The polyhedral model is an important technique adopted in DL compilers. It uses linear programming, affine transformations, and other mathematical methods to optimize loop-based codes with static control flow of bounds and branches. In contrast to Halide, the boundaries of memory reference and loop nests can be polyhedrons with any shapes in the polyhedral model. Such flexibility makes polyhedral models widely used in generic compilers.”).
Regarding Claim 14:
Due to claim language similar to that of Claim 7, Claim 14 is rejected for the same reasons as presented above in the rejection of Claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20250200277 A1 – A method and an apparatus for storing data points
US 12061988 B1 – a method for training parameters of a network
US 20240028665 A1 – An apparatus computing a matrix vector product of a given matrix
US 20230076290 A1 – A method for quantizing a pre-trained neural network
WO 2021158830 A1 – an apparatus for quantizing pre-trained neural network
Any inquiry concerning this communication or earlier communications from the examiner should be directed to COREY M SACKALOSKY whose telephone number is (703)756-1590. The examiner can normally be reached M-F 7:30am-3:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COREY M SACKALOSKY/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights