DETAILED ACTION
This action is in response to the application filed 07/10/2026. Claims 1-6, and 8-21 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8-13, 15, and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites a method and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 1 recites
the selected set of weights and the mask having been produced by a sparsification process that operates on an original set of weights, the sparsification process discriminating between the selected set of weights and a non-selected set of weights, (This limitation is a mental process as it encompasses a human mentally discriminating between the selected set of weights and a non-selected set of weights and is thus a judgement.)
transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result (This limitation is a mental process as it encompasses a human mentally transforming the input embedding and is thus an evaluation.)
wherein the transforming uses a particular mask value of the mask to evaluate plural applications of a particular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to a zero result (This limitation is a mental process as it encompasses a human mentally evaluating plural ways of applying a weight to the input embedding and is thus an evaluation.)
Therefore, claim 1 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 1 further recites additional elements of
for executing a machine-trained model (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
receiving a selected set of weights and a mask (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
the mask describing positions of the selected set of weights and the non-selected set of weights among a combined set of weights (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
storing the selected set of weights and the mask in memory; (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
receiving an input embedding; (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
in a processor (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 1 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
for executing a machine-trained model uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
receiving a selected set of weights and a mask is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
the mask describing positions of the selected set of weights and the non-selected set of weights among a combined set of weights specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
storing the selected set of weights and the mask in memory; is the well understood, routine, and conventional activity of “storing and retrieving information in memory” (see MPEP 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
receiving an input embedding; is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
in a processor uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 1 is subject-matter ineligible.
Regarding Claim 2:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 2 recites
wherein the non-selected set of weights are weights that represent zero values. (This limitation is a mental process as it further defines the mental process of discriminating between the selected set of weights and a non-selected set of weights from claim 1.)
Therefore, claim 2 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 2 does not further recite any additional elements. Therefore, claim 2 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 2 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 2 is subject-matter ineligible.
Regarding Claim 3:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 3 recites
wherein the sparsification process identifies the combined set of weights by setting a prescribed number of original weights in the original set of weights to the non-selected set of weights, based on a prescribed pattern (This limitation is a mental process as it encompasses a human mentally identifies the combined set of weights by setting a prescribed number and is thus an observation.)
Therefore, claim 3 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 3 does not further recite any additional elements. Therefore, claim 3 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 3 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 3 is subject-matter ineligible.
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 4 recites
wherein the prescribed pattern specifies, for a particular group of weights, a number of selected weights to be included in the particular group, (This limitation is a mental process as it further defines the prescribed pattern from claim 3 that is used to identify the combined set of weights.)
wherein each mask value in the mask specifies a location of a particular selected weight in the particular group of weights (This limitation is a mental process as it encompasses a human mentally specifying a location using a value and is thus an evaluation.)
Therefore, claim 4 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 4 does not further recite any additional elements. Therefore, claim 4 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 4 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 4 is subject-matter ineligible.
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 5 recites
wherein the particular mask value in the mask identifies whether a particular pair of weights in the combined set of weights includes a selected weight as a first member or a second member of the particular pair (This limitation is a mental process as it encompasses a human mentally identifying whether a pair of weights includes a selected weight using a value and is thus an observation.)
Therefore, claim 5 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 5 does not further recite any additional elements. Therefore, claim 6 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 5 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 5 is subject-matter ineligible.
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 6 recites the same abstract ideas as claim 1. Therefore, claim 6 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 6 further recites additional elements of
wherein the mask has plural mask values, each mask value being represented by a single bit in the mask (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 6 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 6 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the mask has plural mask values, each mask value being represented by a single bit in the mask specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 6 is subject-matter ineligible.
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 8 recites
multiplying the particular mask value by the particular weight of the selected set of weights and a particular element of the input embedding, to produce a first intermediate value; (This limitation is a mental/mathematical process as it encompasses a human mentally multiplying a value by a weight..)
multiplying a binary opposite of the particular mask value by the particular weight and another particular element of the input embedding, to produce a second intermediate value; (This limitation is a mental/mathematical process as it encompasses a human mentally multiplying a binary opposite by a weight.)
adding the first intermediate value to the second intermediate value (This limitation is a mental/mathematical process as it encompasses a human mentally adding intermediate values together.)
Therefore, claim 8 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 8 does not further recite any additional elements. Therefore, claim 8 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 8 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 8 is subject-matter ineligible.
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 9 recites the same abstract ideas as claim 1. Therefore, claim 9 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 9 further recites additional elements of
wherein the mask has mask values that are separate from the selected set of weights. (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 9 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 9 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the mask has mask values that are separate from the selected set of weights. specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 9 is subject-matter ineligible.
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 10 recites the same abstract ideas as claim 1. Therefore, claim 10 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 10 further recites additional elements of
wherein the mask has mask values that are incorporated into weights in the selected set of weights. (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 10 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 10 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the mask has mask values that are incorporated into weights in the selected set of weights. specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 10 is subject-matter ineligible.
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 11 recites
wherein the method incorporates the particular mask value into the particular weight of the selected set of weights by adding an offset to the particular weight that reflects the particular mask value. (This limitation is a mental process as it encompasses a human mentally incorporating a mask value into a weight by adding an offset and is thus an evaluation.)
Therefore, claim 11 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 11 does not further recite any additional elements. Therefore, claim 11 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 11 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 11 is subject-matter ineligible.
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 12 recites
wherein the transforming computationally extracts the particular mask value from the particular weight. (This limitation is a mental process as it encompasses a human mentally computationally extracting the mask value and is thus an evaluation.)
Therefore, claim 12 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 12 does not further recite any additional elements. Therefore, claim 12 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 12 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 12 is subject-matter ineligible.
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 13 recites the same abstract ideas as claim 1. Therefore, claim 13 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 13 further recites additional elements of
wherein the machine-trained model has plural layers, and wherein the method is performed for transformations executed by each layer of the plural layers. (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 13 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the machine-trained model has plural layers, and wherein the method is performed for transformations executed by each layer of the plural layers. uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 13 is subject-matter ineligible.
Regarding Claim 15:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 15 recites the same abstract ideas as claim 1. Therefore, claim 15 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 15 further recites additional elements of
wherein the processor is a graphics processing unit or a neural processing unit. (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 15 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 15 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the processor is a graphics processing unit or a neural processing unit. uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 15 is subject-matter ineligible.
Regarding Claim 18:
Subject Matter Eligibility Analysis Step 1:
Claim 18 recites a computing system and is thus an apparatus, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 18 recites
determining whether a target item is supported by a source item (This limitation is a mental process as it encompasses a human mentally determining whether a target item is supported by a source item and is thus a judgement.)
the selected set of weights and the mask having been produced by a sparsification process that operates on an original set of weights, the sparsification process discriminating between the selected set of weights and a non-selected set of weights, (This limitation is a mental process as it encompasses a human mentally discriminating between the selected set of weights and a non-selected set of weights and is thus a judgement.)
transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result (This limitation is a mental process as it encompasses a human mentally transforming the input embedding and is thus an evaluation.)
the transforming uses a particular mask value of the mask to evaluate plural applications of a particular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to zero (This limitation is a mental process as it encompasses a human mentally evaluating plural ways of applying a weight to the input embedding and is thus an evaluation.)
Therefore, claim 18 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 18 further recites additional elements of
A computing system for determining whether a target item is supported by a source item, comprising: a memory; a processing system for executing computer-readable instructions, to perform operations including: (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
receiving a selected set of weights and a mask (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
the mask describing positions of the selected set of weights and the non-selected set of weights among a combined set of weights (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
storing the selected set of weights and the mask in memory; (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
receiving an input embedding; (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
Therefore, claim 18 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 18 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A computing system for determining whether a target item is supported by a source item, comprising: a memory; a processing system for executing computer-readable instructions, to perform operations including uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
receiving a selected set of weights and a mask is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
the mask describing positions of the selected set of weights and the non-selected set of weights among a combined set of weights specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
storing the selected set of weights and the mask in memory; is the well understood, routine, and conventional activity of “storing and retrieving information in memory” (see MPEP 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
receiving an input embedding; is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
Therefore, claim 18 is subject-matter ineligible.
Regarding Claim 19:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 19 recites
wherein the sparsification process identifies the combined set of weights by setting a prescribed number of weights in the original set of weights to the non-selected set of weights, based on a prescribed pattern (This limitation is a mental process as it encompasses a human mentally identifies the combined set of weights by setting a prescribed number and is thus an observation.)
wherein the prescribed pattern specifies, for a particular group of weights, a number of selected weights to be included in the particular group, (This limitation is a mental process as it further defines the prescribed pattern from claim 3 that is used to identify the combined set of weights.)
wherein each mask value in the mask specifies a location of a particular selected weight in the particular group of weights (This limitation is a mental process as it encompasses a human mentally specifying a location using a value and is thus an evaluation.)
Therefore, claim 19 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 19 does not further recite any additional elements. Therefore, claim 19 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 19 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 19 is subject-matter ineligible.
Regarding Claim 20:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 20 recites
wherein the prescribed number is 50 percent (This limitation is a mental process as it further defines the mental process of setting a prescribed number of claim 19.)
Therefore, claim 20 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 20 does not further recite any additional elements. Therefore, claim 20 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 20 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 20 is subject-matter ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-6 and 8-21 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Frantar et al. (“SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot”) (hereafter referred to as Frantar).
Regarding claim 1, Frantar teaches
A method for executing a machine-trained model, comprising: receiving a selected set of weights and a mask (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask.),
the selected set of weights and the mask having been produced by a sparsification process that operates on an original set of weights, the sparsification process discriminating between the selected set of weights and a non-selected set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask. Examiner further notes that Algorithm 1 is the sparsification process and the binary pruning mask discriminates between the selected set of weights equal to 1 and the non-selected weights equal to 0.),
and the mask describing positions of the selected set of weights and the non-selected set of weights among a combined set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask. Examiner further notes that Algorithm 1 is the sparsification process and the binary pruning mask discriminates between the selected set of weights equal to 1 and the non-selected weights equal to 0. Examiner further notes that the i and j of the for loops are the positions.);
storing the selected set of weights and the mask in memory (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” (Frantar, page 6, 4. Experiments Setup.). Examiner notes that since all experiments are done on a GPU with memory, the weights and mask of Algorithm 1 is stored in memory.);
receiving an input embedding (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that H is the input embedding. Examiner further notes that according to paragraph 0039 of the specification, “the input embedding represents an input submission of any kind”.);
and in a processor, transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” (Frantar, page 6, 4. Experiments Setup.). Examiner notes that Algorithm 1 transforms H, the input embedding by performing computations on the weights, W, and the mask, M. Examiner further notes that W in the last line the last line of Algorithm 1 is the output result.)
wherein the transforming uses a particular mask value of the mask to evaluate plural applications of a particular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to a zero result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
where “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that M is the mask, the plural applications is pruning or not pruning the weights via the matrix multiplication occurring in the first update line of Algorithm 1 where E includes the mask and weight and is being multiplied by the input embedding H. Examiner further notes that “all but one of the plural applications resolving to a zero result” maps to the zeros that are a result of pruning the weights.).
Regarding claim 2, Frantar teaches
The method of claim 1, wherein the non-selected set of weights are weights that represent zero values(Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner further notes that Algorithm 1 or Sparse GPT is the sparsification process and the binary pruning mask discriminates between the selected set of weights equal to 1 and the non-selected weights equal to 0.).
Regarding claim 3, Frantar teaches
The method of claim 1, wherein the sparsification process identifies the combined set of weights by setting a prescribed number of original weights in the original set of weights to the non-selected set of weights, based on a prescribed pattern (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. We can execute SparseGPT on the largest available open-source models, OPT-175B and BLOOM-176B, in under 4.5 hours, and can reach 60% unstructured sparsity with negligible increase in perplexity: remarkably, more than 100 billion weights from these models can be ignored at inference time. SparseGPT generalizes to semi-structured (2:4 and 4:8) patterns, and is compatible with weight quantization approaches” (Frantar, page 1, Abstract). Examiner notes that the sparsification process is SparseGPT. Examiner further notes that p is the prescribed number in the original weights being set to the non-selected weights. Examiner notes that the prescribed pattern is the semi-structured patterns of 2:4 and 4:8).
Regarding claim 4, Frantar teaches,
The method of 3 wherein the prescribed pattern specifies, for a particular group of weights, a number of selected weights to be included in the particular group (Frantar, page 1, Abstract, “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. We can execute SparseGPT on the largest available open-source models, OPT-175B and BLOOM-176B, in under 4.5 hours, and can reach 60% unstructured sparsity with negligible increase in perplexity: remarkably, more than 100 billion weights from these models can be ignored at inference time. SparseGPT generalizes to semi-structured (2:4 and 4:8) patterns, and is compatible with weight quantization approaches” where “SparseGPT is also easily adapted to semi-structured patterns such as the popular n:m sparsity format (Zhou et al., 2021; Hubara et al., 2021a) which delivers speedups in its 2:4 implementation on Ampere NVIDIA GPUs. Specifically, every consecutive m weights should contain exactly n zeros” (Frantar, page 5, 3.3 Extension to Semi-Structured Sparsity). Examiner notes that the prescribed pattern is the semi-structured pattern of 2:4 and 4:8 and the selected weights are the weights that aren’t 0.),
and wherein each mask value in the mask specifies a location of a particular selected weight in the particular group of weights (Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
Examiner notes that the mask M is the mask and each cell in mask M is a mask value which specifies a location of a selected weight in light gray in W.).
Regarding claim 5, Frantar teaches
The method of claim 1, wherein the particular mask value in the mask identifies whether a particular pair of weights in the combined set of weights includes a selected weight as a first member or a second member of the particular pair (Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
Examiner notes that the mask M is the mask and each cell in mask M is a mask value. Examiner further notes that each cell in W is a weight and that a particular mask value identifies the first member or first cell of the first column of a particular pair of weights, which includes the first and second cell of the first column, as a selected weight.).
Regarding claim 6, Frantar teaches
The method of claim 1, wherein the mask has plural mask values, each mask value being represented by a single bit in the mask(Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
“throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1).Examiner notes that the mask M is the mask and each cell in mask M is a mask value which is either a 0 at the white cells or a 1 on the gray cells. ).
Regarding claim 8, Frantar teaches
The method of claim 1, wherein the transforming comprises: multiplying the particular mask value by the particular weight of the selected set of weights and a particular element of the input embedding, to produce a first intermediate value (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and Frantar, page 4, Figure 4 (see below),
PNG
media_image2.png
276
762
media_image2.png
Greyscale
Examiner notes that the freeze weights line of Algorithm 1 shows this limitation since E has both the weight and the input embedding in it and is multiplied by M, the mask. Examiner further notes that the first intermediate value is -M:,j • E:,j-i);
multiplying a binary opposite of the particular mask value by the particular weight and another particular element of the input embedding, to produce a second intermediate value (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that the freeze weights line of Algorithm 1 shows this limitation since E has both the weight and the input embedding in it and is multiplied by 1, which is a binary opposite of the mask. Examiner further notes that the first intermediate value is E:,j-i);
and adding the first intermediate value to the second intermediate value (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that the freeze weights line of Algorithm 1 shows this limitation E:,j-i and -M:,j • E:,j-i are added together and assigned to E:,j-i.).
Regarding claim 9, Frantar teaches
The method of claim 1, wherein the mask has mask values that are separate from the selected set of weights (Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
“throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1).Examiner notes that the mask M is the mask and each cell in mask M is a mask value which is either a 0 at the white cells or a 1 on the gray cells. Examiner further notes that the mask values are separate from the weights.).
Regarding claim 10, Frantar teaches
The method of claim 1, wherein the mask has mask values that are incorporated into weights in the selected set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that the second update line shows this limitation since W includes E which was calculated using M, the mask values).
Regarding claim 11, Frantar teaches
The method of claim 10, wherein the method incorporates the particular mask value into the particular weight of the selected set of weights by adding an offset to the particular weight that reflects the particular mask value (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that the second update line shows this limitation since W includes E which was calculated using M, the mask values. Examiner notes that the second update line has -E•
H
j
,
j
:
(
i
+
B
)
-
1
which is added as an offset to the weight.).
Regarding claim 12, Frantar teaches
The method of claim 11, wherein the transforming computationally extracts the particular mask value from the particular weight (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that the if statement computationally extracts the particular mask value from the particular weight.).
Regarding claim 13, Frantar teaches
The method of claim 1, wherein the machine-trained model has plural layers, and wherein the method is performed for transformations executed by each layer of the plural layers (Frantar, page 6, 4. Experiments, Setup. “Similar to Yao et al. (2022); Frantar et al. (2022a), we sparsify Transformer layers sequentially in order, which significantly reduces memory requirements.”).
Regarding claim 14, Frantar teaches
The method of claim 1, wherein the memory and the processor are a hardware memory unit an hardware processing unit of a computing system, and wherein the transforming computationally duplicates, in an inference stage, an effect of operating on the combined set of weights without a process of reconstituting the non-selected set of weights in memory prior to the transforming (Frantar, page 6, 4. Experiments Setup. “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” and Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models” (Frantar, page 1, Abstract) where “the achieved speedups are close to the theoretical optimum, which suggests that unstructured sparsity acceleration for LLM inference on CPUs is already quite practical” (Frantar, page 14, CPU Speedups). Examiner notes that the retraining is the reconstituting the non-selected set of weights in memory. Examiner further notes that the updating of the model occurs during the transformation according to Algorithm 1 instead of prior. )
Regarding claim 15, Frantar teaches
The method of claim 1, wherein the processor is a graphics processing unit or a neural processing unit (Frantar, page 6, 4. Experiments Setup. “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory.”).
Regarding claim 16, Frantar teaches
A non-transitory computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform operations, the operations comprising (Frantar, page 6, 4. Experiments Setup. “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory.”):
receiving an original set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask.),
sparsifying the original set of weights in a sparsification process, to produce a selected set of weights and a mask (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask.),
the sparsification process discriminating between the selected set of weights and a non-selected set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask. Examiner further notes that Algorithm 1 is the sparsification process and the binary pruning mask discriminates between the selected set of weights equal to 1 and the non-selected weights equal to 0.),
and the mask describing positions of the selected set of weights and the non-selected set of weights among a combined set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask. Examiner further notes that Algorithm 1 is the sparsification process and the binary pruning mask discriminates between the selected set of weights equal to 1 and the non-selected weights equal to 0. Examiner further notes that the i and j of the for loops are the positions.);
storing the selected set of weights and the mask in memory (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” (Frantar, page 6, 4. Experiments Setup.). Examiner notes that since all experiments are done on a GPU with memory, the weights and mask of Algorithm 1 is stored in memory.);
receiving an input embedding (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that H is the input embedding. Examiner further notes that according to paragraph 0039 of the specification, “the input embedding represents an input submission of any kind”.);
transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” (Frantar, page 6, 4. Experiments Setup.). Examiner notes that Algorithm 1 transforms H, the input embedding by performing computations on the weights, W, and the mask, M. Examiner further notes that W in the last line the last line of Algorithm 1 is the output result.)
the transforming computationally duplicating an effect of operating on the combined set of weights without a process of reconstituting the non-selected set of weights in memory prior to the transforming (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models” (Frantar, page 1, Abstract). Examiner notes that the retraining is the reconstituting the non-selected set of weights in memory. Examiner further notes that the updating of the model occurs during the transformation according to Algorithm 1 instead of prior. ).
Regarding claim 17, Frantar teaches
The non-transitory computer-readable storage medium of claim 16, wherein the transforming uses a particular mask value of the mask to evaluate plural applications of a particular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to a zero result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
where “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that M is the mask, the plural applications is pruning or not pruning the weights via the matrix multiplication occurring in the first update line of Algorithm 1 where E includes the mask and weight and is being multiplied by the input embedding H. Examiner further notes that “all but one of the plural applications resolving to a zero result” maps to the zeros that are a result of pruning the weights.).
Regarding claim 18, Frantar teaches
A computing system for determining whether a target item is supported by a source item, comprising: a memory; a processing system for executing computer-readable instructions, to perform operations including (Frantar, page 6, 4. Experiments Setup. “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory.”):
receiving a selected set of weights and a mask (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask.),
the selected set of weights and the mask having been produced by a sparsification process that operates on an original set of weights, the sparsification process discriminating between the selected set of weights and a non-selected set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask. Examiner further notes that Algorithm 1 is the sparsification process and the binary pruning mask discriminates between the selected set of weights equal to 1 and the non-selected weights equal to 0.),
and the mask describing positions of the selected set of weights and the non-selected set of weights among a combined set of weights (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that W is the set of weights and M or the adaptive mask selection is the mask. Examiner further notes that Algorithm 1 is the sparsification process and the binary pruning mask discriminates between the selected set of weights equal to 1 and the non-selected weights equal to 0. Examiner further notes that the i and j of the for loops are the positions.);
storing the selected set of weights and the mask in memory (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” (Frantar, page 6, 4. Experiments Setup.). Examiner notes that since all experiments are done on a GPU with memory, the weights and mask of Algorithm 1 is stored in memory.);
receiving an input embedding (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that H is the input embedding. Examiner further notes that according to paragraph 0039 of the specification, “the input embedding represents an input submission of any kind”.);
and transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” (Frantar, page 6, 4. Experiments Setup.). Examiner notes that Algorithm 1 transforms H, the input embedding by performing computations on the weights, W, and the mask, M. Examiner further notes that W in the last line the last line of Algorithm 1 is the output result.)
the transforming using a particular mask value of the mask to evaluate plural applications of a particular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to a zero result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
where “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that M is the mask, the plural applications is pruning or not pruning the weights via the matrix multiplication occurring in the first update line of Algorithm 1 where E includes the mask and weight and is being multiplied by the input embedding H. Examiner further notes that “all but one of the plural applications resolving to a zero result” maps to the zeros that are a result of pruning the weights.).
Regarding claim 19, Frantar teaches
The computing system of claim 18, wherein the sparsification process identifies the combined set of weights by setting a prescribed number of original weights in the original set of weights to the non-selected set of weights, based on a prescribed pattern (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. We can execute SparseGPT on the largest available open-source models, OPT-175B and BLOOM-176B, in under 4.5 hours, and can reach 60% unstructured sparsity with negligible increase in perplexity: remarkably, more than 100 billion weights from these models can be ignored at inference time. SparseGPT generalizes to semi-structured (2:4 and 4:8) patterns, and is compatible with weight quantization approaches” (Frantar, page 1, Abstract). Examiner notes that the sparsification process is SparseGPT. Examiner further notes that p is the prescribed number in the original weights being set to the non-selected weights. Examiner notes that the prescribed pattern is the semi-structured patterns of 2:4 and 4:8)
wherein the prescribed pattern specifies, for a particular group of weights, a number of selected weights to be included in the particular group (Frantar, page 1, Abstract, “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. We can execute SparseGPT on the largest available open-source models, OPT-175B and BLOOM-176B, in under 4.5 hours, and can reach 60% unstructured sparsity with negligible increase in perplexity: remarkably, more than 100 billion weights from these models can be ignored at inference time. SparseGPT generalizes to semi-structured (2:4 and 4:8) patterns, and is compatible with weight quantization approaches” where “SparseGPT is also easily adapted to semi-structured patterns such as the popular n:m sparsity format (Zhou et al., 2021; Hubara et al., 2021a) which delivers speedups in its 2:4 implementation on Ampere NVIDIA GPUs. Specifically, every consecutive m weights should contain exactly n zeros” (Frantar, page 5, 3.3 Extension to Semi-Structured Sparsity). Examiner notes that the prescribed pattern is the semi-structured pattern of 2:4 and 4:8 and the selected weights are the weights that aren’t 0.),
and wherein each mask value in the mask specifies a location of a particular selected weight in the particular group of weights (Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
Examiner notes that the mask M is the mask and each cell in mask M is a mask value which specifies a location of a selected weight in light gray in W.).
Regarding claim 20, Frantar teaches
The computing system of claim 19, wherein the prescribed number is 50 percent (Frantar, page 1, Abstract, “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. We can execute SparseGPT on the largest available open-source models, OPT-175B and BLOOM-176B, in under 4.5 hours, and can reach 60% unstructured sparsity with negligible increase in perplexity: remarkably, more than 100 billion weights from these models can be ignored at inference time. SparseGPT generalizes to semi-structured (2:4 and 4:8) patterns, and is compatible with weight quantization approaches”).
Regarding claim 21, Frantar teaches
The computing system of claim 18, wherein the transforming computationally duplicates, in an inference stage, an effect of operating on the combined set of weights without a process of reconstituting the non-selected set of weights in memory prior to the transforming (Frantar, page 6, 4. Experiments Setup. “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” and Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models” (Frantar, page 1, Abstract) where “the achieved speedups are close to the theoretical optimum, which suggests that unstructured sparsity acceleration for LLM inference on CPUs is already quite practical” (Frantar, page 14, CPU Speedups). Examiner notes that the retraining is the reconstituting the non-selected set of weights in memory. Examiner further notes that the updating of the model occurs during the transformation according to Algorithm 1 instead of prior. )
Response to Arguments
Examiner notes that the previous claim objections have been overcome in light of the instant amendments.
Examiner notes that the previous claim interpretation has been withdrawn in light of the instant amendments.
Examiner notes that the previous 112(b) rejections have been overcome in light of the instant amendments.
On pages 17-20, Applicant argues:
Paragraph No. [0002] of the specification, last sentence, summarizes one technical problem addressed by this application: "While all types of pruning improve the efficiency of a machine-trained model in some respects, the execution of even a pruned machine-trained model remains a resource-intensive task, particularly with respect to its utilization of memory." In other words, the process of pruning reduces the storage space that is required to store a machine-trained model ( e.g., on a hard disk). However, during execution of the machine-trained model in inference following pruning, the weights that have been pruned must be reconstituted. (Executing a machine-trained model in inference involves mapping an input, represented by an input embedding, to an output using the weights of the machine-trained model that have already been trained, and, in this case, pruned.) In one approach, weights that are reconstituted during inference are stored in memory as full-length entries. In this approach, even though the pruned model can be stored on a hard disk in an efficient manner, the execution of the pruned machine trained model still requires a significant amount of memory to store the reconstituted weights as full-length entries. This requirement, in turn, limits the types of devices that are capable of running the machine-trained model.
Paragraph No. [0003] of the specification summarizes a solution to this problem: "In an inference stage (during execution of the machine-trained model), a processor directly performs computations on the selected set of weights and the mask, without a preliminary step of reconstituting the non-selected weights in memory as full-length entries. Instead, the processor performs computations that take into account the influence of the non-selected weights in the combined set of weights."
More specifically, the specification describes two stages of operation, a sparsification stage 112 (e.g., a pruning stage) and an execution stage 124. The purpose of the sparsification stage is to reduce the number of selected weights associated with the machine-trained model. The purpose of the execution stage 124 is to execute the machine-trained model that has been sparsified by the sparsification stage. As described in Paragraph No. [0024], a "transformation component 130 directly operates on the selected set of weights 116 and the mask values in the mask 118 in 'desiccated form,' e.g., without first reconstituting the zero values of the combined set of weights 122 in memory 128." Paragraph No. [0049] describes one way of accomplishing this objective:
[0049] Viewed from another perspective, the program instruction accounts for two possibilities when it applies each mask value to a particular weight from the selected set of weights 212, one of which will inherently resolve to zero during the execution of the instruction (depending on whether the mask value is l or 0), and one of which will resolve to a non-zero result (if in fact the particular weight of the selected set of weights 212 is itself non-zero). The transformation component 130 can therefore be said to use each mask value to control a branching decision. In other words, the transformation component 130 uses a particular mask value of the mask to evaluate plural ways of applying a particular weight from the selected set of weights to the input embedding 216, all but one of which will resolve to zero.
The above execution-stage provisions have the illustrative technical advantages
described in Paragraph No. [0026]:
[0026] Third, the processing environment 102 reduces the consumption of the memory 128 during the execution of the model. For instance, the processing environment 102 avoids the need for representing the non-selected weights (e.g., the zero values) in the combined set of weights 122 in the memory 128 as foll 8 bit, 16 bit, 32 bit, or 64 bit entries, etc. Fourth, the processing environment 102 reduces the transactional costs associated with moving weights between the memory 128 and the transformation component 130. All of these characteristics also enable a computing device having limited resources to download, store, and run the model. For instance, the characteristics enable some user computing devices to nm the model without access to a server system and/or with reduced access to the server system.
Each of the independent claims recites subject matter that captures at least some of these technical improvements. For example, the last clause in claim 1 recites: "in a processor, transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result, wherein the transforming uses a particular mask value of the mask to evaluate plural applications of a particular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to a zero result." The last two clauses of claim 16 recite: "transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result," "the transforming computationally duplicating an effect of operating on the combined set of weights without a process of reconstituting the non-selected set of weights in memory prior to the transforming." The last clause of independent claim 18 recites: "the transforming using a particular mask value of the mask to evaluate plural applications of a pa1ticular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to a zero result." These highlighted operations have the effect of reducing the consumption memory during the execution of the model, as explained above (e.g., see Paragraph Nos. [0026] and [0049]).
Regarding the Applicant’s argument that the claims recite a technical improvement, Examiner has fully considered these amendments and has determined these arguments to be persuasive. Examiner has specifically found that the argument regarding how processing environment avoids the need for representing the non-selected weights in the combined set of weights in the memory to be persuasive. However, only claims 14, 16-17, and 21 reflect the improvement provided in paragraph 0026 as cited above. As such, the remaining claims maintain their rejection under 101.
On pages 22-23, Applicant argues:
As to claim 1, first note that the claim is directed to a "method for executing a
machine-trained model" The first four clauses of the claim recite receiving and storing a
set of weights and a mask associated with the machine-trained model. A sparsification
process produces the set of weights and the mask, but the sparsification process per se is
not the point of novelty of the invention recited in claim 1. As described in the
specification, any sparsification process could be used to produce the weights and mask
(see, for instance, Paragraph Nos. [0035] and [0036] of the specification). Indeed, the
specification explicitly mentions the possible use of SparseGPT to perform the pruning,
among other techniques. One point of novelty resides, in part, in the manner by which
the machine-trained model that has already been sparsified is executed. In claim 1, this is
captured by the last two clauses of the claim, which recite "receiving an input
embedding," and "in a processor, transforming the input embedding by performing
computations directly on the selected set of weights and the mask, to produce an output
result, wherein the transforming uses a particular mask value of the mask to evaluate
plural applications of a particular weight of the selected set of weights to the input
embedding, all but one of the plural applications resolving to a zero result."
Frantar is principally directed to how to produce a sparsified machine-trained
model, but is silent on the mechanics of how the machine-trained model that has been
sparsified is subsequently executed during inference. Hence, Frantar fails to anticipate at
least the last two clauses of claim 1, read in the context of claim 1 as a whole. To be
more specific, claim 1 has been amended to incorporate the subject matter of claim 7. On
page 41, the Office Action states that the subject matter of claim 7 is disclosed by
Algorithm 1 on page 5 of Frantar. However, Algorithm 1 of Frantar describes how to
prune the weights of a machine-trained model. It has no relevance to how a model that is
pruned by this algorithm is subsequently executed. This means that Algorithm l of
Frantar cannot be interpreted as the last two clauses of claim 1, which recite how the
machine-trained model is executed, not how it is sparsified.
Regarding the Applicant’s argument that Frantar does not disclose claim 1, Examiner respectfully disagrees. Specifically, Examiner respectfully notes that Frantar teaches:
receiving an input embedding (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
Examiner notes that H is the input embedding. Examiner further notes that according to paragraph 0039 of the specification, “the input embedding represents an input submission of any kind”.);
and in a processor, transforming the input embedding by performing computations directly on the selected set of weights and the mask, to produce an output result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1,
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and “All pruning experiments are conducted on a single NVIDIA A100 GPU with 80GB of memory” (Frantar, page 6, 4. Experiments Setup.). Examiner notes that Algorithm 1 transforms H, the input embedding by performing computations on the weights, W, and the mask, M. Examiner further notes that W in the last line the last line of Algorithm 1 is the output result.)
wherein the transforming uses a particular mask value of the mask to evaluate plural applications of a particular weight of the selected set of weights to the input embedding, all but one of the plural applications resolving to a zero result (Frantar, page 5, 3.4 Full Algorithm Pseudocode, Algorithm 1 (see below),
PNG
media_image1.png
785
752
media_image1.png
Greyscale
and Frantar, page 4, Figure 4,
PNG
media_image2.png
276
762
media_image2.png
Greyscale
where “throughout the paper, by sparsity mask for a given tensor we mean a binary tensor of the same dimensions, with 0 at the indices of the sparsified entries, and 1 at the other indices” (Frantar, page 2, Footnote 1). Examiner notes that M is the mask, the plural applications is pruning or not pruning the weights via the matrix multiplication occurring in the first update line of Algorithm 1 where E includes the mask and weight and is being multiplied by the input embedding H. Examiner further notes that “all but one of the plural applications resolving to a zero result” maps to the zeros that are a result of pruning the weights.).
Examiner further notes that under the broadest reasonable interpretation, producing a sparsification method such as in Frantar falls under “a method for executing a machine-trained model.”
On pages 23-24, Applicant argues:
Independent claims 16 and 18 are also directed to how a sparsified machine
Trained model is executed. Therefore, these claims are also not anticipated by Frantar.
Each of the dependent claims is allowable at least by virtue of its direct or
indirect dependency on an allowable independent claim. The changes made to dependent
claim 14 find illustrative support in the last clause of claim 16, as originally drafted. The
dependent claims also recite additional features that are not disclosed by Frantar For
example, dependent claim 8 sets forth a specific method of operating on the mask values
and weights during the execution of the machine-trained model (which is already in
sparsified form as received). The Office Action again directs the Applicant's attention to
Algorithm 1 on page 5 of Frantar. But to repeat, this algorithm describes how weights
are pruned, not how an already pruned machine-trained model is executed.
Regarding the Applicant’s argument that the dependent claims are allowable at least due in part to their dependency on the independent claims, the Examiner respectfully disagrees and notes the instant rejections and response to arguments regarding the independent claims above.
On page 24, Applicant argues:
New claim 21 depends on claim 1 and is allowable for at least that reason. Claim
21 finds illustrative support in the last clause of claim 16, as originally drafted.
Regarding the Applicant’s argument that the dependent claims are allowable at least due in part to their dependency on the independent claims, the Examiner respectfully disagrees and notes the instant rejections and response to arguments regarding the independent claims above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Sanh et al. (“Movement Pruning: Adaptive Sparsity by Fine-Tuning”) also discloses a pruning method to a transformer base machine learning model.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN R LAU whose telephone number is (571)272-1429. The examiner can normally be reached Monday - Thursday: 8:00 am - 6:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.L./Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights