DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment filed on Jun. 9th, 2026. The amendments are linked to the original application filed on May 30th, 2023.
Response to Amendments
Regarding Claim Rejections – 35 U.S.C. 101
The applicant argues that the claims do not recite abstract ideas and, even if they did, the applicant argues that the remaining limitations, as well as the claims as a whole and in light of the specification, integrate the inventive concepts into a practical application by providing an improvement to technology or technical field. The applicant points to limitations in claim 1 and the specification at ¶¶ 26-28 as reciting the technical improvements. Per the applicant, the improvements include a process of taking a sparse tensor and converting it to another format, during runtime, to further process the input tensor and then reverting the tensor back to another format. The application argues these limitations, and specification paragraphs, recites sufficient information as to integrate the claimed invention into practical application. Because of this, the applicant requests withdrawal of the rejection under 35 U.S.C. 101
The examiner would like to note that after each amendment the Alice/Mayo test is applied to ensure the claims recite patent eligible subject matter. During evaluation of the amended claims using the Alice/Mayo test, the examiner noted multiple limitations which are interpreted as mental steps or abstract ideas and further examination of the additional elements of the claims, as a whole and in light of the specification, is required.
For clarification, the examiner will evaluate claim 1 using the Alice/Mayo test. Regarding claim 1 Step 2A, Prong 1, the examiner noted the BRI of the limitation, “performing, during runtime of a neural network, a permutation on the input data having the sparse format” is broad and can be interpreted to be an abstract idea which is merely implemented on a computing system. A human is able to identify a tensor of any sparsity and convert a tensor with given algorithms or methods to another format. The process of converting from one format to another is interpreted by the examiner to be an abstract concept and further definition in the claims is required to limit the scope. This interpretation is applicable for the other limitation in claim 1 pertaining to converting tensors from a given sparsity to the original sparsity. Further, the examiner noted the BRI of limitation, “performing, by the neural network during runtime of the neural network, a computation on the permuted input data having the dense format …” is also very broad and allows for a broad BRI. The examiner noted that this limitation is merely using the converted tensor to execute a function to generate an output tensor. A human is able to receive data in a given data format, using a generic computing systems, and process it to produce a modified output tensor. The process of computing data to generate a modified output is interpreted by the examiner to be an abstract idea. The examiner did not note any other limitations of claim 1 to recite abstract ideas or mental concepts.
Next, the additional limitations are evaluated using Step 2A, Prong 2 and Step 2B. The applicant argues that the additional limitation integrates the claimed invention into a practical application by reciting improvements to technology or technical fields. The examiner respectfully disagrees. The remaining limitations of claim 1, which were not interpreted to be abstract ideas, are, “A system comprising: …” and “receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;”. The limitation, “A system comprising: …” recites generic instructions using a generic processing system containing generic components such as processors coupled to memory to perform or execute the claimed method. This limitation itself does not recite a technical improvement nor does this limitation integrate the abstract limitations into a practical application when viewing the claims as a whole. Next, the limitation, “receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” merely recites a well understood and routine processes of receiving and/or transmitting data over a network. This limitation itself does not recite a technical improvement, nor does this integrate the abstract limitations into a practical application. The MPEP 2106.04(d)(I) recites examples of limitations that fail to integrate the inventive concept into a practical application, “Merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f); Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g); and Generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h).” (Emphasis added). Using these examples, the remaining limitations fail to provide any inventive concepts and would fail to further integrate the abstract limitations into a practical application. The examiner would like to note that the specification does recite technical improvements, however, the limitations which recite the technical improvements were evaluated as abstract ideas or mental processes. During Step2A, Prong 2, the limitations which considered to be additional elements must be used, along with the claims as a whole, to determine if they integrate the abstract limitations into a practical application.
Using the BRI of the claims and the MPEP, the examiner believes the amended claims still recite abstract ideas and the additional limitations fail to integrate the claimed concept into a practical application as argued by the applicant. As stated above, the claims currently fail the Alice/Mayo test and the claimed limitations recite patent ineligible subject matter. Therefore, the examiner upholds the rejection, and the claims are rejected under 35 U.S.C. 101 for reciting abstract ideas or mental process, see 101 rejection below.
Regarding Claim Rejections – 35 U.S.C. 102/103
The applicant has made many amendments to the claims and argues that Yu fails to teach key elements of the claims. The applicant states, “Yu's compression remains sparse and nothing in Yu suggests converting sparse input data to a dense format within the context of amended claim 1”. The examiner respectfully disagrees. Yu teaches a system that is able to evaluate input tensors from a Neural network and determine the sparsity of the tensor. The system has predefined sparsity requirements for input tensors and will modify, compress or prune, values of the tensor in order to meet the sparsity requirements. The system in Yu will then process the tensor and reshape the tensor back to its original shape. This process is disclosed clearly in figure 1, reference numbers 102-112. Similar to the claimed system, Yu discloses key elements where one of ordinary skill would recognize the similarities. For example, both Yu and the claimed system intake input tensors, compresses the sparse tensors if needed, processes the compressed tensors, and after inference, both systems will reshape the modified input tensors back to their original shapes or dimensions. A POSITA would recognize that the claimed invention and Yu perform the same actions concerning input tensors to produce similar results and outputs. Further, both arts disclose how their invention improves inference of neural networks by allowing the system to process tensors with a recommended sparsity, either sparse or dense and archive this by altering tensors to meet requirements.
The examiner has reviewed the current amendments, remarks, specifications and prior arts. The examiner has noted and reviewed the amended claims. After evaluation of the independent claims the examiner notes that Yu is able to teach each and every limitation. Further, the examiner believes the prior art Yu and the claimed invention share similarities sufficiently equivalent to reject the claims under 35 U.S.C. 102. Therefore, the examiner has upheld the rejection under 35 USC 102, see 102 rejection below.
Next, the amended claims include new claims and as such, further interpretation of the claims is required. After this interpretation of the new claims the examiner notes that Yu is unable to teach each and every element of the new claims and further art is needed to support a 103 rejection. Next, the examiner reviewed Xie and found that this art fails to teach elements of the new claims. As such, at this time, the examiner no longer relies on Xie to teach claimed elements. Finally, the examiner has performed a complete and thorough search and has noted new art. The examiner has found art which, in combination with Yu, is able to teach the newly claimed subject matter. The proposed new art teaches better definitions of tiles and the selection of tiles for filtering input tensors. Therefore, the rejection under 35 U.S.C. 103 is also upheld, see 103 rejection below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 21-39 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 1 recites, "A system comprising: one or more processors; a non-transitory computer-readable medium storing programming instructions that, upon execution by the one or more processors, cause the system to perform the following operations" therefore it is directed to the statutory category of a machine.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“performing, during runtime of a neural network, a permutation on the input data having the sparse format, from the at least one input tensor, to obtain modified input data having a dense format along the at least one dimension, wherein the modified input data having the dense format is shorter than the input data having the sparse format along the at least one dimension;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to receive data, via a generic computing system, and, in real time, modify data using mental or mathematical concepts of observation and applying judgements and/or opinions. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“performing, by the neural network during runtime of the neural network, a computation on the permuted input data having the dense format to generate an output tensor, storing output data having a dense format along the at least one dimension; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to process, or evaluate, information, in real time, to convert data from one format to another. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“performing, during runtime, a reverse permutation on the output data having the dense format, from the output tensor, to obtain modified output data having the sparse format wherein permutation and the reverse permutation are defined as primitives in a sparse kernel.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to receive data post processing, via a generic computing system, and, in real time, convert data into a prior format using mental or mathematical concepts of observation and applying judgements and/or opinions. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “one or more processors; a non-transitory computer-readable medium storing programming instructions that, upon execution by the one or more processors, cause the system to perform the following operations” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “one or more processors; a non-transitory computer-readable medium storing programming instructions that, upon execution by the one or more processors, cause the system to perform the following operations” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data” and (see MPEP § 2106.05(d)(iv); “Storing and retrieving information in memory”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 21
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 22
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 23
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“analyzing, prior to runtime of the neural network, a sparsity pattern of the input data along the at least one dimension;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and determine a sparsity pattern in that data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate and identify data based on given criteria. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to process, or evaluate, data to produce an output or value. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “generating the sparse kernel based on the selected sparse tile.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “generating the sparse kernel based on the selected sparse tile.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 24
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 25
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 26
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“wherein performing the permutation comprises rearranging non-zero values of the input data to be contiguous along the at least one dimension such that zero values of the input data are excluded from the modified input data having the dense format.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a vector and rearrange values within a vector or other data structures. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 27
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 27 recites, "A method for improving runtime performance of a neural network by converting sparse input data to dense input data, the method comprising:" therefore it is directed to the statutory category of a process.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“performing, during runtime of a neural network, a permutation on the input data having the sparse format, from the at least one input tensor, to obtain modified input data having a dense format along the at least one dimension, wherein the modified input data having the dense format is shorter than the input data having the sparse format along the at least one dimension;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to receive data, via a generic computing system, and, in real time, modify data using mental or mathematical concepts of observation and applying judgements and/or opinions. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“performing, by the neural network during runtime of the neural network, a computation on the permuted input data having the dense format to generate an output tensor storing output data having a dense format along the at least one dimension; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to process, or evaluate, information, in real time, to convert data from one format to another. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“performing, during runtime, a reverse permutation on the output data having the dense format, from the output tensor, to obtain modified output data having the sparse format, wherein the permutation and the reverse permutation are defined as primitives in a sparse kernel.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to receive data post processing, via a generic computing system, and, in real time, convert data into a prior format using mental or mathematical concepts of observation and applying judgements and/or opinions. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “A method for improving runtime performance of a neural network by converting sparse input data to dense input data, the method comprising:” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “A method for improving runtime performance of a neural network by converting sparse input data to dense input data, the method comprising:” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data” and (see MPEP § 2106.05(d)(iv); “Storing and retrieving information in memory”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 28
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 29
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 30
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“analyzing, prior to runtime of the neural network, a sparsity pattern of the input data along the at least one dimension;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and determine a sparsity pattern in that data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate and identify data based on given criteria. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to process, or evaluate, data to produce an output or value. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “generating the sparse kernel based on the selected sparse tile.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “generating the sparse kernel based on the selected sparse tile.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 31
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 32
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 33
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“wherein performing the permutation comprises rearranging non-zero values of the input data to be contiguous along the at least one dimension such that zero values of the input data are excluded from the modified input data having the dense format.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a vector and rearrange values within a vector or other data structures. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 34
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 34 recites, "A computer program product for improving runtime performance of a neural network by converting sparse input data to dense input data, the computer program product storing programming instructions for execution by a processor of a system, the programming instructions, upon execution by the processor, causing the system to perform the following operations:" therefore it is directed to the statutory category of a machine.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“performing, during runtime of a neural network, a permutation on the input data having the sparse format, from the at least one input tensor, to obtain modified input data having a dense format along the at least one dimension, wherein the modified input data having the dense format is shorter than the input data having the sparse format along the at least one dimension;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to receive data, via a generic computing system, and, in real time, modify data using mental or mathematical concepts of observation and applying judgements and/or opinions. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“performing, by the neural network during runtime of the neural network, a computation on the permuted input data having the dense format to generate an output tensor storing output data having a dense format along the at least one dimension; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to process, or evaluate, information, in real time, to convert data from one format to another. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“performing, during runtime, a reverse permutation on the output data having the dense format, from the output tensor, to obtain modified output data having the sparse format, wherein the permutation and the reverse permutation are defined as primitives in a sparse kernel.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to receive data post processing, via a generic computing system, and, in real time, convert data into a prior format using mental or mathematical concepts of observation and applying judgements and/or opinions. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “A computer program product for improving runtime performance of a neural network by converting sparse input data to dense input data, the computer program product storing programming instructions for execution by a processor of a system, the programming instructions, upon execution by the processor, causing the system to perform the following operations:” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “A computer program product for improving runtime performance of a neural network by converting sparse input data to dense input data, the computer program product storing programming instructions for execution by a processor of a system, the programming instructions, upon execution by the processor, causing the system to perform the following operations:” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data” and (see MPEP § 2106.05(d)(iv); “Storing and retrieving information in memory”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 35
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 36
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 37
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“analyzing, prior to runtime of the neural network, a sparsity pattern of the input data along the at least one dimension;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and determine a sparsity pattern in that data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate and identify data based on given criteria. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to process, or evaluate, data to produce an output or value. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “generating the sparse kernel based on the selected sparse tile.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “generating the sparse kernel based on the selected sparse tile.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 38
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format, and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format, and” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
“wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 39
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The claim recites, inter alia:
“wherein performing the permutation comprises rearranging non-zero values of the input data to be contiguous along the at least one dimension such that zero values of the input data are excluded from the modified input data having the dense format.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate a vector and rearrange values within a vector or other data structures. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 21, 22, 26-29, 33-36, and 39 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Yu et al, (Yu et al, US 2023/0244942 A1, Filed 2022, Hereinafter “Yu”).
Regarding claim 1, Yu discloses, “A system comprising: one or more processors; a non-transitory computer-readable medium storing programming instructions that, upon execution by the one or more processors, cause the system to perform the following operations” (Computer-Based Systems, pp. 7, [0092]; "In at least one embodiment, one or more processors 702 each include one or more processor cores 707 to process instructions which, when executed, perform operations for system and user software. In at least one embodiment, each of one or more processor cores 707 is configured to process a specific instruction set 709." Yu discloses a system which contains processors which execute instructions from memory)
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” (Detailed Description, pp. 2-3, [0049]; “In at least one embodiment, at step 102, a processor analyzes whether an input tensor is compatible with said processor's processing resources. In at least one embodiment, at step 102, a kernel executed on a GPU automatically analyzes (e.g., detects, calculates, identifies) whether an input tensor, which may be a dense tensor (e.g., a tensor with no values of 0) or sparse tensor, meets a GPU's requirements for further processing (e.g., forward propagation in a neural network) as a sparse tensor (e.g., two out of every four tensor values are zero).” This article discloses the analysis of an input tensor after receiving it. This input tensor can be a sparse tensor.) and (Detailed description, pp. 2, [0048]; “In at least one embodiment, process 100 includes analyzing one or more input tensors and weight tensors with step 102. In at least one embodiment, input tensor of process 100 has dimensions required by a neural network. In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels.” This discloses that an input tensor for this system contains multi-dimensional data structures such as a 3D matrix.)
“performing, during runtime of a neural network, a permutation on the input data having the sparse format, from the at least one input tensor, to obtain modified input data having a dense format along the at least one dimension, wherein the modified input data having the dense format is shorter than the input data having the sparse format along the at least one dimension;” (Detailed Description, pp. 3, 0050]; “In at least one embodiment, at step 104, if a GPU detects that a weight tensor, if used, would not meet said GPU's requirements for accelerated processing, said GPU modifies said weight tensor to meet said requirements, for example, if a weight tensor does not exhibit 2:4 structured sparsity, then said GPU modifies said weight tensor's dimensions through padding (e.g., array padding, adding zeros to a tensor to match dimensional requirements), a reshaping operation, or some combination thereof to meet said GPU's 2:4 structured sparsity requirements. In at least one embodiment, modification of said weight tensor is based, at least in part, on dimensions of an input tensor. In at least one embodiment, modification of said weight tensor adds a minimum number of elements (e.g., least number of elements) to said weight tensor. In at least one embodiment, step 104 can be applied to any structured sparsity requirement (e.g., N:M structured sparsity).” This system will modify an input tensor to meet a given N:M structured sparsity. As stated, values N and M may be modified by the systems’ hardware constraints and model architecture. This system will be able identify an input tensor, which can be less sparse than the given N:M requirements, and modify the tensor, which is in a sparse format, to a denser input tensor to produce a modified input tensor.) and (Detailed Description, pp. 3, [0051]; “In at least one embodiment, a GPU will modify an input tensor based on a weight tensor's dimensions and said GPU's requirement of 2:4 structured sparsity for accelerated processing. In at least one embodiment, a GPU will modify an input tensor by expanding or coalescing said input tensor to make its dimensions and shapes suitable for computing (e.g., perform tensor operations) with a weight tensor meeting a GPU's requirements for structured sparsity.” As stated above, the input tensor may be expanded or coalesced, shortened, to meet the given sparsity requirements.) and (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402.” Further, this is an example compression technique disclosed in this application that can be performed.)
“performing, by the neural network during runtime of the neural network, a computation on the permuted input data having the dense format to generate an output tensor, storing output data having a dense format along the at least one dimension; and” (Detailed Description, pp. 3, [0052]; “In at least one embodiment, at step 108, a GPU performs tensor operations on an input tensor and a weight tensor. In at least one embodiment, computing an input tensor and weight tensor together involves tensor multiplication. In at least one embodiment, a GPU operating on an input tensor and weight tensor together involves convolution. In at least one embodiment, step 108 occurs during a feature extraction stage neural network training or inferencing.” This system will modify the input tensor to meet the sparsity requirements. The input tensor is processed by the model as stated above.)
“performing, during runtime, a reverse permutation on the output data having the dense format, from the output tensor, to obtain modified output data having the sparse format wherein permutation and the reverse permutation are defined as primitives in a sparse kernel.” (Detailed Description, pp. 3, [0054]; “In at least one embodiment, at step 112, a GPU shapes (e.g., reshapes, re-formats, resizes) a sparse output tensor produced with step 110 to match shapes of input tensors used to train a neural network. In at least one embodiment, step 112 modifies one or more output tensors to have a respective shape identical to shapes of one or more input shapes used to train a neural network. In at least one embodiment, said sparse output tensor is based, at least in part on said input tensors used to train a neural network. In at least one embodiment, a GPU reshapes modified input tensors, modified weight tensors, output tensors, or some combination thereof. In at least one embodiment, a GPU shapes an output tensor to match shapes of input tensors so said output tensor can be used as an input tensor in a layer of a neural network without modifying said neural network's framework.” This system will return the input tensor, after it has been modified, back to its original sparsity. As an example, this system can input tensor analyzed for the given sparsity requirement and modifies the tensor to meet the requirements. The system will perform ML operations on and/or using the tensor. Finally, as stated above, the system will reshape the output tensor to its original sparsity, i.e. sparse to dense then back to sparse.) and (Detailed Description, pp. 2, [0048]; “In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels. In at least one embodiment, an input tensor is an output tensor of a neural network layer (e.g., convolutional neural network layer) such as a feature map. In at least one embodiment, a weight tensor is a type of filter applied to (e.g., operated with) an input tensor. In at least one embodiment, a weight tensor is a representation, at least in part, of a weight matrix. In at least one embodiment, an input tensor results from applying a weight tensor to another input tensor. In at least one embodiment, a filter contains weights (e.g., parameters that change values of input data in a neural network), wherein weights can be represented as tensors.” As stated above the input tensor, which is modified by the system, contains parameter values. This would be represented by the system as primitive data structures such as integers or computer characters.)
Regarding claim 21, Yu discloses, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” (Detailed Description, p. 3, [0055]; "FIG. 2 illustrates a process 200 for modifying layers of a neural network, according to at least one embodiment. One or more aspects of process 200 as described herein can be used in combination with any embodiments as discussed in conjunction with at least FIGS. 1 and 3-5. In at least one embodiment, a processor identifies which layers share an input. In at least one embodiment, a GPU automatically, through a kernel execution, analyzes between layers of a machine learning model at step 202. In at least one embodiment, analyzing between layers 202 includes analyzing layers, input tensors, weight tensors, output tensors, or some combination thereof." The system will evaluate the input tensors and determine, during runtime, whether to compress the input tensor to a specified density. Since this is performed during runtime, the input tensor, which was stored in a memory unit, i.e. RAM or general storage, in its original sparse format. Then at inference, the input tensor is moved from general memory to cache memory for processing by the ALU. The input tensor will be processed and compressed at inference, after it was moved from general storage.) and (Computer-Based Systems, pp. 8, [0095]; "In at least one embodiment, memory device 720 can be a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, flash memory device, phase-change memory device, or some other memory device having suitable performance to serve as processor memory. In at least one embodiment memory device 720 can operate as system memory for processing system 700, to store data 722 and instructions 721 for use when one or more processors 702 executes an application or process." This process is completed on a general processing system. This is an example of the memory systems and subsystems used in this application.)
Regarding claim 22, Yu discloses, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” (Detailed Description, pp. 4, [0062]; “In at least one embodiment, modification 300 includes a kernel executed by a GPU that transforms a weight tensor and input tensor. In at least one embodiment, said transformed weight tensor and input tensor are used in operations for training neural network using a GPU's sparse tensor functionality. In at least one embodiment, said transformed weight tensor and input tensor, or some tensor based on said transformed weight tenor and input tensor (e.g., output tensor), are reshaped (e.g., returned) to an original irregular shape to connect with a following layer in a neural network. In at least one embodiment, said kernel designed to Al transform a weight tensor and input tensor to accelerate sparse tensors can improve computation time by 1.73x over time spent processing irregular weight and input tensors in a first convolutional layer of a neural network based on ResNet.” This system will intake and modify an input tensor during runtime and process the tensor using kernel operations. After this the tensor is reshaped to the original tensor sparsity. This process would occur after inference and while the data is stored in cache or RAM system. The proposed system is a processing system, meaning after operations are completed the input tensor if revert to its original shape and then moved out of cache or RAM memory to long term to make space for further data processing.) and (Computer-Based Systems, pp. 8, [0095]; "In at least one embodiment, memory device 720 can be a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, flash memory device, phase-change memory device, or some other memory device having suitable performance to serve as processor memory. In at least one embodiment memory device 720 can operate as system memory for processing system 700, to store data 722 and instructions 721 for use when one or more processors 702 executes an application or process." This process is completed on a general processing system. This is an example of the memory systems and subsystems used in this application.)
Regarding claim 26, Yu discloses, “wherein performing the permutation comprises rearranging non-zero values of the input data to be contiguous along the at least one dimension such that zero values of the input data are excluded from the modified input data having the dense format.” (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402. In at least one embodiment, metadata stored in an index 406 is used by a GPU to select corresponding activations from a second input tensor, letting said GPU skip multiplications by zero to increase throughput.” This system can compress sparse tensors which creates a new array of values to be stored in designed indices of the array. This will remove zero values from the data array prior to processing to meet the sparsity requirements of the system.)
Regarding claim 27, Yu discloses, “A method for improving runtime performance of a neural network by converting sparse input data to dense input data, the method comprising:” (Detailed Description, pp. 2, [0046]; "In at least one embodiment, methods and systems modify dimensions of one or more tensors based, at least in part, on one or more processing resources. In at least one embodiment, methods and systems modify (e.g., reshaping, transforming, converting) tensors (e.g., representations of numbers, scalars, arrays, vectors, two-dimensional (2D) arrays, matrices) used in neural network (e.g., residual neural network (ResNet), convolutional neural network (CNN), generative adversarial networks (GAN), artificial neural network (ANN), recurrent neural network (RNN)) on a graphics processing unit (GPU)) training or inferencing." This application discloses a method which is able to modify input tensors for processing and inference.)
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” (Detailed Description, pp. 2-3, [0049]; “In at least one embodiment, at step 102, a processor analyzes whether an input tensor is compatible with said processor's processing resources. In at least one embodiment, at step 102, a kernel executed on a GPU automatically analyzes (e.g., detects, calculates, identifies) whether an input tensor, which may be a dense tensor (e.g., a tensor with no values of 0) or sparse tensor, meets a GPU's requirements for further processing (e.g., forward propagation in a neural network) as a sparse tensor (e.g., two out of every four tensor values are zero).” This article discloses the analysis of an input tensor after receiving it. This input tensor can be a sparse tensor.) and (Detailed description, pp. 2, [0048]; “In at least one embodiment, process 100 includes analyzing one or more input tensors and weight tensors with step 102. In at least one embodiment, input tensor of process 100 has dimensions required by a neural network. In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels.” This discloses that an input tensor for this system contains multi-dimensional data structures such as a 3D matrix.)
“performing, during runtime of a neural network, a permutation on the input data having the sparse format, from the at least one input tensor, to obtain modified input data having a dense format along the at least one dimension, wherein the modified input data having the dense format is shorter than the input data having the sparse format along the at least one dimension;” (Detailed Description, pp. 3, 0050]; “In at least one embodiment, at step 104, if a GPU detects that a weight tensor, if used, would not meet said GPU's requirements for accelerated processing, said GPU modifies said weight tensor to meet said requirements, for example, if a weight tensor does not exhibit 2:4 structured sparsity, then said GPU modifies said weight tensor's dimensions through padding (e.g., array padding, adding zeros to a tensor to match dimensional requirements), a reshaping operation, or some combination thereof to meet said GPU's 2:4 structured sparsity requirements. In at least one embodiment, modification of said weight tensor is based, at least in part, on dimensions of an input tensor. In at least one embodiment, modification of said weight tensor adds a minimum number of elements (e.g., least number of elements) to said weight tensor. In at least one embodiment, step 104 can be applied to any structured sparsity requirement (e.g., N:M structured sparsity).” This system will modify an input tensor to meet a given N:M structured sparsity. As stated, values N and M may be modified by the systems’ hardware constraints and model architecture. This system will be able identify an input tensor, which can be less sparse than the given N:M requirements, and modify the tensor, which is in a sparse format, to a denser input tensor to produce a modified input tensor.) and (Detailed Description, pp. 3, [0051]; “In at least one embodiment, a GPU will modify an input tensor based on a weight tensor's dimensions and said GPU's requirement of 2:4 structured sparsity for accelerated processing. In at least one embodiment, a GPU will modify an input tensor by expanding or coalescing said input tensor to make its dimensions and shapes suitable for computing (e.g., perform tensor operations) with a weight tensor meeting a GPU's requirements for structured sparsity.” As stated above, the input tensor may be expanded or coalesced, shortened, to meet the given sparsity requirements.) and (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402.” Further, this is an example compression technique disclosed in this application that can be performed.)
“performing, by the neural network during runtime of the neural network, a computation on the permuted input data having the dense format to generate an output tensor storing output data having a dense format along the at least one dimension; and” (Detailed Description, pp. 3, [0052]; “In at least one embodiment, at step 108, a GPU performs tensor operations on an input tensor and a weight tensor. In at least one embodiment, computing an input tensor and weight tensor together involves tensor multiplication. In at least one embodiment, a GPU operating on an input tensor and weight tensor together involves convolution. In at least one embodiment, step 108 occurs during a feature extraction stage neural network training or inferencing.” This system will modify the input tensor to meet the sparsity requirements. The input tensor is processed by the model as stated above.)
“performing, during runtime, a reverse permutation on the output data having the dense format, from the output tensor, to obtain modified output data having the sparse format, wherein the permutation and the reverse permutation are defined as primitives in a sparse kernel.” (Detailed Description, pp. 3, [0054]; “In at least one embodiment, at step 112, a GPU shapes (e.g., reshapes, re-formats, resizes) a sparse output tensor produced with step 110 to match shapes of input tensors used to train a neural network. In at least one embodiment, step 112 modifies one or more output tensors to have a respective shape identical to shapes of one or more input shapes used to train a neural network. In at least one embodiment, said sparse output tensor is based, at least in part on said input tensors used to train a neural network. In at least one embodiment, a GPU reshapes modified input tensors, modified weight tensors, output tensors, or some combination thereof. In at least one embodiment, a GPU shapes an output tensor to match shapes of input tensors so said output tensor can be used as an input tensor in a layer of a neural network without modifying said neural network's framework.” This system will return the input tensor, after it has been modified, back to its original sparsity. As an example, this system can input tensor analyzed for the given sparsity requirement and modifies the tensor to meet the requirements. The system will perform ML operations on and/or using the tensor. Finally, as stated above, the system will reshape the output tensor to its original sparsity, i.e. sparse to dense then back to sparse.) and (Detailed Description, pp. 2, [0048]; “In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels. In at least one embodiment, an input tensor is an output tensor of a neural network layer (e.g., convolutional neural network layer) such as a feature map. In at least one embodiment, a weight tensor is a type of filter applied to (e.g., operated with) an input tensor. In at least one embodiment, a weight tensor is a representation, at least in part, of a weight matrix. In at least one embodiment, an input tensor results from applying a weight tensor to another input tensor. In at least one embodiment, a filter contains weights (e.g., parameters that change values of input data in a neural network), wherein weights can be represented as tensors.” As stated above the input tensor, which is modified by the system, contains parameter values. This would be represented by the system as primitive data structures such as integers or computer characters.)
Regarding claim 28, Yu discloses, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” (Detailed Description, p. 3, [0055]; "FIG. 2 illustrates a process 200 for modifying layers of a neural network, according to at least one embodiment. One or more aspects of process 200 as described herein can be used in combination with any embodiments as discussed in conjunction with at least FIGS. 1 and 3-5. In at least one embodiment, a processor identifies which layers share an input. In at least one embodiment, a GPU automatically, through a kernel execution, analyzes between layers of a machine learning model at step 202. In at least one embodiment, analyzing between layers 202 includes analyzing layers, input tensors, weight tensors, output tensors, or some combination thereof." The system will evaluate the input tensors and determine, during runtime, whether to compress the input tensor to a specified density. Since this is performed during runtime, the input tensor, which was stored in a memory unit, i.e. RAM or general storage, in its original sparse format. Then at inference, the input tensor is moved from general memory to cache memory for processing by the ALU. The input tensor will be processed and compressed at inference, after it was moved from general storage.) and (Computer-Based Systems, pp. 8, [0095]; "In at least one embodiment, memory device 720 can be a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, flash memory device, phase-change memory device, or some other memory device having suitable performance to serve as processor memory. In at least one embodiment memory device 720 can operate as system memory for processing system 700, to store data 722 and instructions 721 for use when one or more processors 702 executes an application or process." This process is completed on a general processing system. This is an example of the memory systems and subsystems used in this application.)
Regarding claim 29, Yu discloses, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” (Detailed Description, pp. 4, [0062]; “In at least one embodiment, modification 300 includes a kernel executed by a GPU that transforms a weight tensor and input tensor. In at least one embodiment, said transformed weight tensor and input tensor are used in operations for training neural network using a GPU's sparse tensor functionality. In at least one embodiment, said transformed weight tensor and input tensor, or some tensor based on said transformed weight tenor and input tensor (e.g., output tensor), are reshaped (e.g., returned) to an original irregular shape to connect with a following layer in a neural network. In at least one embodiment, said kernel designed to Al transform a weight tensor and input tensor to accelerate sparse tensors can improve computation time by 1.73x over time spent processing irregular weight and input tensors in a first convolutional layer of a neural network based on ResNet.” This system will intake and modify an input tensor during runtime and process the tensor using kernel operations. After this the tensor is reshaped to the original tensor sparsity. This process would occur after inference and while the data is stored in cache or RAM system. The proposed system is a processing system, meaning after operations are completed the input tensor if revert to its original shape and then moved out of cache or RAM memory to long term to make space for further data processing.) and (Computer-Based Systems, pp. 8, [0095]; "In at least one embodiment, memory device 720 can be a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, flash memory device, phase-change memory device, or some other memory device having suitable performance to serve as processor memory. In at least one embodiment memory device 720 can operate as system memory for processing system 700, to store data 722 and instructions 721 for use when one or more processors 702 executes an application or process." This process is completed on a general processing system. This is an example of the memory systems and subsystems used in this application.)
Regarding claim 33, Yu discloses, “wherein performing the permutation comprises rearranging non-zero values of the input data to be contiguous along the at least one dimension such that zero values of the input data are excluded from the modified input data having the dense format.” (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402. In at least one embodiment, metadata stored in an index 406 is used by a GPU to select corresponding activations from a second input tensor, letting said GPU skip multiplications by zero to increase throughput.” This system can compress sparse tensors which creates a new array of values to be stored in designed indices of the array. This will remove zero values from the data array prior to processing to meet the sparsity requirements of the system.)
Regarding claim 34, Yu discloses, “A computer program product for improving runtime performance of a neural network by converting sparse input data to dense input data, the computer program product storing programming instructions for execution by a processor of a system, the programming instructions, upon execution by the processor, causing the system to perform the following operations:” (Computer-Based Systems, pp. 7, [0092]; "In at least one embodiment, one or more processors 702 each include one or more processor cores 707 to process instructions which, when executed, perform operations for system and user software. In at least one embodiment, each of one or more processor cores 707 is configured to process a specific instruction set 709." Yu discloses a system which contains processors and execute instructions stored in memory.)
“receiving at least one input tensor storing input data having a sparse format along at least one of a plurality of dimensions;” (Detailed Description, pp. 2-3, [0049]; “In at least one embodiment, at step 102, a processor analyzes whether an input tensor is compatible with said processor's processing resources. In at least one embodiment, at step 102, a kernel executed on a GPU automatically analyzes (e.g., detects, calculates, identifies) whether an input tensor, which may be a dense tensor (e.g., a tensor with no values of 0) or sparse tensor, meets a GPU's requirements for further processing (e.g., forward propagation in a neural network) as a sparse tensor (e.g., two out of every four tensor values are zero).” This article discloses the analysis of an input tensor after receiving it. This input tensor can be a sparse tensor.) and (Detailed description, pp. 2, [0048]; “In at least one embodiment, process 100 includes analyzing one or more input tensors and weight tensors with step 102. In at least one embodiment, input tensor of process 100 has dimensions required by a neural network. In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels.” This discloses that an input tensor for this system contains multi-dimensional data structures such as a 3D matrix.)
“performing, during runtime of a neural network, a permutation on the input data having the sparse format, from the at least one input tensor, to obtain modified input data having a dense format along the at least one dimension, wherein the modified input data having the dense format is shorter than the input data having the sparse format along the at least one dimension;” (Detailed Description, pp. 3, 0050]; “In at least one embodiment, at step 104, if a GPU detects that a weight tensor, if used, would not meet said GPU's requirements for accelerated processing, said GPU modifies said weight tensor to meet said requirements, for example, if a weight tensor does not exhibit 2:4 structured sparsity, then said GPU modifies said weight tensor's dimensions through padding (e.g., array padding, adding zeros to a tensor to match dimensional requirements), a reshaping operation, or some combination thereof to meet said GPU's 2:4 structured sparsity requirements. In at least one embodiment, modification of said weight tensor is based, at least in part, on dimensions of an input tensor. In at least one embodiment, modification of said weight tensor adds a minimum number of elements (e.g., least number of elements) to said weight tensor. In at least one embodiment, step 104 can be applied to any structured sparsity requirement (e.g., N:M structured sparsity).” This system will modify an input tensor to meet a given N:M structured sparsity. As stated, values N and M may be modified by the systems’ hardware constraints and model architecture. This system will be able identify an input tensor, which can be less sparse than the given N:M requirements, and modify the tensor, which is in a sparse format, to a denser input tensor to produce a modified input tensor.) and (Detailed Description, pp. 3, [0051]; “In at least one embodiment, a GPU will modify an input tensor based on a weight tensor's dimensions and said GPU's requirement of 2:4 structured sparsity for accelerated processing. In at least one embodiment, a GPU will modify an input tensor by expanding or coalescing said input tensor to make its dimensions and shapes suitable for computing (e.g., perform tensor operations) with a weight tensor meeting a GPU's requirements for structured sparsity.” As stated above, the input tensor may be expanded or coalesced, shortened, to meet the given sparsity requirements.) and (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402.” Further, this is an example compression technique disclosed in this application that can be performed.)
“performing, by the neural network during runtime of the neural network, a computation on the permuted input data having the dense format to generate an output tensor storing output data having a dense format along the at least one dimension; and” (Detailed Description, pp. 3, [0052]; “In at least one embodiment, at step 108, a GPU performs tensor operations on an input tensor and a weight tensor. In at least one embodiment, computing an input tensor and weight tensor together involves tensor multiplication. In at least one embodiment, a GPU operating on an input tensor and weight tensor together involves convolution. In at least one embodiment, step 108 occurs during a feature extraction stage neural network training or inferencing.” This system will modify the input tensor to meet the sparsity requirements. The input tensor is processed by the model as stated above.)
“performing, during runtime, a reverse permutation on the output data having the dense format, from the output tensor, to obtain modified output data having the sparse format, wherein the permutation and the reverse permutation are defined as primitives in a sparse kernel.” (Detailed Description, pp. 3, [0054]; “In at least one embodiment, at step 112, a GPU shapes (e.g., reshapes, re-formats, resizes) a sparse output tensor produced with step 110 to match shapes of input tensors used to train a neural network. In at least one embodiment, step 112 modifies one or more output tensors to have a respective shape identical to shapes of one or more input shapes used to train a neural network. In at least one embodiment, said sparse output tensor is based, at least in part on said input tensors used to train a neural network. In at least one embodiment, a GPU reshapes modified input tensors, modified weight tensors, output tensors, or some combination thereof. In at least one embodiment, a GPU shapes an output tensor to match shapes of input tensors so said output tensor can be used as an input tensor in a layer of a neural network without modifying said neural network's framework.” This system will return the input tensor, after it has been modified, back to its original sparsity. As an example, this system can input tensor analyzed for the given sparsity requirement and modifies the tensor to meet the requirements. The system will perform ML operations on and/or using the tensor. Finally, as stated above, the system will reshape the output tensor to its original sparsity, i.e. sparse to dense then back to sparse.) and (Detailed Description, pp. 2, [0048]; “In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels. In at least one embodiment, an input tensor is an output tensor of a neural network layer (e.g., convolutional neural network layer) such as a feature map. In at least one embodiment, a weight tensor is a type of filter applied to (e.g., operated with) an input tensor. In at least one embodiment, a weight tensor is a representation, at least in part, of a weight matrix. In at least one embodiment, an input tensor results from applying a weight tensor to another input tensor. In at least one embodiment, a filter contains weights (e.g., parameters that change values of input data in a neural network), wherein weights can be represented as tensors.” As stated above the input tensor, which is modified by the system, contains parameter values. This would be represented by the system as primitive data structures such as integers or computer characters.)
Regarding claim 35, Yu discloses, “wherein the permutation on the input data having the sparse format is performed when loading the input data from general memory to shared memory.” (Detailed Description, p. 3, [0055]; "FIG. 2 illustrates a process 200 for modifying layers of a neural network, according to at least one embodiment. One or more aspects of process 200 as described herein can be used in combination with any embodiments as discussed in conjunction with at least FIGS. 1 and 3-5. In at least one embodiment, a processor identifies which layers share an input. In at least one embodiment, a GPU automatically, through a kernel execution, analyzes between layers of a machine learning model at step 202. In at least one embodiment, analyzing between layers 202 includes analyzing layers, input tensors, weight tensors, output tensors, or some combination thereof." The system will evaluate the input tensors and determine, during runtime, whether to compress the input tensor to a specified density. Since this is performed during runtime, the input tensor, which was stored in a memory unit, i.e. RAM or general storage, in its original sparse format. Then at inference, the input tensor is moved from general memory to cache memory for processing by the ALU. The input tensor will be processed and compressed at inference, after it was moved from general storage.) and (Computer-Based Systems, pp. 8, [0095]; "In at least one embodiment, memory device 720 can be a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, flash memory device, phase-change memory device, or some other memory device having suitable performance to serve as processor memory. In at least one embodiment memory device 720 can operate as system memory for processing system 700, to store data 722 and instructions 721 for use when one or more processors 702 executes an application or process." This process is completed on a general processing system. This is an example of the memory systems and subsystems used in this application.)
Regarding claim 36, Yu discloses, “wherein the reverse permutation on the output data having the dense format is performed when storing the modified output data from shared memory to general memory.” (Detailed Description, pp. 4, [0062]; “In at least one embodiment, modification 300 includes a kernel executed by a GPU that transforms a weight tensor and input tensor. In at least one embodiment, said transformed weight tensor and input tensor are used in operations for training neural network using a GPU's sparse tensor functionality. In at least one embodiment, said transformed weight tensor and input tensor, or some tensor based on said transformed weight tenor and input tensor (e.g., output tensor), are reshaped (e.g., returned) to an original irregular shape to connect with a following layer in a neural network. In at least one embodiment, said kernel designed to Al transform a weight tensor and input tensor to accelerate sparse tensors can improve computation time by 1.73x over time spent processing irregular weight and input tensors in a first convolutional layer of a neural network based on ResNet.” This system will intake and modify an input tensor during runtime and process the tensor using kernel operations. After this the tensor is reshaped to the original tensor sparsity. This process would occur after inference and while the data is stored in cache or RAM system. The proposed system is a processing system, meaning after operations are completed the input tensor if revert to its original shape and then moved out of cache or RAM memory to long term to make space for further data processing.) and (Computer-Based Systems, pp. 8, [0095]; "In at least one embodiment, memory device 720 can be a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, flash memory device, phase-change memory device, or some other memory device having suitable performance to serve as processor memory. In at least one embodiment memory device 720 can operate as system memory for processing system 700, to store data 722 and instructions 721 for use when one or more processors 702 executes an application or process." This process is completed on a general processing system. This is an example of the memory systems and subsystems used in this application.)
Regarding claim 39, Yu discloses, “wherein performing the permutation comprises rearranging non-zero values of the input data to be contiguous along the at least one dimension such that zero values of the input data are excluded from the modified input data having the dense format.” (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402. In at least one embodiment, metadata stored in an index 406 is used by a GPU to select corresponding activations from a second input tensor, letting said GPU skip multiplications by zero to increase throughput.” This system can compress sparse tensors which creates a new array of values to be stored in designed indices of the array. This will remove zero values from the data array prior to processing to meet the sparsity requirements of the system.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 23-25, 30-32, 37, and 38 are rejected under 35 U.S.C. 103 as being unpatentable over Yu in view of Laddha et al, (Laddha et al, US 2021/0090328 A1, Filed 2020, hereinafter “Laddha”).
Regarding claim 23, Yu discloses, “analyzing, prior to runtime of the neural network, a sparsity pattern of the input data along the at least one dimension;” (Detailed Description, pp. 2, [0048]; “In at least one embodiment, process 100 includes analyzing one or more input tensors and weight tensors with step 102. In at least one embodiment, input tensor of process 100 has dimensions required by a neural network. In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels.” This system will analyze a layer of a neural network prior to its execution by the system. The system will identify the sparsity pattern and assess whether the pattern meets the sparsity requirements.)
Yu fails to explicitly disclose:
“selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;”
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and”
“generating the sparse kernel based on the selected sparse tile.”
However, Laddha discloses, “selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;” (Sparsity-Aware Optimal Dataflow, pp. 5, [0049]; “At processing block 530, for given neural network layer and architecture parameters, tile candidates may be selected such that they fit within constrained on-chip (or cache) memory. Tile size may be estimated for a candidate tile (drb, die, doc) using the rulebook-specific max sparsity attributes (already discussed) o2imax and o2rbmax (for an o2i-rulebook) or i2omax and i2rbmax (for an i2o-rulebook).” This system discloses a process for evaluating datasets with large sparse tensors, such as point clouds. This will take an input tensor and evaluate the sparsity and apply different functions, tiles, to the tensors during processing.)
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and” (Description of Embodiments, pp. 2, [0022]; “The system 100 may also include a data sparsity attribute generator 140 that processes the locality-aware rulebook(s) 132 and generates a set of data sparsity attributes 142 representing the sparsity of active data (i.e., active voxels) in the input 3D point cloud data set 110. The data sparsity attributes 142 are further described herein and with reference to FIG. 4A.” This system will process the tensor for the sparsity to identify elements in the tensor and generate a set of data sparsity attributes.)
“generating the sparse kernel based on the selected sparse tile.” (Description of Embodiments, pp. 3, [0031]; “The rulebooks shown in FIG. 2B correspond to a down-sampling network layer for a neural network using 2D sparse convolution with a 3x3 kernel (e.g., filter) such that the output resolution for the layer is one-half of the input. Two rulebook variants are shown, i2o rulebook 240 and o2i rulebook 250. i2o rulebook 240 has an i2o data structure 241 (partially illustrated) which includes three rb-lines corresponding to input voxel indices 4, 5 and 6 and, for each input voxel rb-line, the corresponding bitmask and output voxel indices. The diagram further shows an example set of input activation data 242 with indices representing active (i.e., non-zero) data points and corresponding example set of output activation data 243 with indices representing active (i.e., non-zero) data points.” As stated above, the system will generate a kernel or filter and loop order for processing the tensor.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Yu and Laddha. Yu teaches a system that is able to evaluate an input tensor and modify it to fit system requirements in real time. Laddha teaches a system that processes many sparse tensors and discloses methods to process these sparse tensors using tiles. One of ordinary skill would have motivation to combine a system that performs layer wise evaluation of tensors for modification with a system that is able to provide different compression methods for sparse tensors to solve the problem of handling sparse input tensors efficiently and effectively, “An improved computing system as described herein provides technology to optimize (e.g., accelerate) processing of unstructured sparse data, such as 3D pointcloud data, by a compute engine (which may include a neural network such as a convolution neural network (CNN)) through tile-based execution while orchestrating optimal dataflow for the data processing with input-dependent spatial sparsity.” (Laddha, Description of Embodiments, pp. 1, [0018])
Regarding claim 24, Laddha discloses, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format.” (Description of Embodiments, pp. 3, [0031]; “FIG. 2B provides a diagram illustrating aspects of example locality-aware rulebooks according to one or more embodiments, with reference to components and features described herein including but not limited to the figures and associated description. The rulebooks shown in FIG. 2B correspond to a down sampling network layer for a neural network using 2D sparse convolution with a 3x3 kernel (e.g., filter) such that the output resolution for the layer is one-half of the input.” This system will evaluate the input layer and identify the appropriate tile to generate and apply. This tile will represent the kernel or filter used and in this example the output kernel is in a different dimension as compared to the input.)
Regarding claim 25, Yu discloses, “wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402. In at least one embodiment, metadata stored in an index 406 is used by a GPU to select corresponding activations from a second input tensor, letting said GPU skip multiplications by zero to increase throughput.” As stated above, the system can compress a sparse tensor and during this compress generate metadata which can store values. Further, the system notes the activation functions to be performed during layer wise operations. This is interpreted by the examiner to be the permutation invariant.)
Regarding claim 30, Yu discloses, “analyzing, prior to runtime of the neural network, a sparsity pattern of the input data along the at least one dimension;” (Detailed Description, pp. 2, [0048]; “In at least one embodiment, process 100 includes analyzing one or more input tensors and weight tensors with step 102. In at least one embodiment, input tensor of process 100 has dimensions required by a neural network. In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels.” This system will analyze a layer of a neural network prior to its execution by the system. The system will identify the sparsity pattern and assess whether the pattern meets the sparsity requirements.)
Yu fails to explicitly disclose:
“selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;”
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and”
“generating the sparse kernel based on the selected sparse tile.”
However, Ladda discloses, “selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;” (Sparsity-Aware Optimal Dataflow, pp. 5, [0049]; “At processing block 530, for given neural network layer and architecture parameters, tile candidates may be selected such that they fit within constrained on-chip (or cache) memory. Tile size may be estimated for a candidate tile (drb, die, doc) using the rulebook-specific max sparsity attributes (already discussed) o2imax and o2rbmax (for an o2i-rulebook) or i2omax and i2rbmax (for an i2o-rulebook).” This system discloses a process for evaluating datasets with large sparse tensors, such as point clouds. This will take an input tensor and evaluate the sparsity and apply different functions, tiles, to the tensors during processing.)
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and” (Description of Embodiments, pp. 2, [0022]; “The system 100 may also include a data sparsity attribute generator 140 that processes the locality-aware rulebook(s) 132 and generates a set of data sparsity attributes 142 representing the sparsity of active data (i.e., active voxels) in the input 3D point cloud data set 110. The data sparsity attributes 142 are further described herein and with reference to FIG. 4A.” This system will process the tensor for the sparsity to identify elements in the tensor and generate a set of data sparsity attributes.)
“generating the sparse kernel based on the selected sparse tile.” (Description of Embodiments, pp. 3, [0031]; “The rulebooks shown in FIG. 2B correspond to a down-sampling network layer for a neural network using 2D sparse convolution with a 3x3 kernel (e.g., filter) such that the output resolution for the layer is one-half of the input. Two rulebook variants are shown, i2o rulebook 240 and o2i rulebook 250. i2o rulebook 240 has an i2o data structure 241 (partially illustrated) which includes three rb-lines corresponding to input voxel indices 4, 5 and 6 and, for each input voxel rb-line, the corresponding bitmask and output voxel indices. The diagram further shows an example set of input activation data 242 with indices representing active (i.e., non-zero) data points and corresponding example set of output activation data 243 with indices representing active (i.e., non-zero) data points.” As stated above, the system will generate a kernel or filter and loop order for processing the tensor.)
Regarding claim 31, Laddha discloses, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format.” (Description of Embodiments, pp. 3, [0031]; “FIG. 2B provides a diagram illustrating aspects of example locality-aware rulebooks according to one or more embodiments, with reference to components and features described herein including but not limited to the figures and associated description. The rulebooks shown in FIG. 2B correspond to a down sampling network layer for a neural network using 2D sparse convolution with a 3x3 kernel (e.g., filter) such that the output resolution for the layer is one-half of the input.” This system will evaluate the input layer and identify the appropriate tile to generate and apply. This tile will represent the kernel or filter used and in this example the output kernel is in a different dimension as compared to the input.)
Regarding claim 32, Yu discloses, “wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402. In at least one embodiment, metadata stored in an index 406 is used by a GPU to select corresponding activations from a second input tensor, letting said GPU skip multiplications by zero to increase throughput.” As stated above, the system can compress a sparse tensor and during this compress generate metadata which can store values. Further, the system notes the activation functions to be performed during layer wise operations. This is interpreted by the examiner to be the permutation invariant.)
Regarding claim 37, Yu discloses, “analyzing, prior to runtime of the neural network, a sparsity pattern of the input data along the at least one dimension;” (Detailed Description, pp. 2, [0048]; “In at least one embodiment, process 100 includes analyzing one or more input tensors and weight tensors with step 102. In at least one embodiment, input tensor of process 100 has dimensions required by a neural network. In at least one embodiment, an input tensor is an input image or a feature map. In at least one embodiment, an input tensor is a three-dimensional matrix where two dimensions correspond to an image's length and width in pixels.” This system will analyze a layer of a neural network prior to its execution by the system. The system will identify the sparsity pattern and assess whether the pattern meets the sparsity requirements.)
Yu fails to explicitly disclose:
“selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;”
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and”
“generating the sparse kernel based on the selected sparse tile.”
However, Laddha discloses, “selecting a sparse tile from pre-constructed sparse tiles based on the sparsity pattern;” (Sparsity-Aware Optimal Dataflow, pp. 5, [0049]; “At processing block 530, for given neural network layer and architecture parameters, tile candidates may be selected such that they fit within constrained on-chip (or cache) memory. Tile size may be estimated for a candidate tile (drb, die, doc) using the rulebook-specific max sparsity attributes (already discussed) o2imax and o2rbmax (for an o2i-rulebook) or i2omax and i2rbmax (for an i2o-rulebook).” This system discloses a process for evaluating datasets with large sparse tensors, such as point clouds. This will take an input tensor and evaluate the sparsity and apply different functions, tiles, to the tensors during processing.)
“generating a sparsity index identifying locations of non-zero values within the input data based on the selected sparse tile; and” (Description of Embodiments, pp. 2, [0022]; “The system 100 may also include a data sparsity attribute generator 140 that processes the locality-aware rulebook(s) 132 and generates a set of data sparsity attributes 142 representing the sparsity of active data (i.e., active voxels) in the input 3D point cloud data set 110. The data sparsity attributes 142 are further described herein and with reference to FIG. 4A.” This system will process the tensor for the sparsity to identify elements in the tensor and generate a set of data sparsity attributes.)
“generating the sparse kernel based on the selected sparse tile.” (Description of Embodiments, pp. 3, [0031]; “The rulebooks shown in FIG. 2B correspond to a down-sampling network layer for a neural network using 2D sparse convolution with a 3x3 kernel (e.g., filter) such that the output resolution for the layer is one-half of the input. Two rulebook variants are shown, i2o rulebook 240 and o2i rulebook 250. i2o rulebook 240 has an i2o data structure 241 (partially illustrated) which includes three rb-lines corresponding to input voxel indices 4, 5 and 6 and, for each input voxel rb-line, the corresponding bitmask and output voxel indices. The diagram further shows an example set of input activation data 242 with indices representing active (i.e., non-zero) data points and corresponding example set of output activation data 243 with indices representing active (i.e., non-zero) data points.” As stated above, the system will generate a kernel or filter and loop order for processing the tensor.)
Regarding claim 38, Yu discloses, “wherein the sparse tile identifies the at least one dimension of the plurality of dimensions as being permutation invariant.” (Detailed Description, pp. 5, [0065]; “In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404, which is half sparse tensor's 402 size. In at least one embodiment, a GPU compresses sparse tensor 402 to become compressed tensor 404 and creates an array of metadata (e.g., index 406) to keep track of where non-zeros were in uncompressed sparse tensor 402. In at least one embodiment, metadata stored in an index 406 is used by a GPU to select corresponding activations from a second input tensor, letting said GPU skip multiplications by zero to increase throughput.” As stated above, the system can compress a sparse tensor and during this compress generate metadata which can store values. Further, the system notes the activation functions to be performed during layer wise operations. This is interpreted by the examiner to be the permutation invariant.)
Yu fails to explicitly disclose:
“wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format, and”
However, Ladda discloses, “wherein the sparse kernel comprises a data tile describing a shape of the input data in the at least one input tensor and a computation tile describing a shape of the modified input data having the dense format, and” (Description of Embodiments, pp. 3, [0031]; “FIG. 2B provides a diagram illustrating aspects of example locality-aware rulebooks according to one or more embodiments, with reference to components and features described herein including but not limited to the figures and associated description. The rulebooks shown in FIG. 2B correspond to a down sampling network layer for a neural network using 2D sparse convolution with a 3x3 kernel (e.g., filter) such that the output resolution for the layer is one-half of the input.” This system will evaluate the input layer and identify the appropriate tile to generate and apply. This tile will represent the kernel or filter used and in this example the output kernel is in a different dimension as compared to the input.)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL MICHAEL GALVIN-SIEBENALER whose telephone number is (571)272-1257. The examiner can normally be reached Monday - Friday 8AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL M GALVIN-SIEBENALER/Examiner, Art Unit 2147
/HASSAN MRABI/Primary Examiner, Art Unit 2147
Prosecution Insights