Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office Action is sent in response to Application’s Communication received on 08/22/2022 for application number 17/892145. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawing, Abstract, Oath/Declaration, and Claims.
Claims (1-11), 12 and 13 are presented for examination.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 02/07/2023 was filed prior to current Office Action. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Allowable Subject Matter
Claims 6, 8-11 are directed to allowable subject matter if the 101 is rejection is addressed.
Claim Rejections - 35 USC§ 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims (1-11), 12 and 13 are rejected under 35 U.S.C. 101 because the claimed
invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an
abstract idea) without significantly more.
Step 1: Claims (1-11), 12 and 13 are drawn to a method each of which is within the four statutory categories (e.g., a process, a machine).
Step 2A - Prong One: In prong one of step 2A, the claims are analyzed to evaluate whether they recite a judicial exception.
Claim 1.
A simplification method for neural network model, configured to simplify an original trained neural network model to a simplified trained neural network model, wherein the simplified trained neural network model comprises at most two linear operation layers, and the simplification method for neural network model comprises:
receiving the original trained neural network model;
calculating a first new weight of the at most two linear operation layers of the
simplified trained neural network model by using a plurality of original weights of the original
trained neural network model; and
generating the simplified trained neural network model based on the first new weight.
The limitation recites “calculating a first new weight of the at most two linear operation layers of the simplified trained neural network model by using a plurality of original weights of the original trained neural network model” which recites a mathematical concept. For example, the claimed “calculating” under its broadest reasonable interpretation when read in light of the specification encompasses using mathematical formulas as describes in paragraph [0028] to determine the new weight.
The limitations recite “generating the simplified trained neural network model based on the first new weight” which recites a mathematical concept” which recites a mathematical concept. For example, the claimed “generating” under its broadest reasonable interpretation when read in light of the specification encompasses using mathematical formulas, as describes in paragraph [0028], to generate the new weight based on calculating the most two matrices by using a plurality of original weight.
Step 2A Prong 2:
Claim 1 recites additional elements such as “receiving the original trained neural network model” which are recited at a high level, the elements are merely reciting the words that pertain to a generic computer (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The “receiving” is an additional element amount to merely Insignificant Extra-Solution Activity. The limitation does not integrate the judicial exception into a practical application, therefore; the addition of insignificant extra-solution activity does not amount to an inventive concept
Claim 2
Claim 2 recites a mathematical concept, the limitation includes mathematical formula and words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 3
Claim 3 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 4
Claim 4 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 5
Claim 5 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 6
Claim 6 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 7
Claim 7 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 8
Claim 8 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 9
Claim 9 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 10
Claim 10 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Claim 11
Claim 10 recites a mathematical concept, the limitation expresses the mathematical formula in words describing the formula. under its broadest reasonable interpretation when read in light of the specification, the claim falls under the mathematical concept.
Dependent claims (2-11) fail to include any additional elements. In other words, each of the limitations/elements recited in respective dependent claims (2-11) are further part of the abstract idea as identified by the Examiner for each respective dependent claim (i.e. they are part of the abstract idea recited in each respective claim).
The Examiner has therefore determined that the elements, or combination of additional elements, do not integrate the abstract idea into a practical application. Accordingly, the claims are directed to an abstract idea.
Step 2B: The claim does not provide an inventive concept (significantly more than the abstract idea). The claim is ineligible.
The receiving step is considered insignificant extra solution activity. The limitations are mere data gathering and output using processing circuitry that is recited at a high level of generality and amount to processing input data using processing circuitry that recited at high level of generality using a generic computer. Even when considered in combination, the additional elements represent mere instructions to apply an exception and insignificant extra-solution activity, which cannot provide an inventive concept.
Dependent claims (2-11) fail to include any additional elements. In other words, each of the limitations/elements recited in respective dependent claims (2-11) are further part of the abstract idea as identified by the Examiner for each respective dependent claim (i.e. they are part of the abstract idea recited in each respective claim).
The Examiner has therefore determined that the elements, or combination of additional elements, do not integrate the abstract idea into a practical application. Accordingly, the claims are directed to an abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7, 12-13 are rejected under AIA 35 U.S.C. 103(a) as being unpatentable over Keller et al. US Patent Application Publication US 20220129755 A1 (hereinafter Keller) in view of Renfu Huang Randy. Foreign Patent Application Publication DE 112020003127 T5 (hereinafter Randy).
Regarding claim 1, Keller teaches a simplification method for neural network model, configured to simplify an original trained neural network model to a simplified trained neural network model, wherein the simplified trained neural network model comprises at most two linear operation layers, and the simplification method for neural network model comprises ([0033], [0037], [0040], [0146] wherein Keller simplifies neural network utilizing ternary and identity matrices, which are significantly less computationally expensive to create and execute when compared to fully connected layer and wherein the computational complexity of a neural network layer may be reduced from quadratic to linear complexity) calculating a first new weight of the at most two linear operation layers of the simplified trained neural network model by using a plurality of original weights of the original trained neural network model ([0040], [0163], [0217], [0261] wherein Keller calculates the weights by normalizing and updating the weights values that includes normalizing the whole network by starting to propagate the linear factors from the first layer through the network in a feed-forward fashion) and generating the simplified trained neural network model based on the first new weight ([0037], [0261] wherein Keller describes creating a neural network and scaling it based on the normalized weights).
Keller does not teach receiving the original trained neural network model.
However in analogous art of simplification method for neural network, Randy teaches receiving the original trained neural network model (Abstract, wherein Randy teaches receiving a neural network model).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Keller with Randy by incorporating the method of receiving the original trained neural network model of Randy into the method of a simplification method for neural network model, configured to simplify an original trained neural network model to a simplified trained neural network model, wherein the simplified trained neural network model comprises at most two linear operation layers of Keller for the purpose of speeding up inferences based on a neural network model using multiple computational engines performing sub-operations of a neural network operation in parallel (Randy: Page. 3, paragraph 2).
Regarding claim 2, Keller as modified by Randy teaches wherein the simplified trained neural network model is denoted as y = x@W +B1, y represents an output of the simplified trained neural network model, @ represents any linear operation of the simplified trained neural network model, x represents an input of the simplified trained neural network model, Wi represents the first new weight, and B1 represents a new bias of the simplified trained neural network model (page. 4, paragraph 1, page. 27, paragraphs 7-8, wherein Randy describes mathematical formulas that represent output, linear layer with weight matrix).
Regarding claim 3, Keller as modified by Randy teaches wherein the any linear operation @ comprises a matrix multiply-accumulate operation ([0030], [0035-0036], [0075-0076], [0103], [0126] wherein Keller incorporates matrix multiplication).
Regarding claim 4, Keller as modified by Randy teaches wherein the original trained neural network model is denoted as y = (x@wi+bi)@w2 + b2, wi and bi respectively represent an original weight and an original bias of a first linear operation layer of the original trained neural network model, w2 and b2 respectively represent an original weight and an original bias of a second linear operation layer of the original trained neural network model, and the simplification method further comprises: calculating Wi = wi@w2 to determine the first new weight W1 of the simplified trained neural network model; and calculating B1 = bi@w2+ b2 to determine the new bias B1 of the simplified trained neural network model ([0040], [0261] wherein Keller describes reducing the computational complexity of a neural network layer to linear complexity normalizing the weights of the neural network by propagating the linear factors and multiplying weights of the successive layers by the linear factor and resulting and scaling the outputs of the neural network with resulting weights), (page. 4, paragraph 1, page. 27, paragraphs 7-8, wherein Randy describes mathematical formulas that represent output, linear layer with weight matrix).
Regarding claim 5, Keller as modified by Randy teaches calculating a second new weight of the at most two linear operation layers of the simplified trained neural network model by using at least one original weight of the original trained neural network model, wherein the simplified trained neural network model is denoted as y = Wi@(x@W +B1), y represents an output of the simplified trained neural network model, @represents any linear operation of the simplified trained neural network model, Wit represents the second new weight, x represents an input of the simplified trained neural network model, W1 represents the first new weight, and B1 represents a new bias of the simplified trained neural network model; and calculating the second new weight B1 of the simplified trained neural network model by using at least one original weight and at least one original bias of the original trained neural network model ([0027], [0128], [0154], [0176], [0206] wherein Keller quantizing all weights in a matrix using input, linear factors and using mathematical formulas to get ternary matrix values and simplify the neural network), (Abstract, wherein Randy teaches receiving a neural network model that includes a tensor operation and dividing the tensor operation into sub-operations. The sub-operations include at least two sub-operations that have no data dependency between the two sub-operations. The computer-implemented method further includes assigning a first sub-operation in the two sub-operations to a first computation engine, assigning a second sub-operation in the two sub-operations to a second computation engine, and generating instructions for the first to perform the first sub-operation in parallel Calculation engine and the second sub-operation by the second calculation engine. A conclusion is then drawn based on a result of the first sub-operation, a result of the second sub-operation, or both. The first computational engine and the second computational engine reside in the same integrated circuit device or in two different integrated circuit devices).
Regarding claim 7, Keller as modified by Randy teaches receiving the original trained neural network model; converting the original trained neural network model into an original mathematical function; performing an iterative analysis operation on the original mathematical function to simplify the original mathematical function to a simplified mathematical function, wherein the simplified mathematical function has the first new weight; and converting the simplified mathematical function to the simplified trained neural network model ([0035], [0040], [0057], [0103], [0118], [0125], [0142], [0194], [0222] wherein Keller incorporates mathematical formulas to simplify the neural network by using transformation operations and generating equations defined by vertices for the purpose of normalizing a neural network).
Regarding claim 12, Keller teaches A simplification device for neural network model, comprising: a memory, storing a computer readable program; and a processor, coupled to the memory to execute the computer readable program; wherein the processor executes the computer readable program to realize the simplification method for neural network model ([0044], [0046], [0048], [0052], [0060-0061]). Claim 12 is similar in scope to claim 1 therefore the claim is rejected under similar rationale.
Regarding claim 13, Keller teaches A non-transitory storage medium, for storing a computer readable program, wherein the computer readable program is executed by a computer to realize the simplification method for neural network model ([0095], claim 16 text). Claim 13 is similar in scope to claim 1 therefore the claim is rejected under similar rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HASSAN MRABI whose telephone number is (571)272-8875. The examiner can normally be reached on Monday-Friday, 7:30am-5pm. Alt, Friday, EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HASSAN MRABI/Examiner, Art Unit 2144