DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 and 17 - 20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Guo et al (US 12,536,434, hereafter Guo).
As per claim 1, A computer-implemented method of dividing a neural network comprising one or more layers into chunks of operations executable in a hardware pass of hardware configurable to execute a neural network, the one or more layers of the neural network being divisible into one or more layer groups that comprise a sequence of layers executable in a same hardware pass of the hardware, each layer group being divisible into one or more chunks of operations executable in a hardware pass of the hardware, the one or more chunks for a layer group defined by one or more split parameters, the method comprising:
obtaining a layer group loss function that represents a performance metric associated with executing a layer group on the hardware as a function of the one or more split parameters and one or more neural network architecture parameters for the layer group (column 5 lines 56 – column 6 lines 1);
generating a neural network loss function based on the layer group loss function that represents the performance metric associated with executing the neural network on the hardware (column 7); and
selecting the split parameters for the one or more layer groups that minimize the neural network loss function under one or more constraints imposed by the hardware (column 6 lines 1 – 5).
As per claim 17, Guo discloses the method of claim 1, wherein the hardware comprises a neural network accelerator (column 7 line(s) 56; AI-specific processor indicates a neural network accelerator).
As per claim 18, Guo discloses the method of claim 1, further comprising generating a set of instructions for causing the hardware to execute the neural network in the chunks identified by the selected split parameters for the one or more layer groups (column 7 lines 22 - 38; Guo includes identifiers).
As per claim 19, Guo discloses the method of claim 1, further comprising causing the hardware to execute the neural network in the chunks identified by the selected split parameters for the one or more layer groups (column 6 lines 45 – column 7 lines 5).
Regarding claim 20, arguments analogous to those presented for claim 1 are applicable for claim 20.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2 – 8 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Diamant et al (US 12,106,400, hereafter Diamant).
As per claim 2, Guo discloses the method of claim 1.
However, Guo does not explicitly teach wherein the performance metric associated with executing a layer group on the hardware is a number of cycles to execute the layer group on the hardware.
In the same field of endeavor, Diamant teaches wherein the performance metric associated with executing a layer group on the hardware is a number of cycles to execute the layer group on the hardware (column 17 lines 24 – column 19 lines 28).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
As per claim 3, Guo discloses the method of claim 2.
However, Guo does not explicitly teach wherein the layer group loss function is a ratio of (i) a total number of operations to execute the layer group on the hardware, and (ii) a maximum attainable number of operations performed by the hardware per cycle for the layer group.
In the same field of endeavor, Diamant teaches wherein the layer group loss function is a ratio of (i) a total number of operations to execute the layer group on the hardware, and (ii) a maximum attainable number of operations performed by the hardware per cycle for the layer group (column 17 lines 24 – column 19 lines 28).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
As per claim 4, Guo discloses the method of claim 3.
However, Guo does not explicitly teach wherein the maximum attainable number of operations performed by the hardware per cycle for a layer group is dependent on whether the layer group is bandwidth bound or computation bound, and the determination of whether the layer group is bandwidth bound or computation bound is based on a roofline model (column 17 lines 24 – column 19 lines 28).
In the same field of endeavor, Diamant teaches wherein the maximum attainable number of operations performed by the hardware per cycle for a layer group is dependent on whether the layer group is bandwidth bound or computation bound, and the determination of whether the layer group is bandwidth bound or computation bound is based on a roofline model (column 17 lines 24 – column 19 lines 28).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
As per claim 5, Guo discloses the method of claim 4.
However, Guo does not explicitly teach wherein the roofline model plots operation performance of the hardware as function of a maximum attainable peak operations performed by the hardware per cycle, a peak bandwidth rate for the hardware, and arithmetic intensity for a layer group, wherein the arithmetic intensity for a layer group is a total number of operations for the layer group divided by a total number of bytes transferred into or out of the hardware for the layer group.
In the same field of endeavor, Diamant teaches wherein the roofline model plots operation performance of the hardware as function of a maximum attainable peak operations performed by the hardware per cycle, a peak bandwidth rate for the hardware, and arithmetic intensity for a layer group, wherein the arithmetic intensity for a layer group is a total number of operations for the layer group divided by a total number of bytes transferred into or out of the hardware for the layer group (column 17 lines 24 – column 19 lines 28).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
As per claim 6, Guo teaches the method of claim 3.
However, Guo does not explicitly teach wherein executing a layer group on the hardware comprises performing one or more different types of operations on an input tensor and the total number of operations to execute the layer group comprises a sum of a number of each of the one or more different types of operations to execute the layer group.
In the same field of endeavor, Diamant teaches wherein executing a layer group on the hardware comprises performing one or more different types of operations on an input tensor and the total number of operations to execute the layer group comprises a sum of a number of each of the one or more different types of operations to execute the layer group (column 17 lines 24 – column 19 lines 28).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
As per claim 7, Guo discloses the method of claim 1.
However, Guo does not explicitly teach wherein the performance metric associated with executing a layer group on the hardware is a total bandwidth to transfer data into and out of the hardware to execute the layer group.
In the same field of endeavor, Diamant teaches wherein the performance metric associated with executing a layer group on the hardware is a total bandwidth to transfer data into and out of the hardware to execute the layer group (column 2 lines 32 – column 3 lines 30).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
As per claim 8, Guo discloses the method of claim 7.
However, Guo does not explicitly teach wherein the total bandwidth to transfer data into and out of the hardware to execute a layer group is a sum of a bandwidth associated with transferring each of one or more data elements into and out of the hardware to execute the layer group.
In the same field of endeavor, Diamant teaches wherein the total bandwidth to transfer data into and out of the hardware to execute a layer group is a sum of a bandwidth associated with transferring each of one or more data elements into and out of the hardware to execute the layer group (column 2 lines 32 – column 3 lines 30).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
As per claim 12, Guo discloses the method of claim 1.
However, Guo does not explicitly teach wherein the hardware comprises one or more buffers for storing data input to and/or generated by the hardware, and the one or more constraints imposed by the hardware are based on a size of one or more of the one or more buffers.
In the same field of endeavor, Diamant teaches wherein the hardware comprises one or more buffers for storing data input to and/or generated by the hardware, and the one or more constraints imposed by the hardware are based on a size of one or more of the one or more buffers (column 2 lines 32 – column 30 lines 30).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Diamant. The advantage is improvement on the neural network architecture.
Claim(s) 9 – 11 and 13 - 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Mills et al (US 12,229,657, hereafter Mills).
As per claim 9, Guo discloses the method of claim 1.
However, Guo does not explicitly teach wherein each layer group receives one or more inputs, and the one or more split parameters for a layer group comprise at least one parameter that defines a split of one of the one or more inputs in a dimension of that input.
In the same field of endeavor, Mills teaches wherein each layer group receives one or more inputs, and the one or more split parameters for a layer group comprise at least one parameter that defines a split of one of the one or more inputs in a dimension of that input (column 11 lines 42 – 57).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Mills. The advantage is improvement on the neural network architecture.
As per claim 10, Guo discloses the method of claim 9.
However, Guo does not explicitly teach wherein the one or more split parameters for a layer group comprise at least two parameters that define a split of one of the one or more inputs in a dimension of that input, and a parameter that defines an order that the splits of the one or more inputs are processed.
In the same field of endeavor, Mills teaches wherein the one or more split parameters for a layer group comprise at least two parameters that define a split of one of the one or more inputs in a dimension of that input, and a parameter that defines an order that the splits of the one or more inputs are processed (column 11 lines 42 – 57).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Mills. The advantage is improvement on the neural network architecture.
As per claim 11, Guo discloses the method of claim 9.
However, Guo does not explicitly teach wherein executing a layer group on the hardware comprises performing one or more operations on an input tensor, and the one or more inputs comprises the input tensor.
In the same field of endeavor, Mills teaches wherein executing a layer group on the hardware comprises performing one or more operations on an input tensor, and the one or more inputs comprises the input tensor (column 11 lines 42 – 57).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Mills. The advantage is improvement on the neural network architecture.
As per claim 13, Guo discloses the method of claim 1.
However, Guo does not explicitly teach wherein each layer group is configured to receive an input tensor defined by a width, a height and a number of channels and the one or more split parameters for a layer group comprise an input interleave value that defines a number of channels of the input tensor that are stored together in an interleaved manner.
In the same field of endeavor, Mills teaches wherein each layer group is configured to receive an input tensor defined by a width, a height and a number of channels and the one or more split parameters for a layer group comprise an input interleave value that defines a number of channels of the input tensor that are stored together in an interleaved manner (column 11 lines 42 – 57 and column 17 lines 64 – column 18 lines 23).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Mills. The advantage is improvement on the neural network architecture.
As per claim 14, Guo discloses the method of claim 13.
However, Guo does not explicitly teach wherein the hardware supports one or more input interleave values for the input tensor and the one or more constraints imposed by the hardware comprises a constraint that the input interleave value is one of the one or more supported input interleave values.
In the same field of endeavor, Mills teaches wherein the hardware supports one or more input interleave values for the input tensor and the one or more constraints imposed by the hardware comprises a constraint that the input interleave value is one of the one or more supported input interleave values (column 11 lines 42 – 57 and column 17 lines 64 – column 18 lines 23).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Mills. The advantage is improvement on the neural network architecture.
As per claim 15, Guo discloses the method of claim 1.
However, Guo does not explicitly teach wherein each layer group is configured to generate an output tensor defined by a width, a height and a number of channels and the one or more split parameters for a layer group comprise an output interleave value that defines a number of channels of the output tensor that are stored together in an interleaved manner.
In the same field of endeavor, Mills teaches wherein each layer group is configured to generate an output tensor defined by a width, a height and a number of channels and the one or more split parameters for a layer group comprise an output interleave value that defines a number of channels of the output tensor that are stored together in an interleaved manner (column 11 lines 42 – 57 and column 17 lines 64 – column 18 lines 23).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Mills. The advantage is improvement on the neural network architecture.
As per claim 16, Guo discloses the method of claim 15.
However, Guo does not explicitly teach wherein the hardware supports one or more output interleave values for the output tensor and the one or more constraints imposed by the hardware comprises a constraint that the output interleave value is one of the one or more supported output interleave values.
In the same field of endeavor, Mills teaches wherein the hardware supports one or more output interleave values for the output tensor and the one or more constraints imposed by the hardware comprises a constraint that the output interleave value is one of the one or more supported output interleave values (column 11 lines 42 – 57 and column 17 lines 64 – column 18 lines 23).
Therefore, it would have been obvious for one of ordinary skill in the art at the time the invention was effectively filed to modify the invention of Guo in view of Mills. The advantage is improvement on the neural network architecture.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHIKAODILI E ANYIKIRE whose telephone number is (571)270-1445. The examiner can normally be reached 8 am - 4:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Czekaj can be reached at 571-272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHIKAODILI E ANYIKIRE/Primary Examiner, Art Unit 2487