Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Specification
2. The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims are rejected under 35 U.S.C. 103 as being unpatentable over Molchanov et al (US 2017/0206405, herein Molchanov) in view of Miniskar et al (US 2020/0257972, herein Miniskar).
Regarding claim 1, teaches a processor comprising:
a plurality of processing elements, wherein the processor is configured to execute, by using one or more first processing elements among the plurality of processing elements, operations of a first layer of a neural network including a plurality of layers (Figs 2D & 3, [0041], [0052], [0069], processing elements to implement layers of neural network),
the processor is configured to execute, by using one or more second processing elements among the plurality of processing elements, operations of a second layer of the neural network ([0041], [0052], [0069], multiple layers implemented by processor),
the second layer is at a later processing stage than the first layer (Fig 2A, [0041], [0105], different layers in subsequent stages).
Molchanov fails to teach wherein the one or more first processing elements and the one or more second processing elements do not entirely overlap.
Miniskar teaches a processor comprising a plurality of processing elements to implement layers of a neural network ([0007], [0050], processor to implement layers of neural network) wherein the one or more first processing elements and the one or more second processing elements do not entirely overlap ([0055], [0062-0065], memory buffer may overlap some inputs of neural network layers).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Molchanov and Miniskar to utilize partially overlapping processing elements. While Molchanov does disclose that the pooling layer accounts for non-overlapping elements in an input tensor (Molchanov [0041]), Molchanov does not explicitly state that the processing elements themselves may partially overlap. However, as the use of shared or overlapping components to process large quantities of data is a routine and conventional aspect of the microprocessor art, utilizing overlapping memory regions to reuse certain values in the neural network would merely entail a simple substitution of known prior art elements to achieve predictable results, and thus would have been obvious to one of ordinary skill in the art.
Regarding claim 2, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, further comprising a plurality of memories, wherein the processor is configured to use one or more first memories among the plurality of memories for the execution of the operations of the first layer, the processor is configured to use one or more second memories among the plurality of memories for the execution of the operations of the second layer, and the one or more first memories and the one or more second memories do not entirely overlap (Molchanov Figs 3, 5, shared and partitioned memories & Miniskar [0006], [0055], partially overlapping memory buffers).
Regarding claim 3, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 2, wherein the one or more first memories and the one or more second memories each store at least one of an operand or an instruction (Molchanov [0074], [0091], [0094], storage for instructions & operands).
Regarding claim 4, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 2, wherein the processor is configured to transmit, during the operations of the first layer, at least one of a weight, an operand or an instruction to the one or more second memories (Molchanov [0048], [0094], use of weights & operands in neural network operations).
Regarding claim 5, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, wherein the one or more first processing elements includes at least one processing element that is not included in the one or more second processing elements, and the one or more second processing elements includes at least one processing element that is not included in the one or more first processing elements (Molchanov Figs 2D, 3-5, [0070], [0079-0080], multiple distinct layers processed by different hardware).
Regarding claim 6, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, wherein the plurality of processing elements include at least one processing element that is included in neither the one or more first processing elements nor the one or more second processing elements (Molchanov Figs 3-5, [0081], [0089], additional processing units beyond a first and second PPU).
Regarding claim 7, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 6, wherein the at least one processing element that is included in neither the one or more first processing elements nor the one or more second processing elements is used for executing operations of a third layer of the neural network (Molchanov Figs 2D, 5, [0070], additional layers of neural network operations).
Regarding claim 8, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, wherein the processor is configured to execute, by using one or more third processing elements among the plurality of processing elements, operations of a third layer of the neural network, the one or more first processing elements, the one or more second processing elements, and the one or more third processing elements do not entirely overlap each other, and the third layer being at a later processing stage than the second layer (Molchanov Fig 2D, [0047-0048], third neural network layer & Miniskar [0006-0008], six or more processing layers implemented).
Regarding claim 9, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, wherein the one or more first processing elements are wholly different from the one or more second processing elements (Molchanov Figs 2D, 3-5, [0070], [0079-0080], multiple distinct layers processed by different hardware).
Regarding claim 10, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, wherein the processor is configured to reconfigure the plurality of processing elements for operations of each layer of the neural network (Miniskar [0054-0056], reconfiguration of memory elements according to overlap and reusage).
Regarding claim 11, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, further comprising a plurality of information transfer circuits, wherein the processor is configured to use one or more first information circuits among the plurality of information transfer circuits for the execution of the operations of the first layer, the processor is configured to use one or more second information transfer circuits among the plurality of information transfer circuits for the execution of the operations of the second layer, the one or more first information transfer circuits and the one or more second information transfer circuits do not entirely overlap (Molchanov Figs 3-5, [0087-0089], memory management, partitioning, and crossbars for transferring information between processing elements & other sections of memory & Miniskar [0055], configuring memory to partially overlap).
Regarding claim 12, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 11, wherein each of the plurality of processing elements is connected to at least one output memory, wherein the at least one output memory stores at least one operation result of at least one processing element among the plurality of processing elements, and wherein the at least one operation result stored in the at least one output memory is transmitted to at least one information transfer circuits among the plurality of information transfer circuits (Molchanov Figs 3-5, [0071], input/output unit & [0078], writing results to memory via transfer circuits).
Regarding claim 13, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, wherein the plurality of processing elements are arranged in a matrix of two or more rows and two or more columns (Molchanov Figs 2D, 5, grid of processing elements).
Regarding claim 14, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 10, wherein the plurality of processing elements are reconfigured to include neighboring processing elements for the operations of each layer of the neural network (Molchanov Figs 2D, 5, grid of processing elements neighboring one another & Miniskar [0054-0056], reconfiguration of memory elements according to overlap and reusage).
Regarding claim 15, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, wherein the plurality of layers of the neural network include at least one of a convolution layer, a pooling layer, an activating layer, or a fully-connected layer (Molchanov [0041], convolution, activation, pooling, and fully-connected layers).
Regarding claim 16, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 1, further comprising a plurality of memories, wherein the processor is configured to use a first resource for the execution of the operations of the first layer, the first resource including one or more first memories among the plurality of memories and the one or more first processing elements, wherein the processor is configured to use a second resource for the execution of the operations of the second layer, the second resource including one or more second memories among the plurality of memories and the one or more second processing elements, and wherein the one or more first memories and the one or more second memories do not entirely overlap (Molchanov Figs 3, 5, shared and partitioned memories & Miniskar [0006], [0055], partially overlapping memory buffers).
Regarding claim 17, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 16, herein the first resource and the second resource are constructed based on operations for the neural network, and a size of the first resource is different from a size of the second resource (Miniskar [0054-0056], configuration of memory elements according to data overlap and reusage).
Regarding claim 18, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 16, wherein the processor is configured to transmit, during the execution of the operations of the second layer, operation results of the first layer to a third resource used in operations of a third layer that is at a later processing stage than the second layer in the neural network, the third resource including one or more third memories among the plurality of memories and one or more third processing elements among the plurality of processing elements (Molchanov Figs 3-5, [0071], input/output unit & [0078], writing results to memory via transfer circuits).
Regarding claim 19, the combination of Molchanov and Miniskar teaches the processor as claimed in claim 16, wherein the operations of the first layer and the operations of the second layer include convolutional operations (Molchanov [0041], convolution).
Claims 20-29 refer to a method embodiment of the processor embodiment of claims 1, 2, 4, 8, 10, 11, 12, 14, 16, and 18, respectively. Therefore, the above rejections for claims 1, 2, 4, 8, 10, 11, 12, 14, 16, and 18 are applicable to claims 20-29, respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sur (US 11,645,534) discloses a processor for overlapping execution of layers of a neural network.
Jiang (US 2018/0211130) discloses a processor with fully connected neural network layers which process a partially overlapping feature map.
Young (US 9,842,293) discloses a processor that reuses weight inputs for different layers of neural network operations.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J METZGER whose telephone number is (571)272-3105. The examiner can normally be reached Monday-Friday 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jyoti Mehta can be reached at 571-270-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J METZGER/ Primary Examiner, Art Unit 2183
Prosecution Insights