DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to application 18/439,267 filed 2/12/2024.
Claims 1-20 presented for examination.
Objections to the claims
Claims 2-5 relate to a convolutional operation, wherein the convolutional operation includes implementing a coarse-grained convolution operation in the Binarized-Weight Neural Networks (BWNN) through leveraging a compute-pixel with a non-volatile weight storage at a sensor side and processing network layers with a bit-wise near-sensor in memory computing unit or a near-sensor processing unit. These features are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form. The culmination of the features cited make it non-obvious.
Allowable claims
Independent Claims 6 and 15 are allowable claims. The closest art found is Jaiswal (US 2021/0264973) that discloses [para.0003] a method for operating an integrated pixel and two-terminal NVM cell and for operating an array of such cells (e.g., to perform deep in-sensor, in-memory computing). A sensor does more than just capture data, it performs part of the computation at the point of sensing, reducing the need to move raw data to a separate processor (a Processing-In-Sensor architecture). Jaiswal fails to teach a network structure with at least four convolutional layers and one Fully Connected (FC) layer with an image input data being processed therein; a non-volatile memory (NVM) element for pre-storing a plurality of pixels from the image input data; wherein, the four convolutional layers include a 1st-layer, a 2nd layer, a 3rd layer and a 4th layer, and the 1st-layer is processed in a Binarized-Weight Neural Network (BWNN); and a Process Near Sensor (PNS) unit to perform a low bit-width coarse-grained convolution on the remaining 2nd layer, 3rd layer and 4th layer. Dependent Claims 7-14 and 16-20 are considered allowable because they further limit their respective independent claims 1, 6 and 15.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use,
on sale or otherwise available to the public before the effective filing date of the claimed
invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jaiswal et al., Pub No US 2021/0264973 (hereafter Jaiswal).
Regarding Claim 1, Jaiswal discloses a method for a Processing-In-Sensor architecture (PISA) [para.0003: Discloses performing deep in-sensor, in-memory computing (e.g., of neural networks). Thus, a PISA, the sensor does more than just capture data, it performs part of the computation at the point of sensing, reducing the need to move raw data to a separate processor; para.0036: Discloses the sensor pixel itself stores binary weight values directly in memory elements co-located with the sensing photodiode (sensor architecture).], comprising:
inputting data that represents an image into an edge Internet-of-Things (IoT) device [FIG(s).5A, 5B, & 7 and para.0036: Discloses the pixel (element 130) can include a photodiode (element 131), a reset transistor (element 132, a sense node (element 135), and an amplifying transistor (element 133). This shows that the architecture integrates a photodiode (light sensor) that converts image data into an electrical signal at the pixel level, serving as input into the device; and para.0012: Discloses a light sensing process can be performed by exposing the photodiode of the specific cell to light resulting in a second data value being output on the sense node. The sense node outputs a data value after capturing light, which demonstrates how image information is input into the system at the edge device.]; and
enabling integrated sensing and processing of a 1st layer of a Binarized-Weight Neural Networks (BWNN) [para.0026: Discloses in a write functional computing mode, a first data value (e.g., a binary weight value) can be stored in the two-terminal NVM device. Thus, binary weight values of the neural network are directly written into the memory device associated with each pixel; and FIG.1, para.0046: Discloses binary weight values can be stored in the two-terminal NVM devices (element 140) of the cells (element 101). This shows that the binary weights are stored at the cell level, enabling localized computation rather that external processing; and para.0047: Discloses the functional computing mode will be indicative of the result of a dot product computation. Thus, the result of sensing and stored binary weights is a dot product operation, which is the fundamental first-layer operation in a BWNN.] with weights stored in a non-volatile magnetic memory component [para(s).0026, 0032: Discloses the cell can specifically incorporate a two-terminal NVM device (e.g., a magnetic tunnel junction (MTJ) device. This names MTJ as an example of the two-terminal NVM used to store the binary weights.] for offering energy-efficiency and speed-up at the edge IoT device [para(s).0002, 0026: Discloses processors with hardware-implemented NN's have been developed to increase processing speed. This establishes that the invention aims to accelerate neural network computations, speed benefits; and para.0003: Discloses embodiments of an integrated pixel and a two-terminal non-volatile memory (NVM) cell and embodiments of integrated circuit (IC) structure (i.e., a processing chip) that incorporates an array of such cells for performing deep in-sensor, in-memory computing (e.g., of neural networks). Thus, by integrating sensing and computation, data movement is minimized which improves energy efficiency and supports fast inference at the edge.].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Tidwell et al., (US 2011/0015989) – Discloses providing a multiply accumulate circuit. The multiply accumulate circuit receives m one-bit neuron values from a first layer of a neural network system. The multiply accumulate circuit includes m non-volatile memory cells and m current sources (col.4, lines 50-56).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADIL OCAK whose telephone number is (571) 272-2774. The examiner can normally be reached on M-F 8:00 AM - 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nasser Goodarzi can be reached on 571-272-4195. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system; contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADIL OCAK/Primary Examiner, Art Unit 2426