Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the following communication: Non-Provisional Application filed May 8, 2024.
Claims 1-15 are pending in the case. Claim 1 is independent claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 7-15 are rejected under 35 U.S.C. 103 as being unpatentable over Syed et al. (hereinafter Syed) U.S. Patent No. 11,988,555 in view of Beerel et al. (hereinafter Beerel) U.S. Patent Publication No. 2024/0205563.
With respect to independent claim 1, Syed teaches an image sensor comprising: an array of light sensing elements arranged in logical rows and columns each providing an electrical sensor output indicating a pixel value of an image (see e.g., col. 3 Lines 40-55 – “plurality of sensor units may be arranged in a cross-bar array having row and column lines for addressing respective rows and columns of sensor units to obtain the output signal from each unit. This offers a highly efficient system architecture for various types of sensor apparatus, such as image sensors or tactile sensors, where sensor units are spatially distributed. The sensor apparatus can include a controller for controlling addressing of sensor units via the row and column lines and for programming memory elements of the sensor units.”); a set of multipliers associated with each light sensor receiving the electrical sensor output (see e.g., Fig. 1 col. 4 Lines 15-40 – “In this example, the sensor device 2 comprise a photosensor, here a photodiode, which generates an electrical signal in response to incident light. The CSU also includes a programmable non-volatile memory element 3 which is operable in the circuit as a load resistor R.sub.L for the sensor device 2.”) and a weight value to provide a weighted pixel output (see e.g., Fig. 9 claim 19 and col. 7 Lines 1-16 – “perform multiplications for the sensor output at pixel position A in an image slice, the CSU switches between cells storing the four kernel weights a.sub.11, a.sub.12, a.sub.21 and a.sub.22 as indicated by the dashed rectangles in the figure. Similarly, switching between cells storing weights (b.sub.11 to b.sub.22), (c.sub.11 to c.sub.22), and (d.sub.11 to d.sub.22) performs the multiplications for pixel positions B, C and D respectively.”); a switching network connecting individual weight elements selectively to different light sensing elements to share weight values with the multipliers of light sensing elements (see e.g., col. 3 Lines 25- col. 4 line 15 – “The memory elements may be selectively connectable in the circuit, with the unit including switching circuitry for connecting a selected memory element in the circuit in response to a control signal.”).
Syed does not expressly show a weight generator having multiple weight elements outputting different weight values. However, Beerel teaches similar feature (see e.g., Para [174]-[178] – “The integrated circuit of embodiment 11, wherein the set of weighting transistors comprises a set of weighting transistors each configured to apply a weighting based on a stored weighting value … The integrated circuit of embodiment 20, wherein the stored weighting value is fixed …The integrated circuit of embodiment 20, wherein the stored weighting value is configured to be changed … The integrated circuit of embodiment 20, wherein the stored weighting value corresponds to a weighting value for a layer of a machine learning model.”). Both Syed and Beerel are directed to image sensor pixel data processing methods. Accordingly, it would have been obvious to the skilled artisan before the effective filing date of the claimed invention having Syed and Beerel in front of them to modify the system of Syed to include the above feature. The motivation to combine Syed and Beerel comes from Beerel. Beerel discloses the motivation to employ the weighting architecture to improve sensor pixel data processing performance (see e.g., Fig. 6 and Para [30][56]). This motivation for combination also applies to the remaining claims which depend on this combination.
With respect to dependent claim 3, the modified Syed teaches the light sensing elements of each column share a shared weight value on a single conductor connected to the switching network (see e.g., Summary and col. 7 Lines 35-60 – “The plurality of sensor units may be arranged in a cross-bar array having row and column lines for addressing respective rows and columns of sensor units to obtain the output signal from each unit.”).
With respect to dependent claim 7, the modified Syed teaches the light sensing elements of each column share a common output conductor for their different weight values, the column including a summing junction attached to the common output conductor for summing the different weight values (see e.g., Fig. 10 col. 7 Lines 15-55 – “Signals on the column lines are then accumulated to provide results of multiply-accumulate steps of the convolution operation. In this way, MAC/dot product operations can be performed by addressing multiple CSUs, switched to the required weights, in parallel. Multiple MAC operations can also be performed in parallel in some implementations.”).
With respect to dependent claim 8, the modified Syed teaches the light sensing elements of a column receive individual row activation signals to switch the corresponding weighted pixel outputs to a common conductor so that the summing junction adds together weighted pixel outputs only for activated light sensing elements (see e.g., Fig. 7 col. 5 Lines 40-65 and col. 6 lines 1-34).
With respect to dependent claim 9, the modified Syed teaches a neural network receiving the summed weighted pixel outputs as outputs of a first convolution layer of a convolution neural network (see e.g., col. 3 lines 40-65 and col. 7 Lines 20-65 – “The controller of such apparatus is operable to program memory elements of each sensor unit to programmed states corresponding to respective kernel weights of a CNN layer, and to control addressing of sensor units such that output signals of the sensor units provide results of a convolution operation in the CNN layer.”).
With respect to dependent claim 10, the modified Syed teaches each weight element comprises a set of fixed current sources switchably connected in parallel to provide different weight values (Beerel Para [5][57]-[62][73]- the fixed transistors correspond to the recited “fixed current sources.” Also see e.g., col. 4 Lines 1-40 – “This allows in-sensor computation by switching between load resistors in sensor units to switch-in kernel weights required for convolution, and MAC operations can be performed by addressing units in parallel. Multiple MAC operations may also be performed in parallel in some architectures. The sensor units may also include, for each memory element, a capacitor for storing a charge dependent on the electrical signal from the sensor device and the programmed state of that memory element, the capacitor being selectively connectable to the output to provide the output signal of the sensor unit. This allows temporary storage of output signals and greater flexibility for processing of signals in a required order for convolution.”).
With respect to dependent claim 11, the modified Syed teaches the fixed current sources are transistors with different conductive areas determining a contribution of the transistor to the different weight values (see e.g., Beerel Para [154]-[168] – “the set of weighting elements comprises a set of weighting transistors … The integrated circuit of embodiment 11, wherein the set of weighting transistors comprises transistors of varying widths. … The integrated circuit of embodiment 11, wherein the set of weighting transistors comprises transistors of varying W/L”).
With respect to dependent claim 12, the modified Syed teaches the switching network is a crossbar switch selectively connecting each input to any of each output according to a connection signal (see e.g., col. 3 Lines 40-65 – “The plurality of sensor units may be arranged in a cross-bar array having row and column lines for addressing respective rows and columns of sensor units to obtain the output signal from each unit. ”).
With respect to dependent claim 13, the modified Syed teaches the crossbar switch provides a number of inputs and outputs within the range of 2 to 7 (see e.g., Claim 18 – “columns of sub-arrays each comprising a plurality m of sensor units” Range of 2 to 7 is merely a design choice).
With respect to dependent claim 14, the modified Syed teaches the weight elements provide continuously variable and programmable weights (see e.g., col. 5 Lines 15-35 – “CSU's embodying the invention may include a plurality of programmable non-volatile memory elements each operable in the sensor circuit as a load resistor for the sensor device. FIG. 5 shows one such implementation, again for the basic circuit configuration of FIG. 1.”).
With respect to dependent claim 15, the modified Syed teaches the weight elements are non-volatile memory (NVM) devices selected from the group consisting of a of a magnetic tunnel junction (MTJ) device, a phase change memory (PCM) device, an FeFET transistor, and CTT transistor, a FLASH transistor and a memrister (see e.g., col. 5 Lines 15-35 – “CSU's embodying the invention may include a plurality of programmable non-volatile memory elements each operable in the sensor circuit as a load resistor for the sensor device. FIG. 5 shows one such implementation, again for the basic circuit configuration of FIG. 1.” The above list is well-known in the art.).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Syed in view of Beerel and further in view of Jaiswal et al. (hereinafter Jaiswal) U.S. Patent Publication No. 2024/0381006.
With respect to dependent claim 2, Syed does not expressly show the weight generator is displaced from area between the multiple light sensing elements. However, Jaiswal teaches similar feature (see e.g., Para [69][70][97]-[99] – “the at least one of eDRAM circuitry, ROC, ADS, ASIC, and the combination thereof is monolithically or heterogeneously integrated with the pixel.”). Both Syed and Jaiswal are directed to in-pixel computing. Accordingly, it would have been obvious to the skilled artisan before the effective filing date of the claimed invention having Syed and Jaiswal in front of them to modify the system of Syed to include the above feature. The motivation to combine Syed and Jaiswal comes from Jaiswal. Jaiswal discloses the motivation to use the design placement to preserve pixel area (Para [69][70][97]-[99]).
Claims 4, 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Syed in view of Beerel and further in view of Kaiser et al. (hereinafter Kaiser) “Technology-Circuit-Algorithm Tri-Design for Processing-in-Pixel-in-Memory (P2M)” 2023.
With respect to dependent claim 4, Syed does not expressly show a stride controller selectively activating the switching network to apply the different weight values at a predetermined stride periodicity along a column without changing the different weight values. However, Kaiser teaches similar feature (see e.g., Page 615-616 – overlapping and non-overlapping strides are supported by different weight transistor sets). Both Syed and Kaiser are directed to in-pixel convolution. Accordingly, it would have been obvious to the skilled artisan before the effective filing date of the claimed invention having Syed and Kaiser in front of them to modify the system of Syed to include the above feature. The motivation to combine Syed and Kaiser comes from Kaiser. Kaiser discloses the motivation to avoid bottleneck processing issues so that performance can be improved (see e.g., Page 613).
With respect to dependent claim 5, the modified Syed teaches the stride controller provides an input for receiving a stride input controlling the predetermined stride amount (see e.g., Page 615-616 – each stride can be mapped to a different weight transistor).
With respect to dependent claim 6, the modified Syed teaches the stride controller operates to control the predetermined stride amount from between 2 and 5 (see e.g., Page 615-616).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co. v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert. denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir. 2005); Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEIYONG WENG whose telephone number is (571)270-1660. The examiner can normally be reached on Mon.-Fri. 8 am to 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Matthew Ell, can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://portal.uspto.gov/external/portal. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/PEI YONG WENG/Primary Examiner, Art Unit 2141
Prosecution Insights