DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of the species election requirement in the reply filed on11/13/25 is acknowledged.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 5, 7-10 and 13-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Bilodeau (US2021/0192330).
Regarding claim 1, Bilodeau discloses a device, comprising:
a plurality of optically connected networked smart pixels positioned in one or more planes (Fig. 3: See pixels Display 304 or pixels in Sensor 311; [0088-0094]: See OLED pixel arrays that could implemented as “retina-like” and individual/ sub pixels of LCD display; Fig. 7-8D: Photodiodes 703-714; LED 702; 824; Fig. 9: photodetectors 903 or inhibitory photodetectors 904);
one or more imaging sensors connected to at least a portion of the smart pixels (Fig. 3: See CPU/GPU that are connected to sensing/computing layer 311 or display 304 that are comprising plurality of photodetectors, image sensor pixels and display pixels; [0089-0090]); and wherein the device is configured to perform processing on the plane where an image is captured ([0026; 0089-0096]).
Regarding claim 2, Bilodeau discloses the device of claim 1, wherein the device is configured as a wake-up mechanism for a processor interpreting images from a camera (Abstract; [0017; 0026; 0130-0136]: See inputs 811 from a camera wherein the activation of optical output when a signal received at an input of one or more nonlinear elements rise).
Regarding claim 5, Bilodeau discloses the device of claim 1, wherein the smart pixels comprise an output configured to be conditioned by one or more received inputs (Fig. 7-8D: See output of PD703-714 in response to external input light 716, 701 or 715), wherein the smart pixels evolve over time based on the values of other pixels the smart pixel is connected to (Abstract; [0017; 0026; 0130-0136]: See the activation of optical output overtime when a signal received at an input of one or more nonlinear elements rise), wherein the evolution of the smart pixels is driven by local photocurrents and other input signals ([0030-0031; 0017-0124; 00137]: See current used to drive LEDs or output by photodiodes), and wherein a subset of the smart pixels is reserved for non-image processing tasks (Abstract; [0017; 0026; 0130-0136]: See the input of one or more nonlinear elements that determine the activation of one or more optical output; [0139; 0142]: See also inhibitory photodetectors 904).
Regarding claim 7, Bilodeau discloses a system, comprising: the device of claim 1 (See the rejection of clam 1); and
a computing system communicatively connected to the device, comprising the processor and a non-transitory computer-readable medium ([0071-0072; 0089; 0115]) with instructions stored thereon, which when executed by the processor, perform steps comprising: performing processing on the image plane where the image is captured ([0026; 0089-0096]).
Regarding claim 8, Bilodeau discloses a product comprising the device of claim 1 (See the rejection of clam 1), the product selected from the group consisting of a flat panel display, a curved display, a computer monitor, a computer, a medical monitor, a television, a billboard, a light for interior or exterior illumination and/or signaling, a heads-up display, a fully or partially transparent display, a flexible display, a rollable display, a foldable display, a stretchable display, a laser printer, a telephone, a mobile phone, a tablet, a phablet, a personal digital assistant (PDA), a wearable device, a laptop computer, a digital camera, a camcorder, a viewfinder, a micro-display, a 3-D display, a virtual reality or augmented reality display or device, a vehicle, a video wall comprising multiple displays tiled together, a theater or stadium screen, a light therapy device, a camera, an imaging device, and a sign ([0004; 0062; 0089-0091]: See display, PDAs, computers or all other display devices).
Regarding claim 9, Bilodeau discloses a method, comprising: providing the system of claim 7 (See the rejection of claim 7); and performing processing on the image plane where the image is captured ([0026; 0089-0096]).
Regarding claim 10, Bilodeau discloses the method of claim 9, wherein the processing comprises local processing of data where it is acquired before readout ([0090; 0137; 0139]).
Regarding claim 13, Bilodeau discloses a device, comprising: a plurality of optically connected networked smart pixels positioned in a multiplane configuration (Fig. 3: See pixels Display 304 or pixels in Sensor 311; [0088-0094]: See OLED pixel arrays that could implemented as “retina-like” and individual/ sub pixels of LCD display; Fig. 7-8D: Photodiodes 703-714; LED 702; 824; Fig. 9: photodetectors 903 or inhibitory photodetectors 904), wherein each of the plurality of smart pixels comprises an output configured to be conditioned by one or more received inputs (Fig. 7-8D: See output of PD703-714 in response to external input light 716, 701 or 715); and
wherein the smart pixels evolve over time based on the values of other pixels and smart nodes the smart pixel is connected to (Abstract; [0017; 0026; 0130-0136]: See the activation of optical output overtime when a signal received at an input of one or more nonlinear element rise);
wherein the evolution of the smart pixels is driven by local photocurrents ([0030-0031; 0017-0124; 00137]: See current used to drive LEDs or output by photodiodes); wherein the device is configured for image processing ([0026; 0089-0096]).
Regarding claim 14, Bilodeau discloses the device of claim 13, wherein the smart pixels are configured to implement different network topologies by formatting arbitrary input data as a spatially-resolved optical intensity image ([0106; 0126]), and wherein non-image inputs are directly provided to the smart pixels ([0139; 0142]: See inhibitory photodetectors 904).
Regarding claim 15, Bilodeau discloses the device of claim 13, wherein at least a portion of the smart pixels comprise light- emitting smart pixels configured to display an image, and wherein at least a portion of the smart pixels comprise optical emitters ([0089-0094; 0122]) and non-planar synaptic elements ([0139; 0142]: See inhibitory photodetectors 904).
Regarding claim 16, Bilodeau discloses a system, comprising: the device of claim 13 (See the rejection of claim 13); and
a computing system communicatively connected to the device ([0062; 0074; 0089), comprising a processor and a non-transitory computer-readable medium ([0089-0092]: See computing units/layers/computer or computing network or CPU/GPU/IMU/ AI-Processor in fig. 3) with instructions stored thereon, which when executed by a processor, perform steps comprising:
performing processing of physically-separated inputs on the device (See separate input light that hit the photodetectors 703-714 and further process to block 708 in Fig. 7; [0134- 0135]: See also separate input from independently wired photodetector 824 that are processed to control the LEDs; [0090; 0114-0115]: neural network process 2D data arrays);
wherein the system comprises a multiplane neural network ([0016-0033]) configured to perform stereoscopic image processing or processing of images received from multiple independent cameras ([0026; 0089-0096]).
Regarding claim 17, Bilodeau discloses the system of claim 16, further comprising a cellular neural network with smart nodes on a plane, having arbitrary neighborhoods ([0081]), and wherein the arbitrary neighborhoods include nearest neighbors or beyond nearest neighbors on a single plane ([0134- 0135]: See also separate input from independently wired photodetector 824 that are processed to control the LEDs) or on different planes ([0082]: See different types of neuron networks).
Regarding claim 18, Bilodeau discloses the system of claim 16, wherein the system is configured to perform neural processing of physically-separated inputs ([0134- 0135]: See also separate input from independently wired photodetector 824 that are processed to control the LEDs; [0090; 0114-0115]: neural network process 2D data arrays) or optical communication links ([0074]).
Regarding claim 19, Bilodeau discloses a product comprising the device of claim 13 (See the rejection of claim 13), the product selected from the group consisting of a flat panel display, a curved display, a computer monitor, a computer, a medical monitor, a television, a billboard, a light for interior or exterior illumination and/or signaling, a heads-up display, a fully or partially transparent display, a flexible display, a rollable display, a foldable display, a stretchable display, a laser printer, a telephone, a mobile phone, a tablet, a phablet, a personal digital assistant (PDA), a wearable device, a laptop computer, a digital camera, a camcorder, a viewfinder, a micro-display, a 3-D display, a virtual reality or augmented reality display or device, a vehicle, a video wall comprising multiple displays tiled together, a theater or stadium screen, a light therapy device, a camera, an imaging device, and a sign ([0004; 0062; 0089-0091]: See display, PDAs, computers or all other display devices).
Regarding claim 20, Bilodeau discloses a processing method, comprising: providing the system of claim 16; and performing processing of physically-separated inputs ([0134- 0135]: See also separate input from independently wired photodetector 824 that are processed to control the LEDs; [0090; 0114-0115]: neural network process 2D data arrays).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Bilodeau.
Regarding claim 4, Bilodeau discloses the claimed invention except for “wherein the device is configured to perform completed reduced readout from the camera and processing when required to reduce latency by 25% to 99.9% and power consumption by 25% to 99.9%”. It would have been obvious to one of ordinary skill in the art at the time the invention was made to “reduce latency by 25% to 99.9% and power consumption by 25% to 99.9%”, since it has been held that where the general conditions of a claim are disclosed in the prior art, discovering the optimum or working ranges involves only routine skill in the art. In re Aller, 105 USPQ 233.
Regarding claim 6, Bilodeau discloses the claimed invention except for “ wherein the device is configured to reduce the amount of readout data required by 25% to 99.9% and to reduce a time that the processor utilizes to interpret the images by 50% to 99.9%”. It would have been obvious to one of ordinary skill in the art at the time the invention was made to “reduce the amount of readout data required by 25% to 99.9% and to reduce a time that the processor utilizes to interpret the images by 50% to 99.9%”, since it has been held that where the general conditions of a claim are disclosed in the prior art, discovering the optimum or working ranges involves only routine skill in the art. In re Aller, 105 USPQ 233.
Claims 3 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Bilodeau in view of Wu (US2021/0344881).
Regarding claim 3, Bilodeau fails to disclose the device of claim 2, wherein the device is configured as an event camera to determine when the processor should be energized from stand-by mode into operational mode to provide real time interpretation of the camera image.
In an analogous of art, Wu teaches if at least one pixel value changes, an event camera transmits a trigger signal to wake up the digital processing circuit which is in the power saving mode (abstract; [0066]; claim 6). Wu further teaches when the intensity variation is greater than the predefined value, the comparator 90 can generate the alarm signal utilized to awake the processor 84 and when the processor 84 is operated in the wakeup mode, a real-time image 12 captured by the sensor array 86 is directly transmitted to the processor 84 for digital processing. The processor 84 in the wakeup mode may process the real-time image 12 ([0042-0043]). In light of the teaching from Wu, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the event camera or comparator circuitry to wake a processor. The modification thus provide a means for waking up a processor in power saving mode to process real-time image (Wu: abstract; [0066; 0042-0043]).
Regarding claim 12, Bilodeau discloses a low power video system comprising:
an edge processor ([0089]: Bilodeau teaches The sensor data may then be relayed to an image processing algorithm 306, a motion tracker, edge detector, or the like, which extracts visual or positional data from the raw data received from the sensors an edge detector or the like. Therefore, the CPU/ AI Processor/ IMU in Fig. 3 is also an edge processor for process raw data from the edge detector); and
a camera comprising an in-plane neural network processor ([0090-0103), configured to perform real time preprocessing of the video signal ([0114]) to select when the edge processor is energized; wherein the edge processor is in stand-by mode when not selected by the preprocessor.
However, Bilodeau fails to disclose “processor configured to perform real time preprocessing of the video signal to select when the edge processor is energized; wherein the edge processor is in stand-by mode when not selected by the preprocessor”.
In an analogous of art, Wu teaches if at least one pixel value changes, an event camera transmits a trigger signal to wake up the digital processing circuit which is in the power saving mode (abstract; [0066]; claim 6). Wu further teaches when the intensity variation is greater than the predefined value, the comparator 90 can generate the alarm signal utilized to awake the processor 84 and when the processor 84 is operated in the wakeup mode, a real-time image 12 captured by the sensor array 86 is directly transmitted to the processor 84 for digital processing. The processor 84 in the wakeup mode may process the real-time image 12 ([0042-0043]). In light of the teaching from Wu, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the event camera or comparator circuitry to wake a processor. The modification thus provide a means for waking up a processor in power saving mode to process real-time image (Wu: abstract; [0066; 0042-0043]).
Bilodeau discloses the claimed invention except for “an in-plane neural network processor configured to provide a fast response time and low latency signal within 1 ps to 10 ms”. It would have been obvious to one of ordinary skill in the art at the time the invention was made to “to provide a fast response time and low latency signal within 1 ps to 10 ms”, since it has been held that where the general conditions of a claim are disclosed in the prior art, discovering the optimum or working ranges involves only routine skill in the art. In re Aller, 105 USPQ 233.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Bilodeau in view of Godlieb (US2018/0111278).
Regarding claim 11, Bilodeau fails to explicitly disclose the method of claim 9, further comprising at least one of: offloading general processing to an external processor when a specific signature is detected; and offloading partially processed data to reduce later processing steps the amount of data to transfer.
In an analogous of art, Godlieb teaches an internal signal processing module 206 for detecting relevant audio signals and then offloading some or all of the signal processing described below to an external entity such as an external server (comprising one or more server units at one or more geographical sites) which returns the result to the signal processor 206 on the user terminal 200 ([0046]). In light of the teaching from Godlieb, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to detect relevant audio signal. The modification thus provide a means for detecting the relevant audio signals and then offloading some or all of the signal processing described below to an external entity such as an external server (Godlieb: [0046]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUNG H LAM whose telephone number is (571)272-7367. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TWYLER HASKINS can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUNG H LAM/Primary Examiner, Art Unit 2639 03/06/26