Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not
identically disclosed as set forth in section 102, if the differences between the claimed invention and the
prior art are such that the claimed invention as a whole would have been obvious before the effective filing
date of the claimed invention to a person having ordinary skill in the art to which the claimed invention
pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 1-2, 13-14, 26-27, 33, & 35 are rejected under 35 U.S.C. 103 as being unpatentable over Henderson et al (US 20200158836 A1), hereinafter Henderson, in view of Pacala et al (US 20200341144 A1), hereinafter Pacala.
2. Regarding Claims 1 & 26:
Henderson teaches a Lidar system and method, ([0084]: Embodiments of the present disclosure have been described herein with reference to light-based ranging measurement systems (such as lidar) and related methods of operation).
Henderson teaches a Light Detection and Ranging (LIDAR) circuit, ([Abstract]: A Light Detection And Ranging (LIDAR) measurement circuit). Henderson teaches at least one control circuit configured to: execute first memory storage operations to store data indicated by detection signals received from one or more photodetector elements in the first memory during a time between pulses of an emitter signal output from a LIDAR emitter element, ([0039]: FIGS. 1A and 1B illustrate example ToF measurement systems 100a and 100b and related components in LIDAR applications in accordance with some embodiments of the present invention. As shown in FIG. 1A, the system 100a includes a control circuit 105). Henderson further teaches, ([0051]: Referring to FIG. 2, the accumulator circuits 201, 202 (also referred to herein as accumulators) may be registers of a processor or control circuit 205 that are operable to accumulate and integrate detection events (e.g., single-photon detection events, in some embodiments correlated within a time window) indicated by the detection signal(s) output from a single-photon detector array 110 (described with reference to a SPAD array) in response to detection of incident photons during or over a detection window. The example of FIG. 2 includes first and second accumulators 201 and 202, where the first accumulator 201 functions as a time or temporal accumulator or time integrator that is configured to track detection of photons incident upon the detector array 110 at respective times over the strobe window, and the second accumulator 202 functions as a photon counter to collect a photon count corresponding to a total number of photons that were detected during the strobe window. More particularly, the first accumulator 201 is configured to integrate the number of detection events that have been identified as of respective time intervals or time bins (i.e., the timesteps k) of the detection window as a representation of the respective times of arrivals of the detected photons, and the second accumulator 202 functions as a photon counter that is configured to count the total number detection events that were identified over the detection window). Henderson continues to teach, ([0048]: FIG. 4 is a diagram illustrating relationships between image frames, subframes, laser cycles, and time gates (also referred to herein as strobe windows or detection windows) that may be used in some lidar systems. The time gates/strobe windows define durations of time (corresponding to respective portions of the time between laser pulses in a laser cycle) during which one or more SPADs of the array 110 are active or enabled to detect incident photons. The strobe windows or detection windows may thus define respective measurement intervals, which may be further divided into respective time intervals or timesteps k as described herein).
Henderson does not teach, a non-transitory memory device comprising a first memory and a second memory; and execute second memory storage operations to include previous data indicated by previous detection signals received from the one or more photodetector elements, which was stored in the first memory, in respective memory bins of the second memory, wherein the at least one control circuit is configured to execute the first and second memory storage operations at least partially concurrently.
However, Pacala teaches a Lidar system for use in a vehicle. Pacala further teaches, ([0154]: Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C #, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer readable medium for storage and/or transmission. A suitable non-transitory computer readable medium can include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk), flash memory, and the like. The computer readable medium may be any combination of such storage or transmission devices). Pacala goes on to teach, ([0102]: The ALU 804 may also perform a second aggregation operation that adds the total signal count to an existing value in a memory location of the memory 806. Recall from FIG. 7 that with each shot, a new total signal count may be added to an existing value in the corresponding time bin of the memory 806. In this manner, the histogram 818 can be gradually constructed in the memory 806 over a number of shots. When the total signal count is generated by the ALU 814, a current value 820 of a corresponding memory location for that time bin can be retrieved from the memory 806. The current value 820 can be provided as an operand to the ALU 804, which can be combined with the total signal count from the set of signals 816. In some embodiments, the ALU 804 can be composed of a first stage and a second stage, where the first stage calculates the total signal count from the photosensor 802, and the second stage combines the total signal count with the current value 820 from that time bin's memory location in the memory 806. In some embodiments, the aggregation of the set of signals 816 and the aggregation of total signal count and the current value 820 may be carried out as a single operation. Thus, even though these two operations may functionally be described as separate “aggregations,” they may in fact be performed together using a combination of parallel and sequential circuitry in the ALU 804). Pacala continues to teach, ([0127]-[0129]: [0127]: FIG. 15 illustrates shared aggregation circuitry between the two data paths, according to some embodiments. A single ALU 1501 can implement a first aggregation stage 1502 that sums the total signal count from the photodetectors of the photosensor 802. This first stage 1502 can generate an intermediate result 1504 that is then passed to each of the two data paths. For example, the intermediate result 1504 can be passed to the second stage 1306 of the histogram data path. This second stage 1306 can add the common intermediate result 1504 to the current value 820 for a current time bin in the memory 806 representing the histogram. The updated value can then be stored back in the current time bin in the memory 806. Similarly, the common intermediate result 1504 can be passed to the second stage 1406 of the integration data path. This second stage 1406 can add the intermediate result 1504 to the current value 1010 of the integration register 1004. This updated value can then be stored back in the integration register 1004. [0128]: When the first stage 1502 of the ALU 1501 is shared between both data paths, a periodic signal that clocks the ALU 1501 may also be shared between both data paths. Consequently, the size of the time bins in the integration data path and the size of the time bins in the histogram data path may be the same. However, even though the time bins may be the same size, the overall integration cycle in the integration data path may run continuously and still be independently controlled when compared to the histogram data path. For example, the memory 806 in the histogram data path may only store and update memory locations during pulse train measurements in a shot, while the integration register 1004 may continuously update its current value 1010 regardless of whether or not a shot is active in the histogram data path. [0129]: The example of FIG. 15 illustrates how the different aggregation functions used by both data paths can be shared or separated between the two data paths. Therefore, any recitation of an arithmetic logic circuit may encompass any aggregation function in either data path whether implemented as separate stages or as a single stage).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson with Pacala to include a non-transitory memory as well as second memory storage operations to include previous data indicated by previous detection signals received from the one or more photodetector elements, which was stored in the first memory, in respective memory bins of the second memory, wherein the at least one control circuit is configured to execute the first and second memory storage operations at least partially concurrently, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson with Pacala since, non-transitory memory provides reliability for long term storage and high storage capacity. Regarding the first and second memory, such configurations provide improved SNR, tolerance to background or ambient light, increased sensitivity for weak signals, faster processing, and improved dynamic range.
3. Regarding Claims 2 & 27:
Henderson teaches execution of the second memory storage operations comprises performing read, modify, and write operations to include the previous data in the respective memory bins responsive to a common precharge operation, ([0051]: Referring to FIG. 2, the accumulator circuits 201, 202 (also referred to herein as accumulators) may be registers of a processor or control circuit 205). Henderson further teaches, ([0063]: At the end of the strobe window, the total number of correlated detection events CC<3:0> is also summed by the photon counter 202 (illustrated by way of example as a 18-bit accumulator), which output signal S<17:0> to the readout circuit 209. The signal S<17:0> provides a count value indicating the total number of correlated detection events that were identified by the end of the detection window. The readout circuit 209 outputs a data signal Data <44:0> (e.g., at the end of each subframe). As discussed below with reference to FIGS. 4A and 4B, a subframe may include data collected for a respective strobe window Strobe<i> that is repeated for multiple (e.g., thousands of) laser cycles). The accumulators 201 and/or 202 may be reset by signal FrameRst at the end of a subframe).
4. Regarding Claim 13:
Henderson teaches the at least one control circuit, ([0051]: Referring to FIG. 2, the accumulator circuits 201, 202 (also referred to herein as accumulators) may be registers of a processor or control circuit 205). Henderson teaches executing the read, modify, and write operations for the respective memory banks, responsive to the common precharge operation, ([0063]: At the end of the strobe window, the total number of correlated detection events CC<3:0> is also summed by the photon counter 202 (illustrated by way of example as a 18-bit accumulator), which output signal S<17:0> to the readout circuit 209. The signal S<17:0> provides a count value indicating the total number of correlated detection events that were identified by the end of the detection window. The readout circuit 209 outputs a data signal Data <44:0> (e.g., at the end of each subframe). As discussed below with reference to FIGS. 4A and 4B, a subframe may include data collected for a respective strobe window Strobe<i> that is repeated for multiple (e.g., thousands of) laser cycles). The accumulators 201 and/or 202 may be reset by signal FrameRst at the end of a subframe).
Henderson does not teach respective logic circuits configured to execute the read, modify, and write operations for the respective memory banks in parallel.
However Pacala teaches, ([0102]: The ALU 804 may also perform a second aggregation operation that adds the total signal count to an existing value in a memory location of the memory 806. Recall from FIG. 7 that with each shot, a new total signal count may be added to an existing value in the corresponding time bin of the memory 806. In this manner, the histogram 818 can be gradually constructed in the memory 806 over a number of shots. When the total signal count is generated by the ALU 814, a current value 820 of a corresponding memory location for that time bin can be retrieved from the memory 806. The current value 820 can be provided as an operand to the ALU 804, which can be combined with the total signal count from the set of signals 816. In some embodiments, the ALU 804 can be composed of a first stage and a second stage, where the first stage calculates the total signal count from the photosensor 802, and the second stage combines the total signal count with the current value 820 from that time bin's memory location in the memory 806. In some embodiments, the aggregation of the set of signals 816 and the aggregation of total signal count and the current value 820 may be carried out as a single operation. Thus, even though these two operations may functionally be described as separate “aggregations,” they may in fact be performed together using a combination of parallel and sequential circuitry in the ALU 804). Pacala further teaches, ([0127]-[0129]: [0127]: FIG. 15 illustrates shared aggregation circuitry between the two data paths, according to some embodiments. A single ALU 1501 can implement a first aggregation stage 1502 that sums the total signal count from the photodetectors of the photosensor 802. This first stage 1502 can generate an intermediate result 1504 that is then passed to each of the two data paths. For example, the intermediate result 1504 can be passed to the second stage 1306 of the histogram data path. This second stage 1306 can add the common intermediate result 1504 to the current value 820 for a current time bin in the memory 806 representing the histogram. The updated value can then be stored back in the current time bin in the memory 806. Similarly, the common intermediate result 1504 can be passed to the second stage 1406 of the integration data path. This second stage 1406 can add the intermediate result 1504 to the current value 1010 of the integration register 1004. This updated value can then be stored back in the integration register 1004. [0128]: When the first stage 1502 of the ALU 1501 is shared between both data paths, a periodic signal that clocks the ALU 1501 may also be shared between both data paths. Consequently, the size of the time bins in the integration data path and the size of the time bins in the histogram data path may be the same. However, even though the time bins may be the same size, the overall integration cycle in the integration data path may run continuously and still be independently controlled when compared to the histogram data path. For example, the memory 806 in the histogram data path may only store and update memory locations during pulse train measurements in a shot, while the integration register 1004 may continuously update its current value 1010 regardless of whether or not a shot is active in the histogram data path. [0129]: The example of FIG. 15 illustrates how the different aggregation functions used by both data paths can be shared or separated between the two data paths. Therefore, any recitation of an arithmetic logic circuit may encompass any aggregation function in either data path whether implemented as separate stages or as a single stage).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson with Pacala to include logic units (ALU) and memory operations (read write modify) in parallel, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson with Pacala since, such configurations can maximize throughput, reduce memory latency, perform real-time data compression, and provide high energy efficiency, allowing a Lidar system to efficiently process large streams of time of flight data in real-time.
5. Regarding Claim 14:
Henderson teaches the respective memory bins of the second memory comprise histogram data for an imaging distance subrange comprising up to an entirety of a distance range corresponding to the time between the pulses of the emitter signal, ([0074]: Further examples illustrating advantages provided by center of mass calculation techniques in accordance with embodiments of the present invention (as compared to some conventional timestamp summation and averaging techniques that require time-to-digital conversion operations) are described with reference to FIG. 8. FIG. 8 illustrates a histogram 800 of detection events (shown by arrows ↓) over a detection window according to embodiments of the present invention. In some embodiments, the detection window (including the timesteps k) may be one of a plurality of strobe windows between laser pulses of a LIDAR emitter, where the strobe windows correspond to respective distance subranges of the imaging distance range defined by the duration of time between laser pulses of the emitter. In some embodiments, the detection window (including the timesteps k) may represent the entire duration of time between pulses of the emitter).
6. Regarding Claim 33:
Henderson teaches the one or more photodetector elements comprise single-photon avalanche detectors (SPADs), ([0004]: In some applications, the sensing of the reflected radiation may be performed using an array of photodetectors, including single-photon detectors, such as a Single Photon Avalanche Diode (SPAD) array). Henderson teaches, the data and/or the previous data comprises photon counts indicated by the detection signals corresponding to portions of the imaging distance subrange, ([0074]: Further examples illustrating advantages provided by center of mass calculation techniques in accordance with embodiments of the present invention (as compared to some conventional timestamp summation and averaging techniques that require time-to-digital conversion operations) are described with reference to FIG. 8. FIG. 8 illustrates a histogram 800 of detection events (shown by arrows ↓) over a detection window according to embodiments of the present invention. In some embodiments, the detection window (including the timesteps k) may be one of a plurality of strobe windows between laser pulses of a LIDAR emitter, where the strobe windows correspond to respective distance subranges of the imaging distance range defined by the duration of time between laser pulses of the emitter). Henderson further teaches, ([0052]: The first and second accumulator circuits 201 and 202 provide respective memory arrays (e.g., per detector pixel), where a first memory array effectively stores a histogram of the accumulated product of the respective time bin or timestep k and number of detection events (e.g., the number of SPADs that output signals (or are ‘fired’) in response to incident photons (Σ.sub.k=0.sup.NPNS(k))), and a second memory array stores the accumulated number of counts of the detection events. A ratio of the integrated number of detection events per time interval and the number of counts per detection window (i.e., the average number of detection events per time interval) is used to calculate the center of mass.).
7. Regarding Claim 35:
Henderson teaches LIDAR system coupled to an autonomous vehicle, ([0085]: Lidar systems and arrays described herein may be applied to ADAS (Advanced Driver Assistance Systems), autonomous vehicles). Henderson does not teach the LIDAR emitter element and the one or more photodetector elements are oriented relative to an intended direction of travel of the autonomous vehicle.
However, Pacala teaches, ([0045]: In either the scanning or stationary architectures, objects within the scene can reflect portions of the light pulses that are emitted from the LIDAR light sources. One or more reflected portions then travel back to the LIDAR system and can be detected by the detector circuitry. For example, reflected portion 117 can be detected by detector circuitry 109. The detector circuitry can be disposed in the same housing as the emitters. Aspects of the scanning system and stationary system are not mutually exclusive and thus can be used in combination. For example, the individual LIDAR subsystems 103a and 103b in FIG. 1B can employ steerable emitters such as an optical phased array or whole the composite unit may rotate through mechanical means thereby scanning the entire scene in front of the LIDAR system, e.g., from field of view 119 to field of view 121). FIG. 1B shows FOVs 119 & 121 are oriented relative to an intended direction of travel in the forward direction while additional FOVs in the rear of the vehicle are provided by LIDAR subsystem 103b. See also FIG. 1A, showing a rotating LIDAR system on a vehicle roof, capable of being oriented in any potential direction of travel.
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson with Pacala to include the LIDAR emitter element and the one or more photodetector elements are oriented relative to an intended direction of travel of the autonomous vehicle, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson with Pacala since, such a configuration provides maximum range and resolution in the intended direction of travel, optimized collision avoidance, reduced blind spots, and streamlined data processing.
8. Claims 3, 10-12, 15-20, 24-25, 28, & 34 are rejected under 35 U.S.C. 103 as being unpatentable over Henderson et al (US 20200158836 A1), hereinafter Henderson, in view of Pacala et al (US 20200341144 A1), hereinafter Pacala, as applied to Claims 1, 2, 26, & 27, and further in view of Onal et al (US 20200256963 A1), hereinafter Onal.
9. Regarding Claims 3, 16, & 28:
Henderson teaches a first and second memory array, ([0052]: The first and second accumulator circuits 201 and 202 provide respective memory arrays (e.g., per detector pixel), where a first memory array effectively stores a histogram of the accumulated product of the respective time bin or timestep k and number of detection events (e.g., the number of SPADs that output signals (or are ‘fired’) in response to incident photons (Σk=0NPNS(k))), and a second memory array stores the accumulated number of counts of the detection events).
Henderson as modified by Pacala does not teach, a non-transitory memory device having temporary memory.
However, Onal teaches, ([0068]: The computer readable medium can also include non-transitory computer readable media such as computer-readable media that store data for short periods of time like register memory, processor cache, and random access memory (RAM). The computer readable media can also include non-transitory computer readable media that store program code and/or data for longer periods of time. Thus, the computer readable media may include secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, compact-disc read only memory (CD-ROM), for example. The computer readable media can also be any other volatile or non-volatile storage systems. A computer readable medium can be considered a computer readable storage medium, for example, or a tangible storage device).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson and Pacala with Onal to include non-transitory memory and temporary memory, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson and Pacala with Onal since, non-transitory memory combined with temporary memory provides data persistence, optimal energy efficiency, and faster performance due to increased read and write speeds associated with temporary memory, as evidenced by Onal ([0021]: In an example embodiment, a plurality of SPAD arrays (e.g., 16, 32, 64, or more arrays) may provide digital or digital-like signals to one or more adder stages, a buffer, and a data pipeline. The data pipeline may be coupled to a fast memory, such as an SRAM block).
Henderson as modified by Pacala does not teach, a pipeline memory as well as second memory storage operations comprise retrieving the previous data from the temporary memory and integrating the previous data in the respective memory bins of the main memory.
However, Onal teaches, ([0005]: In a first aspect, a system is provided. The system includes a plurality of macropixels. Each macropixel of the plurality of macropixels includes an array of single photon avalanche diodes (SPADs). Each SPAD is configured to provide a respective photosignal when triggered in response to detecting light from an external environment of the system. The system also includes a plurality of pipelined adders. Each pipelined adder of the plurality of pipelined adders is communicatively coupled to a respective portion of the plurality of macropixels. The system additionally includes a controller having a memory and at least one processor. The at least one processor executes instructions stored in the memory so as to carry out operations. The operations include, during a listening period, receiving, at each pipelined adder of the plurality of pipelined adders, respective photosignals from the respective portion of the plurality of macropixels. The operations also include causing each pipelined adder of the plurality of pipelined adders to provide an output based on the respective photosignals from the respective portion of the plurality of macropixels. The output includes a series of frames. Each frame of the series of frames includes an average number of SPAD devices of the respective portion of the plurality of macropixels that were triggered during a given listening period).
Onal further teaches, ([0021]: In an example embodiment, a plurality of SPAD arrays (e.g., 16, 32, 64, or more arrays) may provide digital or digital-like signals to one or more adder stages, a buffer, and a data pipeline. The data pipeline may be coupled to a fast memory, such as an SRAM block. In such scenarios, the adder stages and pipeline may provide averaged frames in a serial or parallel manner to the memory. In some embodiments, each frame may include an average number of SPAD cell elements triggered over two or more listening periods for each SPAD array. In other embodiments, each frame may include, for each SPAD array, the number of SPAD cell elements triggered during each time bin (e.g., 400 picoseconds) of the listening period (e.g., 2 microseconds)). Onal continues to teach, ([0022]: The memory may provide the averaged frames to a pulse processing block for processing. For example, the pulse processing block may be configured to process the frames to determine pulse arrival time (e.g., a time stamp to determine range), intensity, a first moment (e.g., a center of mass), and/or a second moment, which may help determine “mixed” pixels (where a light pulse hits the edge of an object). In some implementations, the pulse processing block may include approximately 200,000 resistor-transistor logic (RTL) gates, but different numbers and types of digital logic gates are possible and contemplated herein).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson and Pacala with Onal to include a pipeline memory as well as second memory storage operations comprise retrieving the previous data from the temporary memory and integrating the previous data in the respective memory bins of the main memory, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson and Pacala with Onal since, such configurations provide increased throughput, higher clock frequencies, and optimized hardware usage. In addition to, eliminating or reducing memory bottlenecks.
10. Regarding Claim 10:
Henderson teaches the first memory storage operations comprise sampling the data from the detection signals at a predetermined sampling rate, ([0050]: Many repetitions of Strobe<i> are aggregated (e.g., in the pixel) to define a sub-frame for Strobe<i>, with subframes 1 to i defining an image frame. Each sub-frame for Strobe<i> may correspond to a respective distance sub-range of the overall imaging distance range, which is defined by the frequency of the laser cycle. The strobe windows may be referred to below with reference to timesteps k, where k denotes the kth time interval in a strobe/detection window which is sampled by a clock (referred to in some examples herein with reference to clock signal FastClk). The clock may have a period of some factor (e.g., 10 times or 20 times shorter than the in some examples described herein) shorter than the strobe window time duration). Henderson further teaches, ([0056]: In some embodiments, the accumulator(s) 201, 202 may be operable responsive to output from a parallel counter 204, the correlator 203 and correlation counter 203c, and/or a gated clock 207. The parallel counter 204 is configured to provide output signal C<4:0> that indicates detection of photons at multiple SPADs of the array, even under simultaneous detection conditions. The correlator 203 is configured to generate output signal Corr identifying a detection event only responsive to detection of two or more “correlated” photons that arrive within a predetermined or adjustable correlation window. The correlation counter 203c is configured to increment and output a counter signal CC<3:0> in response to each detection event. The gated clock 207 is configured to be controlled based on signal StrobeB<i> to provide clock source signal FastClk to the time accumulator 202. The clock source signal FastClk provides sampling of the counter signal CC<3:0> at a frequency corresponding to the number of timesteps k).
Henderson does not teach, writing the data to respective bins of the pipeline memory.
However, Henderson as modified by Pacala and Onal teaches this, see Claim 3
11. Regarding Claim 11:
Henderson teaches the predetermined sampling rate corresponds to a period of a clock signal, ([0056]: In some embodiments, the accumulator(s) 201, 202 may be operable responsive to output from a parallel counter 204, the correlator 203 and correlation counter 203c, and/or a gated clock 207. The parallel counter 204 is configured to provide output signal C<4:0> that indicates detection of photons at multiple SPADs of the array, even under simultaneous detection conditions. The correlator 203 is configured to generate output signal Corr identifying a detection event only responsive to detection of two or more “correlated” photons that arrive within a predetermined or adjustable correlation window. The correlation counter 203c is configured to increment and output a counter signal CC<3:0> in response to each detection event. The gated clock 207 is configured to be controlled based on signal StrobeB<i> to provide clock source signal FastClk to the time accumulator 202. The clock source signal FastClk provides sampling of the counter signal CC<3:0> at a frequency corresponding to the number of timesteps k).
Henderson as modified by Onal does not teach the second memory storage operations are independent of the period of the clock signal.
However, Pacala teaches, ([0006]: The integration register may be clocked independently from the memory that represents the histogram).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson and Onal with Pacala to include the second memory storage operations are independent of the period of the clock signal, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson and Onal with Pacala since, making LiDAR memory storage independent of the clock signal allows systems to asynchronously process photon events over multiple clock cycles. This decoupling dramatically reduces system power consumption, minimizes clock-induced circuit noise, and prevents high-frequency processing bottlenecks during dense, multi-channel data collection.
12. Regarding Claim 12:
Henderson teaches the second memory storage operations are performed over two or more periods of the clock signal, ([0050]: Many repetitions of Strobe<i> are aggregated (e.g., in the pixel) to define a sub-frame for Strobe<i>, with subframes 1 to i defining an image frame. Each sub-frame for Strobe<i> may correspond to a respective distance sub-range of the overall imaging distance range, which is defined by the frequency of the laser cycle. The strobe windows may be referred to below with reference to timesteps k, where k denotes the k.sup.th time interval in a strobe/detection window which is sampled by a clock (referred to in some examples herein with reference to clock signal FastClk). The clock may have a period of some factor (e.g., 10 times or 20 times shorter than the in some examples described herein) shorter than the strobe window time duration). Henderson further teaches, ([0051]: Referring to FIG. 2, the accumulator circuits 201, 202 (also referred to herein as accumulators) may be registers of a processor or control circuit 205 that are operable to accumulate and integrate detection events (e.g., single-photon detection events, in some embodiments correlated within a time window) indicated by the detection signal(s) output from a single-photon detector array 110 (described with reference to a SPAD array) in response to detection of incident photons during or over a detection window. The example of FIG. 2 includes first and second accumulators 201 and 202, where the first accumulator 201 functions as a time or temporal accumulator or time integrator that is configured to track detection of photons incident upon the detector array 110 at respective times over the strobe window, and the second accumulator 202 functions as a photon counter to collect a photon count corresponding to a total number of photons that were detected during the strobe window. More particularly, the first accumulator 201 is configured to integrate the number of detection events that have been identified as of respective time intervals or time bins (i.e., the timesteps k) of the detection window as a representation of the respective times of arrivals of the detected photons, and the second accumulator 202 functions as a photon counter that is configured to count the total number detection events that were identified over the detection window).
13. Regarding Claim 15:
Henderson teaches a Light Detection and Ranging (LIDAR) detector circuit, ([Abstract]: A Light Detection And Ranging (LIDAR) measurement circuit). Henderson teaches one or more photodetector elements defining a LIDAR detector pixel, ([0004]: In some applications, the sensing of the reflected radiation may be performed using an array of photodetectors, including single-photon detectors, such as a Single Photon Avalanche Diode (SPAD) array. One or more photodetectors may define a detector pixel of the array). Henderson further teaches, ([0009]: According to some embodiments of the present invention, a Light Detection And Ranging (LIDAR) measurement circuit includes at least one processor circuit. The at least one processor circuit is configured to perform operations including receiving detection signals output from a plurality of detector elements). Henderson teaches the at least one control circuit is configured to execute the first memory storage operations responsive to a first clock signal, ([0010]: In some embodiments, the at least one processor circuit may include an accumulator circuit. The accumulator circuit may be configured to output the sum of the respective numbers of the detection events that have been identified at the respective time intervals based on a counter signal that is incremented responsive to each of the detection events, and based on a clock signal corresponding to the respective time intervals. For example, the accumulator circuit may define a time integration circuit configured to integrate a respective number of the detection events at each of the time intervals over the detection window responsive to respective pulses of the clock signal at the respective time intervals).
Henderson does not teach a pipeline memory device.
However Henderson as modified by Pacala and Onal teach this, See Claim 3.
Henderson does not teach a main memory device; and at least one control circuit configured to execute first and second memory storage operations to store current and previous data indicated by detection signals received from the LIDAR detector pixel in the pipeline and main memory devices, respectively.
However, Henderson as modified by Pacala, as applied to Claim 1, and Henderson as modified by Pacala and Onal, as applied to Claim 3, teaches this, See Claims 1 & 3
14. Regarding Claim 17:
Henderson as modified by Onal does not teach the at least one control circuit is configured to execute the second memory storage operations at least partially concurrently with execution of the first memory storage operations.
However, Pacala teaches, ([0102]: The ALU 804 may also perform a second aggregation operation that adds the total signal count to an existing value in a memory location of the memory 806. Recall from FIG. 7 that with each shot, a new total signal count may be added to an existing value in the corresponding time bin of the memory 806. In this manner, the histogram 818 can be gradually constructed in the memory 806 over a number of shots. When the total signal count is generated by the ALU 814, a current value 820 of a corresponding memory location for that time bin can be retrieved from the memory 806. The current value 820 can be provided as an operand to the ALU 804, which can be combined with the total signal count from the set of signals 816. In some embodiments, the ALU 804 can be composed of a first stage and a second stage, where the first stage calculates the total signal count from the photosensor 802, and the second stage combines the total signal count with the current value 820 from that time bin's memory location in the memory 806. In some embodiments, the aggregation of the set of signals 816 and the aggregation of total signal count and the current value 820 may be carried out as a single operation. Thus, even though these two operations may functionally be described as separate “aggregations,” they may in fact be performed together using a combination of parallel and sequential circuitry in the ALU 804).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson and Onal with Pacala to include at least one control circuit is configured to execute the first and second memory storage operations at least partially concurrently, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson with Pacala since, such configurations provide improved real-time object detection, increased system throughput, continuous data acquisition, faster processing, and improved fault tollerance. See Claim 1.
15. Regarding Claim 18:
Henderson as modified by Onal does not teach the main memory is partitioned into respective memory banks, and execution of the second memory storage operations comprises addressing respective memory bins of each of the respective memory banks in parallel.
However, Pacala teaches, ([0102]: The ALU 804 may also perform a second aggregation operation that adds the total signal count to an existing value in a memory location of the memory 806. Recall from FIG. 7 that with each shot, a new total signal count may be added to an existing value in the corresponding time bin of the memory 806. In this manner, the histogram 818 can be gradually constructed in the memory 806 over a number of shots. When the total signal count is generated by the ALU 814, a current value 820 of a corresponding memory location for that time bin can be retrieved from the memory 806. The current value 820 can be provided as an operand to the ALU 804, which can be combined with the total signal count from the set of signals 816. In some embodiments, the ALU 804 can be composed of a first stage and a second stage, where the first stage calculates the total signal count from the photosensor 802, and the second stage combines the total signal count with the current value 820 from that time bin's memory location in the memory 806. In some embodiments, the aggregation of the set of signals 816 and the aggregation of total signal count and the current value 820 may be carried out as a single operation. Thus, even though these two operations may functionally be described as separate “aggregations,” they may in fact be performed together using a combination of parallel and sequential circuitry in the ALU 804). Pacala further teaches, ([0127]-[0129]: [0127]: FIG. 15 illustrates shared aggregation circuitry between the two data paths, according to some embodiments. A single ALU 1501 can implement a first aggregation stage 1502 that sums the total signal count from the photodetectors of the photosensor 802. This first stage 1502 can generate an intermediate result 1504 that is then passed to each of the two data paths. For example, the intermediate result 1504 can be passed to the second stage 1306 of the histogram data path. This second stage 1306 can add the common intermediate result 1504 to the current value 820 for a current time bin in the memory 806 representing the histogram. The updated value can then be stored back in the current time bin in the memory 806. Similarly, the common intermediate result 1504 can be passed to the second stage 1406 of the integration data path. This second stage 1406 can add the intermediate result 1504 to the current value 1010 of the integration register 1004. This updated value can then be stored back in the integration register 1004. [0128]: When the first stage 1502 of the ALU 1501 is shared between both data paths, a periodic signal that clocks the ALU 1501 may also be shared between both data paths. Consequently, the size of the time bins in the integration data path and the size of the time bins in the histogram data path may be the same. However, even though the time bins may be the same size, the overall integration cycle in the integration data path may run continuously and still be independently controlled when compared to the histogram data path. For example, the memory 806 in the histogram data path may only store and update memory locations during pulse train measurements in a shot, while the integration register 1004 may continuously update its current value 1010 regardless of whether or not a shot is active in the histogram data path. [0129]: The example of FIG. 15 illustrates how the different aggregation functions used by both data paths can be shared or separated between the two data paths. Therefore, any recitation of an arithmetic logic circuit may encompass any aggregation function in either data path whether implemented as separate stages or as a single stage).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson and Onal with Pacala to include the main memory is partitioned into respective memory banks, and execution of the second memory storage operations comprises addressing respective memory bins of each of the respective memory banks in parallel, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson and Onal with Pacala since, such configurations can increase bandwidth, reduce memory access latency, eliminate data bottlenecks, and improve real-time processing for ADAS. Indeed, such a configuration is optimized for histogram processing. See Claim 13.
16. Regarding Claim 19:
Henderson teaches execution of the second memory storage operations comprises performing read, modify, and write operations to include the previous data in respective memory bins of the main memory responsive to a common precharge operation, See Claim 2.
17. Regarding Claim 20:
Henderson teaches, at least one control circuit, ([0051]: Referring to FIG. 2, the accumulator circuits 201, 202 (also referred to herein as accumulators) may be registers of a processor or control circuit 205). Henderson teaches a common precharge operation, ([0063]: At the end of the strobe window, the total number of correlated detection events CC<3:0> is also summed by the photon counter 202 (illustrated by way of example as a 18-bit accumulator), which output signal S<17:0> to the readout circuit 209. The signal S<17:0> provides a count value indicating the total number of correlated detection events that were identified by the end of the detection window. The readout circuit 209 outputs a data signal Data <44:0> (e.g., at the end of each subframe). As discussed below with reference to FIGS. 4A and 4B, a subframe may include data collected for a respective strobe window Strobe<i> that is repeated for multiple (e.g., thousands of) laser cycles). The accumulators 201 and/or 202 may be reset by signal FrameRst at the end of a subframe).
Henderson as modified by Onal does not teach, respective logic circuits configured to perform the read, modify, and write operations for the respective memory banks in parallel.
However, Pacala teaches, ([0102]: The ALU 804 may also perform a second aggregation operation that adds the total signal count to an existing value in a memory location of the memory 806. Recall from FIG. 7 that with each shot, a new total signal count may be added to an existing value in the corresponding time bin of the memory 806. In this manner, the histogram 818 can be gradually constructed in the memory 806 over a number of shots. When the total signal count is generated by the ALU 814, a current value 820 of a corresponding memory location for that time bin can be retrieved from the memory 806. The current value 820 can be provided as an operand to the ALU 804, which can be combined with the total signal count from the set of signals 816. In some embodiments, the ALU 804 can be composed of a first stage and a second stage, where the first stage calculates the total signal count from the photosensor 802, and the second stage combines the total signal count with the current value 820 from that time bin's memory location in the memory 806. In some embodiments, the aggregation of the set of signals 816 and the aggregation of total signal count and the current value 820 may be carried out as a single operation. Thus, even though these two operations may functionally be described as separate “aggregations,” they may in fact be performed together using a combination of parallel and sequential circuitry in the ALU 804). Pacala further teaches, ([0127]-[0129]: [0127]: FIG. 15 illustrates shared aggregation circuitry between the two data paths, according to some embodiments. A single ALU 1501 can implement a first aggregation stage 1502 that sums the total signal count from the photodetectors of the photosensor 802. This first stage 1502 can generate an intermediate result 1504 that is then passed to each of the two data paths. For example, the intermediate result 1504 can be passed to the second stage 1306 of the histogram data path. This second stage 1306 can add the common intermediate result 1504 to the current value 820 for a current time bin in the memory 806 representing the histogram. The updated value can then be stored back in the current time bin in the memory 806. Similarly, the common intermediate result 1504 can be passed to the second stage 1406 of the integration data path. This second stage 1406 can add the intermediate result 1504 to the current value 1010 of the integration register 1004. This updated value can then be stored back in the integration register 1004. [0128]: When the first stage 1502 of the ALU 1501 is shared between both data paths, a periodic signal that clocks the ALU 1501 may also be shared between both data paths. Consequently, the size of the time bins in the integration data path and the size of the time bins in the histogram data path may be the same. However, even though the time bins may be the same size, the overall integration cycle in the integration data path may run continuously and still be independently controlled when compared to the histogram data path. For example, the memory 806 in the histogram data path may only store and update memory locations during pulse train measurements in a shot, while the integration register 1004 may continuously update its current value 1010 regardless of whether or not a shot is active in the histogram data path. [0129]: The example of FIG. 15 illustrates how the different aggregation functions used by both data paths can be shared or separated between the two data paths. Therefore, any recitation of an arithmetic logic circuit may encompass any aggregation function in either data path whether implemented as separate stages or as a single stage).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson and Onal with Pacala to include logic units (ALU) and memory operations (read write modify) in parallel, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson and Onal with Pacala since, such configurations can maximize throughput, reduce memory latency, perform real-time data compression, and provide high energy efficiency, allowing a Lidar system to efficiently process large streams of time of flight data in real-time.
18. Regarding Claim 24:
Henderson teaches the first memory storage operations comprise sampling the data from the detection signals at a predetermined sampling rate, ([0050]: Many repetitions of Strobe<i> are aggregated (e.g., in the pixel) to define a sub-frame for Strobe<i>, with subframes 1 to i defining an image frame. Each sub-frame for Strobe<i> may correspond to a respective distance sub-range of the overall imaging distance range, which is defined by the frequency of the laser cycle. The strobe windows may be referred to below with reference to timesteps k, where k denotes the k.sup.th time interval in a strobe/detection window which is sampled by a clock (referred to in some examples herein with reference to clock signal FastClk). The clock may have a period of some factor (e.g., 10 times or 20 times shorter than the in some examples described herein) shorter than the strobe window time duration). Henderson further teaches, ([0056]: In some embodiments, the accumulator(s) 201, 202 may be operable responsive to output from a parallel counter 204, the correlator 203 and correlation counter 203c, and/or a gated clock 207. The parallel counter 204 is configured to provide output signal C<4:0> that indicates detection of photons at multiple SPADs of the array, even under simultaneous detection conditions. The correlator 203 is configured to generate output signal Corr identifying a detection event only responsive to detection of two or more “correlated” photons that arrive within a predetermined or adjustable correlation window. The correlation counter 203c is configured to increment and output a counter signal CC<3:0> in response to each detection event. The gated clock 207 is configured to be controlled based on signal StrobeB<i> to provide clock source signal FastClk to the time accumulator 202. The clock source signal FastClk provides sampling of the counter signal CC<3:0> at a frequency corresponding to the number of timesteps k). Henderson teaches the predetermined sampling rate corresponds to the period of the first clock signal, ([0056]: In some embodiments, the accumulator(s) 201, 202 may be operable responsive to output from a parallel counter 204, the correlator 203 and correlation counter 203c, and/or a gated clock 207. The parallel counter 204 is configured to provide output signal C<4:0> that indicates detection of photons at multiple SPADs of the array, even under simultaneous detection conditions. The correlator 203 is configured to generate output signal Corr identifying a detection event only responsive to detection of two or more “correlated” photons that arrive within a predetermined or adjustable correlation window. The correlation counter 203c is configured to increment and output a counter signal CC<3:0> in response to each detection event. The gated clock 207 is configured to be controlled based on signal StrobeB<i> to provide clock source signal FastClk to the time accumulator 202. The clock source signal FastClk provides sampling of the counter signal CC<3:0> at a frequency corresponding to the number of timesteps k).
Henderson as modified by Pacala and Onal teaches writing the data to respective bins of the pipeline memory device, See Claim 3.
19. Regarding Claim 25:
Henderson teaches the second memory storage operations are performed over two or more periods of the first clock signal, See Claim 12. Henderson teaches, second memory storage operations are performed during a time between pulses of an emitter signal output from a LIDAR emitter element, See Claim 14.
20. Regarding Claim 34:
Henderson as modified by Onal does not teach the pipeline memory, the main memory, and/or the temporary memory comprises a static random access memory (SRAM).
However, Pacala teaches, ([0101]: As described above, the output of the ALU 804 may characterize the total number of photons received by the photosensor 802 during a particular time bin. Each time the ALU 804 completes an aggregation operation, the total signal count can be added to a corresponding memory location in a memory 806 representing histogram 818. In some embodiments, the memory 806 may be implemented using an SRAM).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson and Onal with Pacala to include the pipeline memory, the main memory, and/or the temporary memory comprises a static random access memory (SRAM), since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson and Onal with Pacala since, the utilization of such memory enables, low to zero refresh latency, fast access times, optimized pipeline throughput, and robust temporary buffering.
21. Claims 4, 21, & 29 are rejected under 35 U.S.C. 103 as being unpatentable over Henderson et al (US 20200158836 A1), hereinafter Henderson, in view of Pacala et al (US 20200341144 A1), hereinafter Pacala, as applied to Claims 1 & 2, further in view of Onal et al (US 20200256963 A1), hereinafter Onal, as applied to Claims 3, 15, 16, & 28, and further in view of Sakaguchi et al (WO 2020116039 A1), hereinafter Sakaguchi.
22. Regarding Claims 4, 21 & 29:
Henderson as modified by Pacala and Onal does not teach, the at least one control circuit is further configured to execute third memory storage operations to transfer the previous data from the pipeline memory to the temporary memory before execution of the first memory storage operations.
However, Sakaguchi teaches a histograming Lidar system. Sakaguchi further teaches, ([0097]: 2.8.3.2.2.2 Operation to be Performed for One Light Emission In addition, as shown in FIG. 16, the sampling operation described with reference to FIG. 15 has a predetermined number of sampling times, for example, the first pixel mode. Is repeated twice (512 times in this example). At this time, in the second pixel mode, only the memories 170, 172, 174, and 176 for the first half of the second pixel mode histogram are used during the first half of the second pixel mode histogram (for example, the period of sampling numbers P0 to P255). High level enable signals EN0, EN2, EN4 and EN6 for permitting writing are applied, and low level enable signals EN1, EN3, EN5 and EN7 for inhibiting writing are provided to the memories 171, 173, 175 and 177 for the second half. give. As a result, during the period of the sampling numbers P0 to P255, the first half of the histogram (for example, corresponding to BINIDX=0 to 255) is stored in the memory 170 using the pixel values D of i=0, 2, 4, and 6. , 72, 174 and 176). Sakaguchi goes on to teach, ([0103]: In other words, in the third pixel mode, the histogram generation unit 163 creates one histogram using each of the four memories 170 to 173 and 174 to 177 that are the storage destinations of the four pixel values D having the same value. .. At that time, the first memories 170 and 174 are assigned to the first 1/4 bin of the histogram, the second memories 171 and 175 are assigned to the second 1/4 bin of the histogram, and the third memories 172 and 176 are assigned. To the third quarter BIN of the histogram and the last memories 173 and 177 to the remaining quarter BIN of the histogram. As a result, the number of BINs in the histogram can be increased four times, so that the distance measuring range can be further widened). Sakaguchi continues to teach, ([0223]: “A memory that stores, for each pixel, a histogram of the pixel value for each sampling period calculated by the pixel value generation unit”; “The memory includes a memory area corresponding to each of the first predetermined number of light receiving elements in a one-to-one relationship, When the first pixel mode is held in the control register, the histogram of one pixel is stored in one memory area, and when the second pixel mode is held in the control register, at least The distance measuring apparatus according to any one of (2) to (4), wherein the histogram of one pixel is stored in two memory areas. The memory includes a memory area corresponding to each of the first predetermined number of light receiving elements on a one-to-one basis, When the first pixel mode is held in the control register, the histogram of one pixel is stored in one memory area, and when the second pixel mode is held in the control register, at least Storing the histogram of one of the pixels in two of the memory areas, The pixel value having the same value output from the variable parallel adder is stored in different memory areas of the at least two memory areas storing the histogram of one pixel. The distance measuring device described in. The pixel value generation unit outputs the pixel value for each pixel as a pixel value matrix in which a row direction element is the third predetermined number and a column direction is the second predetermined number”, “The memory comprises the third predetermined number of first memories, Each first memory includes the second predetermined number of memory areas, The pixel value generation unit outputs the pixel value for each pixel as a pixel value matrix in which elements in the row direction are the third predetermined number and column directions are the second predetermined number”, “One memory address of the memory corresponds to one BIN of the histogram, The method further comprises a histogram generation unit that creates the histogram by writing the pixel values calculated by the pixel value generation unit for each sampling period in the memory address order of the memory. The distance measuring device according to any one of items”).
It would have been obvious to one of ordinary skill in the art at the time of filing to modify Henderson as modified by Pacala and Onal with Sakaguchi to include the at least one control circuit is further configured to execute third memory storage operations to transfer the previous data from the pipeline memory to the temporary memory before execution of the first memory storage operations, since it is the same field of endeavor and results would have been predictable. One of ordinary skill in the art at the time of filing would have been motivated to modify Henderson as modified by Pacala and Onal with Sakaguchi since, (Sakaguchi: [0103]: As a result, the number of BINs in the histogram can be increased four times, so that the distance measuring range can be further widened). In addition, such a configuration can prevent data overwriting, maximize memory bandwidth, limit interruptions to pipeline flow, and reduce latency.
Allowable Subject Matter
23. Claims 5-9, 22-23, & 30-32 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Examiner’s Note: Henderson as modified by Pacala and Onal teach all limitations of Claim 31, when the additional limitations of the base claim and all intervening claims are not considered.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
US 20210096263 A1: Discloses multiple memory configurations, including the use of SRAM and SDRAM.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES W NAPIER whose telephone number is (571)272-7451. The examiner can normally be reached Monday - Friday 8:00 am - 4:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Helal Algahaim can be reached at (571) 270-5227. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.W.N./Examiner, Art Unit 3645
/HELAL A ALGAHAIM/SPE , Art Unit 3645
Prosecution Insights