Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 8-9, and 15-16 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Gleason et al. (U.S. Pat. No. 10686523), hereinafter Gleason.
Regarding claim 1, Gleason teaches,
A multi-modality sensor (col. 4, lines 4-20, “As shown in FIG. 1a, a dual Optical/RF (O/RF) PIC 100 (and RF phased arrays) may be mounted on a mobile platform 102 such as a drone, manned vehicle etc. In one configuration, the steerable RF and optical beams 104 and 106 may be used to interrogate another mobile platform 108 with wavelengths covering LADAR/LIDAR and RADAR bands from a single integrated system”) comprising:
a radio detection and ranging (radar) sensor including a radio-frequency integrated circuit (RFIC) (fig. 4A, O/RF PIC 258. See also fig. 1A and col. 4, lines 4-20. See also col. 6, “In reference to FIGS. 4a and 4b, an embodiment of dual O/RF phased array 250 includes N×O RF patch antennas 252 that define an RF phased array 254 mounted on a carrier 256. An O/RF PIC 258 is mounted on carrier 256 in the space between RF patch antennas 252 within an RF antenna array footprint 260 defined by an outer periphery of the RF phased array 254.” See also fig. 5, optical phased array 306 is stacked on top of layer 312, which carries the shared circuitry, i.e., optical feed network 304); and
an image sensor, including a sensor array, stacked on a portion of the radar sensor (fig. 5, optical phased array 306 is stacked on a portion of the radar sensor. See also col. 6, “the O/RF PIC 258 receives optical input signals ω1 and ω2 via optical fibers 262 and 264 and generates a steerable optical beam 266 perpendicular to the surface of the PIC and carrier 256.” The examiner notes that the radar sensor is understood to be encapsulated by the RF antenna array footprint 260), with at least a circuitry portion of the image sensor being configured in a single chip with the RFIC (“In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308. Full monolithic integration provides a desirable advantage of having the entire feed modulate formed on a single chip.”).
Regarding claim 2, Gleason teaches,
The multi-modality sensor of claim 1, wherein the single chip includes the image sensor and the RFIC being configured as the single chip using a same substrate, or configured as the single chip in a form of a system in package (SIP) or a chiplet package (“In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308. Full monolithic integration provides a desirable advantage of having the entire feed modulate formed on a single chip.”).
Regarding claim 3, Gleason teaches,
The multi-modality sensor of claim 1, wherein the circuitry portion of the image sensor includes the sensor array (“An optical via 322 routes light vertically from optically phased array 306 out of the chip. Photo-detectors 324 are formed on top of the optical layer(s) and coupled to RF bond pads 326.”).
Regarding claim 4, Gleason teaches,
The multi-modality sensor of claim 1, wherein a first sampling rate of the image sensor and a second sampling rate of the radar sensor have a controlled correspondence with each other (“In reference to FIG. 2, an embodiment of a dual optical and RF phase array 150 that comprises a PIC 152 that provides a common architecture to feed both optical phased arrays 154 and RF phased arrays 156 from a single chip to generate either coincident or time multiplexed steerable optical and RF beams 158 and 160.” See also, “The switches may be “binary” switches that redirect effectively 100% of the optical power to either the optical phased array or the combiner, in which case the controller time multiplexes the optical and RF beams. The controller may modify the duty cycle as dictated by a particular application or changing conditions of the application.” The examiner notes that time multiplexing the optical and RF beams indicates that their transmit/receive pathways occur at some sampling rate. Since the transmit/receive paths can be controlled by a single switch that is controlled by the controller, the sampling rate of the two beams (i.e., the time in which one beam or the other is used) must share a controlled correspondence with each other).
Regarding claim 8, Gleason teaches,
The multi-modality sensor of claim 1, further comprising: an antenna array, including a plurality of antennas (fig. 4A, patch antennas 252), configured to radiate a first electromagnetic wave signal through the antenna array, and to receive a second electromagnetic wave signal corresponding to a reflection of the radiated first electromagnetic wave signal off of an object (“As shown in FIG. 1b, a number of dual O/RF PICs 110 (and RF phased arrays) are mounted on an autonomous vehicle 112 and controlled by an autonomous controller 113. The steerable optical beam 114 and steerable RF beam 116 cover LADAR/LIDAR and RADAR bands from a single integrated system to interrogate the environment and facilitate autonomous navigation.” The examiner notes that RADAR works by radiating electromagnetic signals and receiving signals reflected by the environment), wherein the antenna array is arranged, within a package of the multi-modality sensor, in a first lateral direction away from one or two first opposing sides of the sensor array, and/or in a second lateral direction, which is perpendicular to the first lateral direction, away from one or two second opposing sides of the sensor array (fig. 4a, antenna array is formed in a lateral direction away from the two horizontal sides of the optical phased array. See also fig. 2, “. Optical antennas 222 are formed in optical layer 216 as gratings that redirect light 224 (phase modulated light at ω1) upward away from the substrate. The optical waveguides and splitter for light at frequency ω1 are formed in optical layer 217 and directed upwards to optical layer 216 to the combiners. Photo-detectors 208 are formed on top of optical layer 216, typically of Germanium. The photo-detectors 208 convert light into an electrical feed signal 226 (RF frequency) that is output via one side of RF bond pads 214 (the other side being ground).”), to radiate a signal of the radar sensor to a target object located in a corresponding direction (fig. 2, fig. 4B both show that the RF antenna array and optical antenna array radiate a signal in the same (i.e., a corresponding) direction).
Regarding claim 9, Gleason teaches,
The multi-modality sensor of claim 8, wherein the antenna array and the sensor array are configured in a same chip (“In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308. Full monolithic integration provides a desirable advantage of having the entire feed modulate formed on a single chip.”).
Regarding claim 15, Gleason teaches,
A multi-modality sensor (col. 4, lines 4-20, “As shown in FIG. 1a, a dual Optical/RF (O/RF) PIC 100 (and RF phased arrays) may be mounted on a mobile platform 102 such as a drone, manned vehicle etc. In one configuration, the steerable RF and optical beams 104 and 106 may be used to interrogate another mobile platform 108 with wavelengths covering LADAR/LIDAR and RADAR bands from a single integrated system”) comprising:
a radio detection and ranging (radar) sensor including a radio-frequency integrated circuit (RFIC) formed on a substrate (fig. 4A, O/RF PIC 258. See also fig. 1A and col. 4, lines 4-20. See also col. 6, “In reference to FIGS. 4a and 4b, an embodiment of dual O/RF phased array 250 includes N×O RF patch antennas 252 that define an RF phased array 254 mounted on a carrier 256. An O/RF PIC 258 is mounted on carrier 256 in the space between RF patch antennas 252 within an RF antenna array footprint 260 defined by an outer periphery of the RF phased array 254.” See also fig. 5, “In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308.”);
an image sensor, including a sensor array, stacked on a portion of the radar sensor (col. 6, “the O/RF PIC 258 receives optical input signals ω1 and ω2 via optical fibers 262 and 264 and generates a steerable optical beam 266 perpendicular to the surface of the PIC and carrier 256.” The examiner notes that the radar sensor is understood to be encapsulated by the RF antenna array footprint 260. See also, “In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308.”); and
a plurality of antennas at least partially arranged around the sensor array, wherein the sensor array is arranged between the plurality of antennas (fig. 5, optical phased array 306 is arranged between antenna elements 310), and the sensor array and the plurality of antennas are configured in a form of a single chip (“In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308. Full monolithic integration provides a desirable advantage of having the entire feed modulate formed on a single chip.”).
Regarding claim 16, Gleason teaches,
The multi-modality sensor of claim 15, wherein a first sampling rate of first data of the image sensor and a second sampling rate of second data of the radar sensor have a controlled correspondence to each other (“In reference to FIG. 2, an embodiment of a dual optical and RF phase array 150 that comprises a PIC 152 that provides a common architecture to feed both optical phased arrays 154 and RF phased arrays 156 from a single chip to generate either coincident or time multiplexed steerable optical and RF beams 158 and 160.” See also, “The switches may be “binary” switches that redirect effectively 100% of the optical power to either the optical phased array or the combiner, in which case the controller time multiplexes the optical and RF beams. The controller may modify the duty cycle as dictated by a particular application or changing conditions of the application.” The examiner notes that time multiplexing the optical and RF beams indicates that their transmit/receive pathways occur at some sampling rate. Since the transmit/receive paths can be controlled by a single switch that is controlled by the controller, the sampling rate of the two beams (i.e., the time in which one beam or the other is used) must share a controlled correspondence with each other).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 5-7, 14, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Burlina et al. (U.S. Pub. No. 2025/0076486 A1), hereinafter Burlina.
Regarding claim 5, Gleason teaches the multi-modality sensor of claim 1. Although Gleason teaches that the sensor can be used for radar and lidar imaging (col. 4, “The steerable optical beam 114 and steerable RF beam 116 cover LADAR/LIDAR and RADAR bands from a single integrated system to interrogate the environment and facilitate autonomous navigation.”), Gleason is silent as to how data gathered from the environment should be processed, and therefore does not teach,
…wherein the multi-modality sensor is configured to combine first data generated by the image sensor and second data generated by the radar sensor into a combined time-synchronized data.
Burlina teaches,
…wherein the multi-modality sensor is configured to combine first data generated by the image sensor and second data generated by the radar sensor into a combined time-synchronized data (paras. 0058-0059, “Each of the infrared-radar fusion system 106, the image object detection pipeline 328, and the lidar object detection pipeline 330 may operate independently based on their respective sensor modalities, and may output separate sets of time-synchronized object detections. For instance, the infrared-radar fusion system 106 may output bounding shapes 332 (including size, position data, object attributes, confidence scores, etc.), the image object detection pipeline 328 may output a separate set of bounding shapes 334, and the lidar object detection pipeline 330 may output another separate set of bounding shapes 336. The object detection synchronization component 338 may receive the bounding shapes 332-336 from the various independent object detection systems, and may combine the object detections into a synchronized set of object detections to be provided to the downstream prediction and planning components of the vehicle 102.”).
Gleason and Burlina are analogous to the claimed invention because they are in the same field of endeavor. It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the multi-modality sensor of Gleason with the sensor fusion processing of Burlina to arrive at the claimed invention. Gleason teaches using the multi-modality sensor for combined lidar/radar imaging, but does not specify how radar and lidar data should be combined. Burlina teaches using time synchronization to perform multi-sensor radar fusion. The process of synchronizing timestamps to fuse data from disparate sensors is well-known in the art, and thus would be an obvious technique to use to realize the integrated lidar/radar system of Gleason.
Regarding claim 6, Gleason as previously combined with Burlina teaches the multi-modality sensor of claim 5. Gleason further teaches that the radar and lidar devices of the multi-modality sensor are aligned according to a field of view (FoV) of the image sensor (fig. 7A, FOV of optical beam 414 and radar beam 416 have the same boresight. See also abs.). It thus follows that any collected data should be aligned according to that shared field of view. However, because Gleason is silent as to how collected radar and lidar data are treated, Gleason cannot explicitly teach,
…wherein the combined time-synchronized data represents image and radar data that are aligned according to a field of view (FoV) of an image frame captured by the image sensor Burlina teaches,
…wherein the combined time-synchronized data represents image and radar data that are aligned according to a field of view (FoV) of an image frame captured by the image sensor (para. 0059, “The object detection synchronization component 338 may receive the bounding shapes 332-336 from the various independent object detection systems, and may combine the object detections into a synchronized set of object detections to be provided to the downstream prediction and planning components of the vehicle 102. For example, in some cases, two or more of the different object detection systems (e.g., infrared-radar fusion system 106, image object detection pipeline 328, lidar object detection pipeline 330, etc.), can output different object detections that correspond to the same real-world object in the physical environment.”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the multi-modality sensor of Gleason with the data processing of Burlina because the multi-modality sensor of Gleason is taught to be useful for combined radar/lidar imaging but silent as to how exactly the images are aligned, and the data processing of Burlina offers a method of fusing radar and lidar data.
Regarding claim 7, Gleason as previously combined with Burlina teaches the multi-modality sensor of claim 5. Gleason does not teach,
…wherein the combined time-synchronized data comprises least one respective information of a direction vector, speed, or three-dimensional location including an angle and a distance from the multi-modality sensor to a target object.
Burlina teaches,
…wherein the combined time-synchronized data comprises least one respective information of a direction vector, speed, or three-dimensional location including an angle and a distance from the multi-modality sensor to a target object (para. 0058, “For instance, the infrared-radar fusion system 106 may output bounding shapes 332 (including size, position data, object attributes, confidence scores, etc.), the image object detection pipeline 328 may output a separate set of bounding shapes 334, and the lidar object detection pipeline 330 may output another separate set of bounding shapes 336.” See also para. 0040, “For example, position and/or of a detected object may be tracked over time, and characteristics of the motion (e.g., speed, yaw, uniformity of direction and speed, and the like) may be used to classify the object as a pedestrian, animal, cyclist, etc.”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the multi-modality sensor of Gleason with the specific data of Burlina. Gleason teaches using the multi-modality sensor, “to interrogate the environment and facilitate autonomous navigation” (col. 4), but is silent as to how the sensor processes radar and lidar data to do so. However, outputting bounding shapes as taught by Burlina is a well-known technique in the art to interrogate the environment and facilitate autonomous navigation. Therefore, it would be obvious to use the bounding box data of Burlina to achieve the combined radar/lidar sensing of Gleason.
Regarding claim 14, Gleason teaches the multi-modality sensor of claim 1. Gleason does not teach,
…wherein the multi-modality sensor is configured to provide a driving reference signal for time synchronization with at least one external sensor
Burlina teaches,
…wherein the multi-modality sensor is configured to provide a driving reference signal for time synchronization with at least one external sensor (para. 0058, “Each of the infrared-radar fusion system 106, the image object detection pipeline 328, and the lidar object detection pipeline 330 may operate independently based on their respective sensor modalities, and may output separate sets of time-synchronized object detections.” See also fig. 3. The examiner notes that the time stamps of the combined infrared-radar fusion system is being understood to be a driving reference signal used to synchronized the multi-modality infrared-radar sensor with the ).
The examiner notes that the multi-modality sensor of Gleason does not provide a driving reference signal, but the multi-modality (radar-IR) sensor of Burlina does provide a driving reference signal to the external lidar and camera sensors, indicating that it would be reasonable to modify the multi-modality sensor of Burlina to do so.
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the multi-modality sensor of Gleason to provide a driving reference signal to interact with data from other sensors like the multi-modality sensor of Burlina does. Gleason teaches using the multi-modality sensor to facilitate autonomous driving. As Burlina teaches, autonomous vehicles usually use multiple different types of sensors because each kind of sensor has different strengths and weaknesses (see, e.g., the discussion of the differences in accuracy between different sensors in different environments in paras. 0060-0061). Thus, enabling sensor fusion between the multi-modality sensor system of Gleason and other types of sensors as Burlina teaches would increase the utility of the multi-modality sensor in real-world autonomous vehicle environments.
Regarding claim 17, Gleason teaches the multi-modality sensor of claim 15. Although Gleason teaches that the sensor can be used for radar and lidar imaging (col. 4, “The steerable optical beam 114 and steerable RF beam 116 cover LADAR/LIDAR and RADAR bands from a single integrated system to interrogate the environment and facilitate autonomous navigation.”), Gleason is silent as to how data gathered from the environment should be processed, and therefore does not teach,
…wherein the multi-modality sensor is configured to combine first data generated by the image sensor and second data generated by the radar sensor into a combined time-synchronized data.
Burlina teaches,
…wherein the multi-modality sensor is configured to combine first data generated by the image sensor and second data generated by the radar sensor into a combined time-synchronized data (paras. 0058-0059, “Each of the infrared-radar fusion system 106, the image object detection pipeline 328, and the lidar object detection pipeline 330 may operate independently based on their respective sensor modalities, and may output separate sets of time-synchronized object detections. For instance, the infrared-radar fusion system 106 may output bounding shapes 332 (including size, position data, object attributes, confidence scores, etc.), the image object detection pipeline 328 may output a separate set of bounding shapes 334, and the lidar object detection pipeline 330 may output another separate set of bounding shapes 336. The object detection synchronization component 338 may receive the bounding shapes 332-336 from the various independent object detection systems, and may combine the object detections into a synchronized set of object detections to be provided to the downstream prediction and planning components of the vehicle 102.”).
Gleason and Burlina are analogous to the claimed invention because they are in the same field of endeavor. It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the multi-modality sensor of Gleason with the sensor fusion processing of Burlina to arrive at the claimed invention. Gleason teaches using the multi-modality sensor for combined lidar/radar imaging, but does not specify how radar and lidar data should be combined. Burlina teaches using time synchronization to perform multi-sensor radar fusion. The process of synchronizing timestamps to fuse data from disparate sensors is well-known in the art, and thus would be an obvious technique to use to realize the integrated lidar/radar system of Gleason.
Regarding claim 18, Gleason in view of Burlina teaches the multi-modality sensor of claim 17. Gleason further teaches that the radar and lidar devices of the multi-modality sensor are aligned according to a field of view (FoV) of the image sensor (fig. 7A, FOV of optical beam 414 and radar beam 416 have the same boresight. See also abs.). It thus follows that any collected data should be aligned according to that shared field of view. However, because Gleason is silent as to how collected radar and lidar data are treated, Gleason cannot explicitly teach,
… wherein the combined time-synchronized data is aligned according to a field of view (FoV) within an image frame of the image sensor
Burlina teaches,
… wherein the combined time-synchronized data is aligned according to a field of view (FoV) within an image frame of the image sensor (para. 0059, “The object detection synchronization component 338 may receive the bounding shapes 332-336 from the various independent object detection systems, and may combine the object detections into a synchronized set of object detections to be provided to the downstream prediction and planning components of the vehicle 102. For example, in some cases, two or more of the different object detection systems (e.g., infrared-radar fusion system 106, image object detection pipeline 328, lidar object detection pipeline 330, etc.), can output different object detections that correspond to the same real-world object in the physical environment.”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the multi-modality sensor of Gleason with the data processing of Burlina because the multi-modality sensor of Gleason is taught to be useful for combined radar/lidar imaging but silent as to how exactly the images are aligned, and the data processing of Burlina offers a method of fusing radar and lidar data.
Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Gleason in view of Long (CN 112067004 B).
Regarding claim 10, Gleason teaches the multi-modality sensor of claim 1. Gleason does not teach,
…further comprising: an upper portion of the multi-modality sensor comprising a first extraction circuitry configured to extract image information by reading out a signal of the sensor array, based on a clock signal; and a lower portion of the multi-modality sensor comprising one of: a second extraction circuitry configured to extract radar information from a signal of the radar sensor, based on a source signal, or the second extraction circuitry and a transmission module configured to transmit a combined time-synchronized data that has the image information combined with the radar information.
Long teaches (note: what Long does not teach is struck through),
…further comprising: (“As described above, a first type of radar sensor (e.g., liDAR) may be included in an autonomous driving system in which a clock source signal output by an inertial navigation device may be synchronized to each first type of radar sensor. The first type of radar sensors can stamp the sensor data acquired by the first type of radar sensors according to the clock source signal, and upload the data with the time stamp to a computing platform for processing”); and (“In addition, a second type of Radar sensor (e.g., millimeter wave Radar) in the automatic driving system can send the sensor data collected by the second type of Radar sensor to a computing platform for processing, and after receiving the sensor data, the computing platform can set a timestamp for the sensor data according to a clock source signal synchronized by a raspberry group. Since the timestamp of the sensor data of Radar is also based on the clock of the computing platform, time domain synchronization is also achieved with other devices such as LiDAR, cameras, and the like.”), or the second extraction circuitry and a transmission module configured to transmit a combined time-synchronized data that has the image information combined with the radar information.
Although neither Gleason nor Long teach the particular physical positioning of the extraction circuitry, the positioning appears to be no more than a design choice that would not affect the operation of the device.
Long is analogous to the claimed invention because it is in the same field of endeavor. It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the multi-modality sensor of Gleason with the extraction circuitry of Long. As stated above, Gleason teaches using the multi-modality sensor for radar/lidar sensor fusion but is silent as to how data is extracted. The system of Long enables radar/lidar data fusion, thus offering structure that achieves the radar/lidar use case of the multi-modality system of Gleason.
Regarding claim 11, Gleason in view of Long teaches the multi-modality sensor of claim 10. Gleason does not teach,
…wherein the first extraction circuitry comprises at least one of: a control logic circuit configured to generate and transmit a control signal for reading out a first signal of the sensor array; a decoder configured to decode an analog signal of the sensor array; a first analog-to-digital converter (ADC) configured to convert the decoded analog signal into a digital signal; or a clock generator configured to generate the clock signal
Long teaches,
…wherein the first extraction circuitry comprises at least one of: a control logic circuit configured to generate and transmit a control signal for reading out a first signal of the sensor array; a decoder configured to decode an analog signal of the sensor array; a first analog-to-digital converter (ADC) configured to convert the decoded analog signal into a digital signal; or a clock generator configured to generate the clock signal (“Referring to fig. 1, the clock source signal output by the Novatel device is actually used as the clock source of each device to be synchronized in the automatic driving system, such as a radar sensor LiDAR, computing platforms Tegra a and Tegra B, a camera, and the like, and the clock sources of these devices are all directly or indirectly from the GPS/PPS signal output by the Novatel device.”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to use the extraction circuitry of Long in the multi-modal sensor of Gleason. As noted above, Gleason teaches using the multi-modality sensor for integrated radar/lidar sensing. Gleason’s multi-modality sensor does not explicitly contain circuitry for extracting sensor data. However, the circuitry of Long is standard circuitry for a lidar front end, and thus would be obvious to incorporate when using the multi-modal sensor of Gleason for radar/lidar imaging.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Gleason in view of Long and further in view of Wikipedia (Wikipedia. "RF front end" (2021 Apr. 24). Accessed via Wayback Machine).
Regarding claim 12, Gleason in view of Long teaches the multi-modality sensor of claim 10. Gleason in view of Long does not teach,
…wherein the second extraction circuitry comprises at least one of: a ramp generator configured to generate the source signal for the radar sensor; a synthesizer configured to change a frequency band of the source signal; a phase controller configured to generate a radiation signal in a frequency band in which a signal with the changed frequency band is up-converted by a multiple of 4; a filter low-noise amplifier (LNA) configured to detect and amplify a reflection signal in which the radiation signal is reflected by hitting a target object; an intermediate frequency (IF) circuit configured to extract the source signal from the amplified reflection signal; or a second ADC configured to convert the extracted source signal into a digital signal
Wikipedia teaches,
…wherein the second extraction circuitry comprises at least one of: a ramp generator configured to generate the source signal for the radar sensor; a synthesizer configured to change a frequency band of the source signal; a phase controller configured to generate a radiation signal in a frequency band in which a signal with the changed frequency band is up-converted by a multiple of 4; a filter low-noise amplifier (LNA) configured to detect and amplify a reflection signal in which the radiation signal is reflected by hitting a target object; an intermediate frequency (IF) circuit configured to extract the source signal from the amplified reflection signal; or a second ADC configured to convert the extracted source signal into a digital signal (Wikipedia, list of bullet points in para. 2 notes that an LNA and IF circuit are both standard parts of a radar front end).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Gleason in view of Long with the second extraction circuitry of Wikipedia because the extraction circuitry of Wikipedia is a standard part of a radar front end, and thus standard practice for extracting data from a radar sensor, a necessity for realizing the combined radar/lidar sensing use case suggested by Gleason.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Gleason in view of Long and further in view of Texas Instruments (Texas Instruments. "Design guide: TIDA-010131" (2019). PDF).
Regarding claim 13, Gleason in view of Long teaches the multi-modality sensor of claim 10. Gleason further teaches (note: what Gleason does not teach is struck through)
…wherein the multi-modality sensor is configured to combine first data of the image sensor and second data of the radar sensor to generate the combined time-synchronized data by controlling a first sampling rate of the image sensor and a second sampling rate of the radar sensor (“The switches may be “binary” switches that redirect effectively 100% of the optical power to either the optical phased array or the combiner, in which case the controller time multiplexes the optical and RF beams.” The examiner notes that the time multiplexing of the beams constitutes a sensor that is configured to combine first data of the image sensor and second data of the radar sensor to generate the combined time-synchronized data by controlling sampling rates because the time-multiplexed switching of Gleason would result in data that can be combined into time-synchronized data) based on the source signal (“The controller can modify the amount of optical power directed to either channel as dictated by a particular application or changing conditions of the application.” The examiner notes that the signal from the controller is the source signal).
Texas Instruments teaches,
… controlling a first sampling rate of the image sensor and a second sampling rate of the radar sensor, based on…the clock signal generated by the clock generator comprised in the multi-modality sensor (p. 1, left col., para. 1, “Each transceiver signal chain includes high-speed, analog-to-digital converters (ADCs), digital-to-analog converters (DACs), and a clock subsystem”).
Texas Instruments is analogous to the claimed invention because it is in the same field of endeavor. It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the sensor of Gleason with the sampling rate based on a clock signal of Texas Instruments. Gleason teaches controlling a sampling rate but is silent as to the structure used to control said sampling rate. However, a clock (such as the structure taught by Texas Instruments) is a standard method of controlling sampling rate. Thus, it would be obvious to a person of ordinary skill in the art at the time of filing to use a clock to control the binary switch sampling rate of Gleason.
Claims 19-22 are rejected under 35 U.S.C. 103 as being unpatentable over Gleason in view of Schmalenberg et al. (U.S. Pub. No. 2016/0223663 A1), hereinafter Schmalenberg.
Regarding claim 19, Gleason teaches (note: what Gleason does not teach is struck through),
A multi-modality sensor comprising: a radio detection and ranging (radar) sensor including a radio-frequency integrated circuit (RFIC) (“In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308.”);
an image sensor, including a sensor array, stacked on a portion of the radar sensor (fig. 5, optical phased array 306 is stacked on a portion of the radar sensor. See also col. 6, “the O/RF PIC 258 receives optical input signals ω1 and ω2 via optical fibers 262 and 264 and generates a steerable optical beam 266 perpendicular to the surface of the PIC and carrier 256.” The examiner notes that the radar sensor is understood to be encapsulated by the RF antenna array footprint 260); and
a common circuit (“The present invention describes a photonic integrated circuit (PIC) that provides a common architecture to feed both optical and RF phased arrays to produce co-boresighted optical and RF beams”)
wherein the radar sensor, the image sensor, and the common circuit are packaged as a single chip (“In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308. Full monolithic integration provides a desirable advantage of having the entire feed modulate formed on a single chip.”).
Schmalenberg teaches,
A multi-modality sensor comprising: a radio detection and ranging (radar) sensor including…a common circuit including at least one of a down-sampling circuit configured to down-sample a signal generated by the radar sensor, a read-out circuit configured to read out data from the sensor array, or a clock generator (fig. 1, processor 130 reads out data from Lidar sensors 110)…
Schmalenberg is analogous to the claimed invention because it is in the same field of endeavor. It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Gleason with the read-out circuit of Schmalenberg. Gleason teaches using the multi-modality sensor for radar/lidar detection but is silent as to how data is read out. The processor of Schmalenberg is one method of using the data collected by the multi-modality sensor to realize the combined radar/lidar detection unit of Gleason, and using a standard processor to process the kind of sensor data produced by the sensor of Gleason is a technique that is well-known in the art.
Regarding claim 20, Gleason in view of Schmalenberg teaches the multi-modality sensor of claim 19. Gleason further teaches,
…wherein the common circuit is arranged between the radar sensor and the image sensor (fig. 5, common circuit 304 is arranged between optical phased array 306 and left-most radar patch 310).
Regarding claim 21, Gleason in view of Schmalenberg teaches the multi-modality sensor of claim 19. Gleason further teaches,
…wherein the common circuit is arranged on a same substrate as the radar sensor (“In reference to FIG. 5, an embodiment of a dual optical and RF phased array PIC 300 is monolithically fabricated using Silicon processing on a silicon substrate 302 to include an optical feed network 304, an optical phased array 306 and an RF phased array 308. Full monolithic integration provides a desirable advantage of having the entire feed modulate formed on a single chip.”).
Regarding claim 22, Gleason in view of Schmalenberg teaches the multi-modality sensor of claim 19. Gleason further teaches,
…wherein the common circuit is arranged at least partially around the radar sensor and the image sensor without being in contact with the radar sensor and the image sensor (fig. 5, common circuit 304 is arranged at least partially around the optical phased array 306 and radar patches 310, without being in contact with the radar and image sensors. The blue line of the image below surrounds the optical phased array and radar patch antennas, and thus the red line outlining the common circuit elements at least partially surrounds said elements.).
PNG
media_image1.png
436
749
media_image1.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Anna K Benjamin Gosling whose telephone number is (571)272-0401. The examiner can normally be reached Monday - Friday, 9-5 Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vladimir Magloire can be reached at (571) 270-5144. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Anna K. Gosling/Examiner, Art Unit 3648
/VLADIMIR MAGLOIRE/Supervisory Patent Examiner, Art Unit 3648
Prosecution Insights