DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The following addresses Applicant’s remarks/amendments dated 07 August 2026.
Claims 1-5, 7, 10, 12-14, 16, and 19-20 were amended; no claims were added; no claims were cancelled; therefore, Claims 1-20 are pending in the current application and will be addressed below.
Response to Argument
Applicant’s arguments filed 07 August 2026 with respect to Claims 1-20 have been fully considered but are moot because the arguments do not apply to the specific combination of references being used in the current rejection.
Claim Objections
Claims 1, 10, and 19 are objected to because of the following informalities:
Claims 1 and 19 recite “the second SFOV”
This should be amended to recite “the SFOV” as there is only one SFOV
Claim 10 recites “FOV” and “S-FOV”
These should be amended to distinguish “FOV” from “SFOV” vs “FFOV” and “S-FOV” should be amended to recite “SFOV” for the sake of uniform language across claims.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-10, and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Benemann et al. (US 2022/0394156 A1) in view of Lee et al. (US 2022/0373661 A1).
Regarding Claim 1, Benemann teaches a system ([0022] the computing system may synchronize image data capture (by the image sensor) with LIDAR data capture (by the LIDAR sensor)) comprising:
a first sensor having a first field-of-view (FFOV), the first sensor being disposed in an environment and arranged to capture first sensor data from the environment within the FFOV;
a second sensor having a second field of view (SFOV), the second sensor being disposed in the environment proximal to the first sensor and arranged to capture the second sensor data from the environment within the second SFOV ([0019] a LIDAR sensor of the vehicle may capture LIDAR data (e.g., LIDAR points) within a second field of view of the LIDAR sensor which may at least partially overlap with the first field of view of the image sensor);
a non-transient memory ([0040] According to some examples, the components 300 may include the timestamp component(s) 110, a sensor 302 (e.g., an image sensor), and/or a memory 304);
one or more processors coupled to the memory, the first sensor, and the second sensor, the memory including instructions ([0040] In a non-limiting example, the timestamp component(s) 110 may receive a stream or sequence of scan lines from the sensor 302 via one or more interfaces (e.g., a Mobile Industry Processor Interface (MIPI)) and/or one or more other components configured to transmit data (e.g., image data, scan lines, etc.) between the sensor 302 and the timestamp component(s) 110) configured to cause the one or more processors to:
direct at least the FFOV of the first sensor at different regions of space within the environment according to a scan cycle ([0023] a spinning LIDAR sensor may be associated with a field of view that moves relative to a field of view associated with the image sensor);
determine, a direction of the FFOV of the first sensor, and a direction of the SFOV of the second sensor, the second sensor being configured to capture second sensor data from the environment within the second SFOV ([0023] the computing system may determine a first orientation of the first field of view and/or a first pose (position and orientation) associated with the image sensor. Furthermore, the computing system may determine a second orientation of the second field of view and/or a second pose associated with the LIDAR sensor. In some examples, the respective orientations and/or the respective poses associated with the image sensor and/or the LIDAR sensor may be tracked), an estimated time between when a state of the first scan cycle of the first sensor in which a first point within the FFVO of the first sensor is aligned with a second point within the SFOV of the second sensor and when a state of a second scan cycle of the first sensor in which a first point within the FFOV of the first sensor realigns with the second point within the SFOV of the second sensor ([0023] According to some examples, the orientations and/or the poses may be tracked relative to one another. The computing system may use field of view orientation information and/or pose information as an input for causing the image sensor to initiate the rolling shutter image capture of the first field of view, e.g., such that at least a first portion of the first field of view (associated with the image sensor) overlaps at least a second portion of the second field of view (associated with the LIDAR sensor) in accordance with the synchronization condition(s). In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view);
determine a time offset for the second sensor based on the estimated time ([0023] In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view); and
send, to the second sensor, identifying the time offset to trigger the second sensor to capture second sensor data at or after specific time intervals defined by the time offset when the FFOV and the SFOV are aligned ([0023] In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view).
Benemann is not relied upon as teaching that the instructions cause the processors to: determine a frequency of the scan cycle of the first sensor; and determine, based on the frequency of the scan cycle of the first sensor, a direction of the FFOV of the first sensor.
However, Lee teaches that the instructions cause the processors to:
determine a frequency of the scan cycle of the first sensor ([0036] the computing device 124 may make such inference, prediction, or determination based on an assumption of a constant rotation speed of the LiDAR sensor 204 over time, for example, 25 degrees every 50 milliseconds); and
determine, based on the frequency of the scan cycle of the first sensor, a direction of the FFOV of the first sensor ([0036] The data may include historical angles and corresponding timestamps of the LiDAR sensor 204. For example, the computing device 124 may receive an indication, or otherwise have previously determined or predicted, that at a timestamp 205 indicating a time of 8:00:00.0, the LiDAR sensor 204 has an orientation 206 of 25 degrees with respect to a reference axis. The computing device 124 may also receive an indication, or otherwise have previously determined or predicted, that at a timestamp 215 indicating a time of 8:00:00.05, the LiDAR sensor 204 has an orientation 216 of 50 degrees with respect to a reference axis).
Benemann and Lee are considered to be analogous to the claimed invention because they are both in the same field of multi-sensor synchronization and spatial tracking systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the sensor control system of Benemann to include determining the frequency of the scan cycle and determining sensor direction based on the scan cycle frequency of Lee with a reasonable expectation of success. This modification would have been motivated by the desire to accurately predict dynamic field-of-view alignments over time without relying on continuous active polling or manual calibration. By integrating Lee’s teaching of predicting sensor orientation and timing using rotation frequency assumptions and timestamped angles into Benemann’s multi-sensor trigger control pipeline, the system can dynamically compute precise time offsets for triggering data capture exactly when fields of view overlap across repeated scan cycles. A person of ordinary skill in the art would recognize that combining known frequency-based tracking calculations with automated multi-sensor synchronization would yield the predictable result of optimizing field-of-view overlap while reducing computational processing overhead.
Regarding Claims 6 and 15, Benemann teaches that the state of the first scan cycle and the state of the second scan cycle are associated with a scan direction of the first sensor during the first scan cycle and the second scan cycle ([0050] According to some examples, the second field of view 510 of the LIDAR sensor 504 may move relative to the first field of view 508 of the image sensor 502… the rotation 516 of the second field of view 510 is in a clockwise direction. It should be understood, however, that the rotation 516 may be in a counter-clockwise direction and/or may change in direction in some examples).
Regarding Claims 7 and 16, Benemann teaches that the first sensor comprises a light detection and ranging (LIDAR) sensor, wherein each scan cycle comprises a cycle of rotation performed by at least one of a spinning mirror and laser of the LIDAR sensor while collecting sensor data ([0050] the second field of view 510 of the LIDAR sensor 504 may move relative to the first field of view 508 of the image sensor 502. In some examples, the second field of view 510 may be rotatable (e.g., about one or more axes). In some non-limiting examples, the second field of view 510 may be rotatable 360 degrees).
Regarding Claim 8 and 17, Benemann teaches that the second sensor comprises a camera sensor, a radio detection and ranging sensor (RADAR) sensor, a time-of-flight (TOF) sensor, or a light detection and ranging (LIDAR) sensor ([0022] the computing system may synchronize image data capture (by the image sensor) with LIDAR data capture (by the LIDAR sensor). For example, the computing system may trigger the image sensor to perform a rolling shutter image capture of a scene during a time period in which the LIDAR sensor is capturing LIDAR data of at least a portion of the scene corresponding to a field of view of the image sensor).
Regarding Claims 9 and 18, Benemann teaches that the first sensor and the second sensor are mounted on a vehicle ([0025] As another example, by synchronizing sensor data capture using the techniques described herein, the computing system of the vehicle may be able to accurately associate (e.g., temporally and/or spatially) image data with LIDAR data).
Regarding Claim 10, Benemann teaches a method ([0025] by synchronizing sensor data capture using the techniques described herein, the computing system of the vehicle may be able to accurately associate (e.g., temporally and/or spatially) image data with LIDAR data) comprising:
directing a first field of view (FFOV) of a first sensor disposed in an environment to capture first sensor data from the environment within the FFOV at different regions of space within the environment according to a scan cycle ([0023] a spinning LIDAR sensor may be associated with a field of view that moves relative to a field of view associated with the image sensor);
determining, a direction of the FFOV of the first sensor, and a direction of a second field of view (S-FOV) of a second sensor, the second sensor being configured to capture second sensor data from the environment within the second SFOV ([0023] the computing system may determine a first orientation of the first field of view and/or a first pose (position and orientation) associated with the image sensor. Furthermore, the computing system may determine a second orientation of the second field of view and/or a second pose associated with the LIDAR sensor. In some examples, the respective orientations and/or the respective poses associated with the image sensor and/or the LIDAR sensor may be tracked), an estimated time between when a state of a first scan cycle of the first sensor in which a first point within the FOV of the first sensor is aligned with a second point within the FOV of the second sensor and when a state of a second scan cycle of the first sensor in which the first point within the FOV of the first sensor realigns with the second point within the FOV of the second sensor ([0023] According to some examples, the orientations and/or the poses may be tracked relative to one another. The computing system may use field of view orientation information and/or pose information as an input for causing the image sensor to initiate the rolling shutter image capture of the first field of view, e.g., such that at least a first portion of the first field of view (associated with the image sensor) overlaps at least a second portion of the second field of view (associated with the LIDAR sensor) in accordance with the synchronization condition(s). In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view);
determining a time offset for the second sensor based on the estimated time ([0023] In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view); and
sending, to the second sensor, a signal identifying the time offset to trigger the second sensor to capture second sensor data at or after specific time intervales defined by the time offset when the FFOV and SFOV are aligned ([0023] In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view).
Benemann is not relied upon as teaching determining a frequency of the scan cycle of the first sensor and determining, based on the frequency of the scan cycle of the first sensor, a direction of the FFOV of the first sensor.
However, Lee teaches determining a frequency of the scan cycle of the first sensor ([0036] the computing device 124 may make such inference, prediction, or determination based on an assumption of a constant rotation speed of the LiDAR sensor 204 over time, for example, 25 degrees every 50 milliseconds) and determining, based on the frequency of the scan cycle of the first sensor, a direction of the FFOV of the first sensor ([0036] The data may include historical angles and corresponding timestamps of the LiDAR sensor 204. For example, the computing device 124 may receive an indication, or otherwise have previously determined or predicted, that at a timestamp 205 indicating a time of 8:00:00.0, the LiDAR sensor 204 has an orientation 206 of 25 degrees with respect to a reference axis. The computing device 124 may also receive an indication, or otherwise have previously determined or predicted, that at a timestamp 215 indicating a time of 8:00:00.05, the LiDAR sensor 204 has an orientation 216 of 50 degrees with respect to a reference axis).
Benemann and Lee are considered to be analogous to the claimed invention because they are both in the same field of multi-sensor synchronization and spatial tracking systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the sensor control system of Benemann to include determining the frequency of the scan cycle and determining sensor direction based on the scan cycle frequency of Lee with a reasonable expectation of success. This modification would have been motivated by the desire to accurately predict dynamic field-of-view alignments over time without relying on continuous active polling or manual calibration. By integrating Lee’s teaching of predicting sensor orientation and timing using rotation frequency assumptions and timestamped angles into Benemann’s multi-sensor trigger control pipeline, the system can dynamically compute precise time offsets for triggering data capture exactly when fields of view overlap across repeated scan cycles. A person of ordinary skill in the art would recognize that combining known frequency-based tracking calculations with automated multi-sensor synchronization would yield the predictable result of optimizing field-of-view overlap while reducing computational processing overhead.
Regarding Claim 19, A non-transitory computer-readable medium having stored thereon instructions which ([0105] the memory 1118 of the vehicle computing device 1104 stores a localization component 1120, a perception component 1122, a planning component 1124, one or more system controllers 1126, the timestamp component(s) 110, the sensor association component(s) 418, the sensor synchronization component(s) 512, and/or the image processing component(s) 602), when executed by one or more processors, cause the one or more processors to:
direct at least the FFOV of a first sensor at different regions of space within an environment according to a scan cycle, wherein the first sensor has first field-of-view (FFOV) arranged to capture first sensor data from the environment ([0023] a spinning LIDAR sensor may be associated with a field of view that moves relative to a field of view associated with the image sensor);
determine, a direction of the FFOV of the first sensor, and a direction of a second field-of-view (SFOV) of a second sensor, the second sensor being configured to capture second sensor data from the environment within the second SFOV ([0023] the computing system may determine a first orientation of the first field of view and/or a first pose (position and orientation) associated with the image sensor. Furthermore, the computing system may determine a second orientation of the second field of view and/or a second pose associated with the LIDAR sensor. In some examples, the respective orientations and/or the respective poses associated with the image sensor and/or the LIDAR sensor may be tracked), an estimated amount of time between when a state of a first scan cycle of the first sensor in which a first point within the FFOV of the first sensor is aligned with a second point within the SFOV of the second sensor and when a state of a second scan cycle of the first sensor in which the first point within the FFOV of the first sensor realigns with the second point within the SFOV of the second sensor ([0023] According to some examples, the orientations and/or the poses may be tracked relative to one another. The computing system may use field of view orientation information and/or pose information as an input for causing the image sensor to initiate the rolling shutter image capture of the first field of view, e.g., such that at least a first portion of the first field of view (associated with the image sensor) overlaps at least a second portion of the second field of view (associated with the LIDAR sensor) in accordance with the synchronization condition(s). In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view);
determine a time offset for the second sensor based on the estimated time ([0023] In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view); and
send, to the second sensor, a signal identifying the time offset to trigger the second sensor to capture second sensor data at or after specific time intervals defined by the time offset when the FFOV and SFOV are aligned ([0023] In at least some examples, such initialization may be timed so as to optimize (e.g., maximize) an overlap of the respective fields of view).
Benemann is not relied upon as teaching that the instructions cause the processor to: determine a frequency of the scan cycle of the first sensor; and determine, based on the frequency of the scan cycle of the first sensor, a direction of the FFOV of the first sensor.
However, Lee teaches that the instructions cause the processor to:
determine a frequency of the scan cycle of the first sensor ([0036] the computing device 124 may make such inference, prediction, or determination based on an assumption of a constant rotation speed of the LiDAR sensor 204 over time, for example, 25 degrees every 50 milliseconds); and
determine, based on the frequency of the scan cycle of the first sensor, a direction of the FFOV of the first sensor ([0036] The data may include historical angles and corresponding timestamps of the LiDAR sensor 204. For example, the computing device 124 may receive an indication, or otherwise have previously determined or predicted, that at a timestamp 205 indicating a time of 8:00:00.0, the LiDAR sensor 204 has an orientation 206 of 25 degrees with respect to a reference axis. The computing device 124 may also receive an indication, or otherwise have previously determined or predicted, that at a timestamp 215 indicating a time of 8:00:00.05, the LiDAR sensor 204 has an orientation 216 of 50 degrees with respect to a reference axis).
Benemann and Lee are considered to be analogous to the claimed invention because they are both in the same field of multi-sensor synchronization and spatial tracking systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the sensor control system of Benemann to include determining the frequency of the scan cycle and determining sensor direction based on the scan cycle frequency of Lee with a reasonable expectation of success. This modification would have been motivated by the desire to accurately predict dynamic field-of-view alignments over time without relying on continuous active polling or manual calibration. By integrating Lee’s teaching of predicting sensor orientation and timing using rotation frequency assumptions and timestamped angles into Benemann’s multi-sensor trigger control pipeline, the system can dynamically compute precise time offsets for triggering data capture exactly when fields of view overlap across repeated scan cycles. A person of ordinary skill in the art would recognize that combining known frequency-based tracking calculations with automated multi-sensor synchronization would yield the predictable result of optimizing field-of-view overlap while reducing computational processing overhead.
Claims 2, 4-5, and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Benemann et al. (US 2022/0394156 A1) and Lee et al. (US 2022/0373661 A1) in view of Chen et al. (US 2018/0332240 A1).
Regarding Claim 2, Benemann is not relied upon as teaching that the SFOV of the second sensor is directed at a fixed region of space within the environment.
However, Chen teaches that the SFOV of the second sensor is directed at a fixed region of space within the environment ([0024] In the aforesaid embodiment, the processor 120 applies the image procedure to obtain the first region, and calculates the coordinate of the first region in the image. In one embodiment, the processor 120 records at least one coordinate of the first region in the image. For example, when the first region is a rectangle, the processor 120 records the coordinates of the four vertexes of the first region. It should be noted that, in this embodiment, the processor 120 records the coordinate of the optical sensing module 110 and the image coordinate of the image that the optical sensing module 110 captures. The processor 120 can predict the position the IR light covers in the second image. Therefore, the processor 120 can locate the second region of the second image according to the first region of the first image such that the processor 120 can perform image analyzing at the second region of the second image).
Benemann (as previously modified by Lee) and Chen are considered to be analogous to the claimed invention because they are both in the same field of multi-sensor tracking and environment sensing systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the sensor system of Benemann (as previously modified) to include directing the SFOV of the second sensor at a fixed region of space within the environment as taught by Chen with a reasonable expectation of success. This modification would have been motivated by the desire to establish a static, reliable spatial reference frame for multi-image region analysis. By integrating Chen’s teaching of fixing the secondary optical module’s field of view to a known target coordinate region into Benemann (as previously modified)’s multi-sensor system, the system can predictably map and analyze secondary image coordinates relative to primary scan data without requiring continuous movement tracking for both sensors. A person of ordinary skill in the art would recognize that combining a fixed secondary field of view with a dynamic primary scanner would yield the predictable result of simplifying spatial coordinate mapping and reducing computational overhead during image processing.
Regarding Claims 4 and 13, Benemann is not relied upon as teaching that the first point within the FFOV of the first sensor is on a first plane along a center of the FFOV of the first sensor, and wherein the second point within the SFOV of the second sensor is on a second plane extending from or along a center of the SFOV of the second sensor.
However, Chen teaches that the first point within the FFOV of the first sensor is on a first plane along a center of the FFOV of the first sensor, and wherein the second point within the SFOV of the second sensor is on a second plane extending from or along a center of the SFOV of the second sensor ([0044] the optical sensing module 110 has the first FOV 510 and the second FOV 520. The first FOV 510 and the second FOV 520 have the overlapped portion 535, and there is a bias angle θ between the first FOV 510 and the second FOV 520 Examiner Note: Chen’s disclosure of specific FOVs with a defined bias angle and an overlapped portion implies a geometric center and central planes for each FOV. To determine the “bias angle” and “overlapped portion,” one must establish a reference point or plane along the center of each FOV. [0035] By coordinate transformation, any point (x,y) of the first image can be translated to the point (x+c,y) of the second image Examiner Note: Defining the reference points on central planes of the FOV is a standard geometric approach used to simplify the “coordinate transformation” and constant-based translation disclosed in Chen).
Benemann (as previously modified by Lee) and Chen are considered to be analogous to the claimed invention because they are both in the same field of multi-sensor tracking and environment sensing systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Benemann (as previously modified by Lee) to include establishing reference points on central planes extending along the centers of the respective fields of view, as taught by Chen, with a reasonable expectation of success. This modification would have been motivated by the desire to simplify coordinate transformations and accurately calculate relative bias angles between overlapping sensor fields of view. By integrating Chen’s teaching of utilizing central FOV reference planes to define angular offsets and coordinate transformations into Benemann’s multi-sensor system (As modified by Lee), the system can standardize geometric reference frames for fast spatial mapping across disparate sensors. A person of ordinary skill in the art would recognize that defining reference points along the center planes of overlapping fields of view would yield the predictable result of reducing geometric calculation complexity when translating image coordinates between sensors.
Regarding Claims 5 and 14, Benemann is not relied upon as teaching that the first point within the FFOV of the first sensor is on a first plane that extends from a vertex of a first angle of the FFOV of the first sensor, and wherein the second point within the SFOV of the second sensor is on a second plane that extends from a vertex of a second angle of the SFOV of the second sensor.
However, Chen teaches that the first point within the FFOV of the first sensor is on a first plane that extends from a vertex of a first angle of the FFOV of the first sensor, and wherein the second point within the SFOV of the second sensor is on a second plane that extends from a vertex of a second angle of the SFOV of the second sensor([0044] the optical sensing module 110 has the first FOV 510 and the second FOV 520. The first FOV 510 and the second FOV 520 have the overlapped portion 535, and there is a bias angle θ between the first FOV 510 and the second FOV 520 Examiner Note: Chen’s disclosure of a bias angle θ between fields of view inherently defines the FOVs as angular sectors originating from a vertex. Establishing reference planes that extends from these vertices is a necessary geometric step to define the boundaries and the angular relationship (the bias angle) of the sensors [0037] The motion model calculates model calculates the movement of the object, and whether the object enters within the second image or not can be determined. Examiner Note: Utilizing the planes extending from the vertices of the FOV angles allows the motion model to calculate the exact moment of transition between sensor ranges).
Benemann (as previously modified by Lee) and Chen are considered to be analogous to the claimed invention because they are both in the same field of multi-sensor tracking and environment sensing systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Benemann (as previously modified by Lee) to include establishing reference planes extending from a vertex of an angle of the first field of view and second field of view, as taught by Chen, with a reasonable expectation of success. This modification would have been motivated by the desire to accurately define angular field-of-view boundaries and calculate object entry thresholds between sensor ranges. By integrating Chen’s teaching of utilizing planes extending from FOV vertices to model angular relationships and object motion into Benemann’s multi-sensor system (as modified by Lee), the system can precisely project spatial boundaries to track target movement across overlapping sensor regions. A person of ordinary skill in the art would recognize that establishing reference planes from the vertices of field-of-view angles would yield the predictable result of enabling exact motion-model calculations for predicting when an object enters or exits a sensor’s detection field.
Claims 3, 11-12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Benemann et al. (US 2022/0394156 A1) and Lee et al. (US 2022/0373661 A1) in view of Ma et al. (US 2022/0404478 A1).
Regarding Claims 3, 12, and 20, Benemann is not relied upon as teaching that the one or more processors are configured to:
determine a time delay between a first time when the second sensor initiates an operation to capture the second sensor data and a second time when the second sensor captures the second sensor data, wherein each specific time interval of the specific time intervals comprises the estimated time minus the time delay.
However, Ma teaches that the one or more processors are configured to:
determine a time delay between a first time when the second sensor initiates an operation to capture the second sensor data and a second time when the second sensor captures the second sensor data, wherein each specific time interval of the specific time intervals comprises the estimated time minus the time delay ([0033] Synchronizing the data capturing between a LiDAR sensor and an image-capturing sensor using the self-adaptive LiDAR-camera synchronization system may involve determining a delay timing between initiation of data capture by the LiDAR sensor and initiation of data capture by the image-capturing sensor. The delay timing may include an alignment timing component that accounts for differences in the fields of view and data-capturing ranges of the LiDAR sensor and the image-capturing sensor. The delay timing may additionally or alternatively account for communication times and/or communication latencies between software components and/or hardware components, which may be represented as a packet-capture timing).
Benemann (as previously modified by Lee) and Ma are considered to be analogous to the claimed invention because they are all in the same field of multi-sensor synchronization and spatial tracking systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Benemann (as previously modified) to include determining a time delay between initiation and actual capture, and adjusting the capture interval by subtracting the time delay, as taught by Ma with a reasonable expectation of success. This modification would have been motivated by the desire to account for hardware, software, and communication latencies when triggering sensor data capture. By integrating Ma’s teaching of calculating delay timing to compensate for latencies between software and hardware components into the sensor synchronization system of Benemann and Lee, the system can precisely offset the trigger timing so the actual data capture occurs exactly when the fields of view align. A person of ordinary skill in the art would recognize that adjusting calculated synchronization intervals to compensate for operational time delays would yield the predictable result of preventing timing misalignments and ensuring accurate multi-sensor data correlation.
Regarding Claim 11, Benemann is not relied upon as teaching that each specific time interval of the specific time intervals comprises the amount of time estimated to lapse between the state of the first scan cycle and the state of the second scan cycle.
However, Ma teaches that each specific time interval of the specific time intervals comprises the amount of time estimated to lapse between the state of the first scan cycle and the state of the second scan cycle ([0033] Synchronizing the data capturing between a LiDAR sensor and an image-capturing sensor using the self-adaptive LiDAR-camera synchronization system may involve determining a delay timing between initiation of data capture by the LiDAR sensor and initiation of data capture by the image-capturing sensor. The delay timing may include an alignment timing component that accounts for differences in the fields of view and data-capturing ranges of the LiDAR sensor and the image-capturing sensor. The delay timing may additionally or alternatively account for communication times and/or communication latencies between software components and/or hardware components, which may be represented as a packet-capture timing).
Benemann (as previously modified by Lee) and Ma are considered to be analogous to the claimed invention because they are all in the same field of multi-sensor synchronization and spatial tracking systems. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Benemann (as previously modified) to include configuring each specific time interval of the specific time intervals to comprise the amount of time estimated to lapse between the state of the first scan cycle and the state of the second scan cycle as taught by Ma with a reasonable expectation of success. This modification would have been motivated by the desire to accurately synchronize data capture across dynamic sensor scan cycles by predicting relative cycle timing. By integrating Ma’s teaching of determining alignment timing components and time intervals between sensor capture cycles into Benemann’s multi-sensor control system (as modified by Lee), the system can predictably schedule data acquisition intervals to match recurring scan states across disparate sensors. A person of ordinary skill in the art would recognize that using the estimated elapsed time between scan cycle states to set capture intervals would yield the predictable result of ensuring spatial and temporal field-of-view alignment while compensating for dynamic sensor scanning motion.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EVAN H HAUT whose telephone number is (571)272-7927. The examiner can normally be reached Monday-Thursday 10am-3pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Helal Algahaim can be reached at (571) 272-9358. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.H.H./Patent Examiner, Art Unit 3645
/JAMES R HULKA/Primary Examiner, Art Unit 3645
Prosecution Insights