DETAILED ACTION
This is the first office action on the merits. Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 5/6/2024 and 4/2/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4-6, 11, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Diederichs et al., US 20210192788 A1 (“Diederichs”) in view of Yang Mao et al., US 20180107904 A1 (“Yang Mao”) and Kato et al., US 20230341749 A1 (“Kato”).
Regarding claims 1, Diederichs discloses A method for determining light detection and ranging (LiDAR) to camera calibration parameters for a vehicle, the method comprising:
collecting pre-alignment LiDAR data including a first plurality of LiDAR data points and pre-alignment camera data including a first plurality of camera data points (Fig. 22A, 2201, 2202, Paragraph [0178]);
determining pre-alignment LiDAR to camera calibration parameters based on the pre-alignment LiDAR data and the pre-alignment camera data (Fig. 22A, 2205, Paragraph [0179]);
collecting deep-alignment LiDAR data including a second plurality of LiDAR data points and deep-alignment camera data including a second plurality of camera data points based at least in part on the pre-alignment LiDAR to camera calibration parameters (Fig. 22B, 2206, 2207, Paragraph [0180]),[…]; and
determining final LiDAR to camera calibration parameters based at least in part on the deep-alignment LiDAR data and the deep-alignment camera data (Fig. 22B, 2208 – 2211, Paragraph [0181]).
Diederichs does not teach: wherein the second plurality of LiDAR data points includes a greater quantity of LiDAR data points than the first plurality of LiDAR data points, and wherein the second plurality of camera data points includes a greater quantity of camera data points than the first plurality of camera data points.
However, Yang Mao teaches an image scanning system that includes a camera and LIDAR sensor (Fig. 1A, first optical unit 11, second optical unit 12, Paragraph [0020]). After deciding a region of interest from the first dataset, a second LIDAR dataset is collected with a greater density of points (Figs. 3A-B, path 146 of multi-point scanning in the ROI 145, Paragraph [0026]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Diederichs’ second LIDAR and camera datasets by collecting higher density second LIDAR datasets, which is disclosed by Yang Mao. One of ordinary skill in the art would have been motivated to make this modification in order to “[obtain] higher image resolutions, faster scanning speeds and more accurate recognition results”, as suggested by Yang Mao (Abstract).
In addition, Kato teaches a control method for a gating camera that captures images of objects in a field of view. The gating camera is controlled to capture a first plurality of images using two exposures and to capture a second plurality of images using four exposures (Fig. 20A, RNG1, RNG-N, Paragraph [0202]). Thus, the second plurality of images has a greater quantity of images than the first plurality of images.
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the second plurality of images discloses by Diederichs and Yang Mao by adding additional exposures, which is disclosed by Kato. One of ordinary skill in the art would have been motivated to make this modification in order to accumulate a larger amount of reflected light which improves the image quality, as suggested by Kato (Paragraph [0063]).
Regarding claim 2, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 1, wherein collecting the pre-alignment LiDAR data and the pre-alignment camera data further comprises:
maneuvering the vehicle through a parking lot using a predetermined driving path (Diederichs, Fig. 1, trajectory 198, Paragraph [0081]; See also Paragraph [0073], Fig. 9, trajectory planning 910);
collecting the first plurality of LiDAR data points using a LiDAR sensor (Diederichs, Fig. 22A, 2201, Paragraph [0178]) while the vehicle is following the predetermined driving path (Diederichs, Fig. 8, Paragraph [0119]: LIDAR operates as AV travels); and
collecting the first plurality of camera data points using a camera (Diederichs, Fig. 22A, 2202, Paragraph [0178]) while the vehicle is following the predetermined driving path (Diederichs, Fig. 7, image 702, Paragraph [0118]).
Regarding claim 4, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 2, wherein collecting the pre-alignment LiDAR data further comprises:
measuring the first plurality of LiDAR data points, wherein each of the first plurality of LiDAR data points corresponds to a location of one of a plurality of edges of one of a plurality of objects in an environment surrounding the vehicle (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1601, Paragraph [0144]-[0145]), and wherein one or more of the first plurality of LiDAR data points has a location outside of a field-of-view of the camera (Diederichs, Fig. 7, FOV of image 702 and data points 704 are different, Paragraph [0118]).
Regarding claim 5, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 4, wherein collecting the pre-alignment camera data further comprises:
capturing a first plurality of images using the camera, wherein the first plurality of images includes a first quantity of images (Diederichs, Fig. 22A, 2202, Paragraph [0178]);
generating a first plurality of bounding boxes on each of the first plurality of images, wherein each of the first plurality of bounding boxes identifies one of the plurality of objects in the first plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145], [0157]);
detecting the plurality of edges of each of the plurality of objects in the plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145]); and
determining the first plurality of camera data points, wherein each of the first plurality of camera data points corresponds to one of the plurality of edges which does not overlap with any of the first plurality of bounding boxes (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0155]-[0157]).
Regarding claim 6, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 5, wherein determining pre-alignment LiDAR to camera calibration parameters further comprises:
determining a pre-alignment spatial transformation necessary to align the first plurality of LiDAR data points with the first plurality of camera data points (Diederichs, Fig. 15, extrinsic calibration module 1505, Paragraph [0158],[0163]); and
determining the pre-alignment LiDAR to camera calibration parameters based at least in part on the pre-alignment spatial transformation (Diederichs, Fig. 22A, 2205, Paragraph [0179]).
Regarding claim 11, Diederichs discloses A system for determining light detection and ranging (LiDAR) to camera calibration parameters for a vehicle, the system comprising:
a LiDAR sensor (Fig. 5, LIDAR 502a, Paragraph [0112]);
a camera (Fig. 5, camera 502c, Paragraph [0144]);
a controller in electrical communication with the LiDAR sensor and the camera (Fig. 4, control module 406, Paragraph [0107]), wherein the controller is programmed to:
determine a pre-alignment LiDAR to camera calibration parameter using a pre-alignment procedure, […];
apply the pre-alignment LiDAR to camera calibration parameter to decrease an initial LiDAR to camera alignment error (Fig. 22A, 2201, 2202, Paragraph [0178]); and
determine a final LiDAR to camera calibration parameter using a deep-alignment procedure (Fig. 22B, 2208 – 2211, Paragraph [0181]), […].
Diederichs does not teach: wherein the pre-alignment procedure is performed using a sparse dataset and wherein the deep-alignment procedure is performed using a dense dataset.
However, Yang Mao teaches an image scanning system that includes a camera and LIDAR sensor (Fig. 1A, first optical unit 11, second optical unit 12, Paragraph [0020]). After deciding a region of interest from the first dataset, a second LIDAR dataset is collected with a greater density of points (Figs. 3A-B, path 146 of multi-point scanning in the ROI 145, Paragraph [0026]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Diederichs’ second LIDAR and camera datasets by collecting higher density second LIDAR datasets, which is disclosed by Yang Mao. One of ordinary skill in the art would have been motivated to make this modification in order to “[obtain] higher image resolutions, faster scanning speeds and more accurate recognition results”, as suggested by Yang Mao (Abstract).
In addition, Kato teaches a control method for a gating camera that captures images of objects in a field of view. The gating camera is controlled to capture a first plurality of images using two exposures and to capture a second plurality of images using four exposures (Fig. 20A, RNG1, RNG-N, Paragraph [0202]). Thus, the second plurality of images has a greater quantity of images than the first plurality of images.
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the second plurality of images discloses by Diederichs and Yang Mao by adding additional exposures, which is disclosed by Kato. One of ordinary skill in the art would have been motivated to make this modification in order to accumulate a larger amount of reflected light which improves the image quality, as suggested by Kato (Paragraph [0063]).
Regarding claim 18, Diederichs discloses A method for determining light detection and ranging (LiDAR) to camera calibration parameters for a vehicle, the method comprising:
maneuvering the vehicle through a parking lot using a predetermined driving path (Fig. 1, trajectory 198, Paragraph [0081]; See also Paragraph [0073], Fig. 9, trajectory planning 910);
collecting a first plurality of LiDAR data points using a LiDAR sensor while the vehicle is following the predetermined driving path (Fig. 22A, 2201, Paragraph [0178]; Fig. 8, Paragraph [0119]: LIDAR operates as AV travels);
collecting a first plurality of camera data points using a camera while the vehicle is following the predetermined driving path (Fig. 22A, 2202, Paragraph [0178]; Fig. 7, image 702, Paragraph [0118]);
determining pre-alignment LiDAR to camera calibration parameters based on the first plurality of LiDAR data points and the first plurality of camera data points (Fig. 22A, 2205, Paragraph [0179]);
collecting a second plurality of LiDAR data points using the LiDAR sensor while the vehicle is following the predetermined driving path (Fig. 22B, 2206, Paragraph [0180]; Fig. 8, Paragraph [0119]: LIDAR operates as AV travels), […];
collecting a second plurality of camera data points using the camera while the vehicle is following the predetermined driving path (Fig. 22B, 2207, Paragraph [0180]; Fig. 7, image 702, Paragraph [0118]), […];
performing a spatial transformation on the second plurality of LiDAR data points based at least in part on the pre-alignment LiDAR to camera calibration parameters (Fig. 22A, 2205, Paragraph [0179]); and
determining final LiDAR to camera calibration parameters based at least in part on the second plurality of LiDAR data points and the second plurality of camera data points (Fig. 22B, 2211, Paragraph [0181]).
Diederichs does not teach: wherein the second plurality of LiDAR data points includes a greater quantity of LiDAR data points than the first plurality of LiDAR data points and wherein the second plurality of camera data points includes a greater quantity of camera data points than the first plurality of camera data points.
However, Yang Mao teaches an image scanning system that includes a camera and LIDAR sensor (Fig. 1A, first optical unit 11, second optical unit 12, Paragraph [0020]). After deciding a region of interest from the first dataset, a second LIDAR dataset is collected with a greater density of points (Figs. 3A-B, path 146 of multi-point scanning in the ROI 145, Paragraph [0026]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Diederichs’ second LIDAR and camera datasets by collecting higher density second LIDAR datasets, which is disclosed by Yang Mao. One of ordinary skill in the art would have been motivated to make this modification in order to “[obtain] higher image resolutions, faster scanning speeds and more accurate recognition results”, as suggested by Yang Mao (Abstract).
In addition, Kato teaches a control method for a gating camera that captures images of objects in a field of view. The gating camera is controlled to capture a first plurality of images using two exposures and to capture a second plurality of images using four exposures (Fig. 20A, RNG1, RNG-N, Paragraph [0202]). Thus, the second plurality of images has a greater quantity of images than the first plurality of images.
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the second plurality of images discloses by Diederichs and Yang Mao by adding additional exposures, which is disclosed by Kato. One of ordinary skill in the art would have been motivated to make this modification in order to accumulate a larger amount of reflected light which improves the image quality, as suggested by Kato (Paragraph [0063]).
Regarding claim 19, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 18, wherein collecting the first plurality of LiDAR data points, the second plurality of LiDAR data points, the first plurality of camera data points, and the second plurality of camera data points further comprises:
measuring the first and second plurality of LiDAR data points, wherein each of the first and second plurality of LiDAR data points corresponds to a location of one of a plurality of edges of one of a plurality of objects in an environment surrounding the vehicle (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1601, Paragraph [0144]-[0145]), and wherein one or more of the first and second plurality of LiDAR data points has a location outside of a field-of-view of the camera (Diederichs, Fig. 7, FOV of image 702 and data points 704 are different, Paragraph [0118]);
capturing a first plurality of images and a second plurality of images using the camera, wherein the second plurality of images includes a greater quantity of images than the first plurality of images (Kato, Fig. 20A, RNG1, RNG-N, Paragraph [0202]);
generating a first plurality of bounding boxes on each of the first plurality of images and a second plurality of bonding boxes on each of the second plurality of images, wherein each of the first and second plurality of bounding boxes identifies one of the plurality of objects in the first plurality of images and the second plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145], [0157]);
detecting a first plurality of edges of each of the plurality of objects in the first plurality of images and a second plurality of edges of each of the plurality of objects in the second plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145]);
determining the first plurality of camera data points, wherein each of the first plurality of camera data points corresponds to one of the first plurality of edges which does not overlap with any of the first plurality of bounding boxes (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0155]-[0157]); and
determining the second plurality of camera data points, wherein each of the second plurality of camera data points corresponds to one of the second plurality of edges which does not overlap with any of the second plurality of bounding boxes (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0155]-[0157]).
Regarding claim 20, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 19, wherein determining the final LiDAR to camera calibration parameters further comprises:
determining a final spatial transformation necessary to align the second plurality of LiDAR data points with the second plurality of camera data points (Diederichs, Fig. 15, extrinsic calibration module 1505, Paragraph [0158],[0163]; Fig. 22B, 2209, Paragraph [0181]); and
determining the final LiDAR to camera calibration parameters based at least in part on the final spatial transformation (Diederichs, Fig. 22B, 2209 – 2211, Paragraph [0181]).
Claims 3, 7-10, and 12-17are rejected under 35 U.S.C. 103 as being unpatentable over Diederichs in view of Yang Mao and Kato in further view of Portnoy et al., US 20220414923 A1 (“Portnoy”).
Regarding claim 3, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 2, wherein maneuvering the vehicle through the parking lot using the predetermined driving path further comprises:
driving the vehicle through an aisle of the parking lot at less than or equal to a predetermined maximum speed, wherein the parking lot is populated with a plurality of parked vehicles (Diederichs, Fig. 1, trajectory 198, Paragraph [0081] Fig. 9, trajectory planning, speed constraints 912, Paragraph [0121]; See also Paragraph [0073]); and
[…].
Diederichs, as modified in view of Yang Mao and Kato, does not teach: maneuvering the vehicle using the predetermined driving path, wherein the predetermined driving path is an S-shaped driving path.
However, Portnoy teaches a vehicle that performs LIDAR and camera alignment. The vehicle moves in an S-shaped driving path while scanning stationary objects (Fig. 3, vehicle 10, Paragraph [0026]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Diederichs’ predetermined driving path by shaping the path in an S-shape, which is disclosed by Portnoy. One of ordinary skill in the art would have been motivated to make this modification in order to “allow for accurate alignment between a lidar and a camera of vehicles in target-less environments”, as suggested by Portnoy (Paragraph [0002]).
Regarding claim 7, Diederichs, as modified in view of Yang Mao and Kato, discloses The method of claim 1, wherein collecting the deep-alignment LiDAR data and the deep-alignment camera data further comprises:
driving the vehicle through an aisle of a parking lot at less than or equal to a predetermined maximum speed, wherein the parking lot is populated with a plurality of parked vehicles (Diederichs, Fig. 1, trajectory 198, Paragraph [0081]; Fig. 9, trajectory planning, speed constraints 912, Paragraph [0121]; See also Paragraph [0073]);
[…];
collecting the second plurality of LiDAR data points using a LiDAR sensor (Diederichs, Fig. 22B, 2206, Paragraph [0180])
performing a spatial transformation on the second plurality of LiDAR data points based at least in part on the pre-alignment LiDAR to camera calibration parameters (Diederichs, Fig. 22B, 2209, Paragraph [0181]); and
collecting the second plurality of camera data points using a camera (Diederichs, Fig. 22B, 2207, Paragraph [0180])
Diederichs, as modified in view of Yang Mao and Kato, does not teach: maneuvering the vehicle using a predetermined driving path, wherein the predetermined driving path is an S-shaped driving path.
However, Portnoy teaches a vehicle that performs LIDAR and camera alignment. The vehicle moves in an S-shaped driving path while scanning stationary objects (Fig. 3, vehicle 10, Paragraph [0026]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Diederichs’ predetermined driving path by shaping the path in an S-shape, which is disclosed by Portnoy. One of ordinary skill in the art would have been motivated to make this modification in order to “allow for accurate alignment between a lidar and a camera of vehicles in target-less environments”, as suggested by Portnoy (Paragraph [0002]).
Regarding claim 8, Diederichs, as modified in view of Yang Mao and Kato and Portnoy, discloses The method of claim 7, wherein collecting the deep-alignment LiDAR data further comprises:
measuring the second plurality of LiDAR data points (Diederichs, Fig. 22B, 2206, Paragraph [0180]), wherein each of the second plurality of LiDAR data points corresponds to a location of one of a plurality of edges of one of a plurality of objects in an environment surrounding the vehicle (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1601, Paragraph [0144]-[0145]), and wherein one or more of the second plurality of LiDAR data points has a location outside of a field-of-view of the camera (Diederichs, Fig. 7, FOV of image 702 and data points 704 are different, Paragraph [0118]).
Regarding claim 9, Diederichs, as modified in view of Yang Mao and Kato and Portnoy, discloses The method of claim 8, wherein collecting the deep-alignment camera data further comprises:
capturing a second plurality of images using the camera, wherein the second plurality of images includes a second quantity of images (Diederichs, Fig. 22B, 2207, Paragraph [0180]);
generating a second plurality of bounding boxes on each of the second plurality of images, wherein each of the second plurality of bounding boxes identifies one of the plurality of objects in the second plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145], [0157]);
detecting the plurality of edges of each of the plurality of objects in the second plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145]); and
determining the second plurality of camera data points, wherein each of the second plurality of camera data points corresponds to one of the plurality of edges which does not overlap with any of the second plurality of bounding boxes (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0155]-[0157]).
Regarding claim 10, Diederichs, as modified in view of Yang Mao and Kato and Portnoy, discloses The method of claim 9, wherein determining the final LiDAR to camera calibration parameters further comprises:
determining a final spatial transformation necessary to align the second plurality of LiDAR data points with the second plurality of camera data points (Diederichs, Fig. 15, extrinsic calibration module 1505, Paragraph [0158],[0163]; Fig. 22B, 2209, Paragraph [0181]); and
determining the final LiDAR to camera calibration parameters based at least in part on the final spatial transformation (Diederichs, Fig. 22B, 2209 – 2211, Paragraph [0181]).
Regarding claim 12, Diederichs, as modified in view of Yang Mao and Kato, discloses The system of claim 11, wherein to determine the pre-alignment LiDAR to camera calibration parameter, the controller is further programmed to:
collect a first plurality of LiDAR data points using the LiDAR sensor (Diederichs, Fig. 22A, 2201, Paragraph [0178]) while the vehicle is following an
collect a first plurality of camera data points using the camera (Diederichs, Fig. 22A, 2202, Paragraph [0178]) while the vehicle is following the parking lot populated with the plurality of parked vehicles (Diederichs, Fig. 1, trajectory 198, Paragraph [0081]; See also Paragraph [0073]).
Diederichs, as modified in view of Yang Mao, does not teach: an S-shaped driving path.
However, Portnoy teaches a vehicle that performs LIDAR and camera alignment. The vehicle moves in an S-shaped driving path while scanning stationary objects (Fig. 3, vehicle 10, Paragraph [0026]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Diederichs’ predetermined driving path by shaping the path in an S-shape, which is disclosed by Portnoy. One of ordinary skill in the art would have been motivated to make this modification in order to “allow for accurate alignment between a lidar and a camera of vehicles in target-less environments”, as suggested by Portnoy (Paragraph [0002]).
Regarding claim 13, Diederichs, as modified in view of Yang Mao and Kato and Portnoy, discloses The system of claim 12, wherein to collect the first plurality of LiDAR data points and the first plurality of camera data points, the controller is further programmed to:
measure the first plurality of LiDAR data points, wherein each of the first plurality of LiDAR data points corresponds to a location of one of a plurality of edges of one of the [objects] in the parking lot (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1601, Paragraph [0144]-[0145]), and wherein one or more of the first plurality of LiDAR data points has a location outside of a field-of-view of the camera (Diederichs, Fig. 7, FOV of image 702 and data points 704 are different, Paragraph [0118]);
capture a first plurality of images using the camera (Diederichs, Fig. 22A, 2202, Paragraph [0178]);
generate a first plurality of bounding boxes on each of the first plurality of images, wherein each of the first plurality of bounding boxes identifies one of the plurality of [objects] in the first plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145], [0157]);
detect the plurality of edges of each of the plurality of [objects] in the first plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145]); and
determine the first plurality of camera data points, wherein each of the first plurality of camera data points corresponds to one of the plurality of edges which does not overlap with any of the first plurality of bounding boxes (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0155]-[0157]).
Diederichs, as modified in view of Yang Mao and Portnoy, does not teach: the identified objects being parked vehicles.
However, Yang Mao teaches an image scanning system that includes a camera and LIDAR sensor (Fig. 1A, first optical unit 11, second optical unit 12, Paragraph [0020]). Yang Mao selects a boundary around a car as an object of interest in the LIDAR and camera images (Fig. 3A, ROI 145, Paragraph [0024]).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Diederichs’ calibrating object by using a car instead of a calibration target, which is disclosed by Yang Mao. One of ordinary skill in the art could have substituted one object for another and the results would have been predictable.
Regarding claim 14, Diederichs, as modified in view of Yang Mao and Kato and Portnoy, discloses The system of claim 13, wherein to determine the pre-alignment LiDAR to camera calibration parameter, the controller is further programmed to:
determine a pre-alignment spatial transformation necessary to align the first plurality of LiDAR data points with the first plurality of camera data points (Diederichs, Fig. 15, extrinsic calibration module 1505, Paragraph [0158],[0163]); and
determine the pre-alignment LiDAR to camera calibration parameter based at least in part on the pre-alignment spatial transformation (Diederichs, Fig. 22A, 2205, Paragraph [0179]).
Regarding claim 15, Diederichs, as modified in view of Yang Mao and Kato and Portnoy, discloses The system of claim 14, wherein to determine the final LiDAR to camera calibration parameter, the controller is further programmed to:
collect a second plurality of LiDAR data points using the LiDAR sensor (Diederichs, Fig. 22B, 2206, Paragraph [0180]) while the vehicle is following the S-shaped driving path (Portnoy, Fig. 3, vehicle 10, Paragraph [0026]) through the aisle of the parking lot populated with the plurality of parked vehicles (Diederichs, Paragraph [0073]), wherein the second plurality of LiDAR data points includes a greater quantity of LiDAR data points than the first plurality of LiDAR data points (Yang Mao, Figs. 3A-B, path 146 of multi-point scanning in the ROI 145, Paragraph [0026]); and
collect a second plurality of camera data points using the camera (Diederichs, Fig. 22B, 2207, Paragraph [0180]) while the vehicle is following the S-shaped driving path (Portnoy, Fig. 3, vehicle 10, Paragraph [0026]) through the aisle of the parking lot populated with the plurality of parked vehicles (Diederichs, Paragraph [0073]), wherein the second plurality of camera data points includes a greater quantity of camera data points than the first plurality of camera data points (Yang Mao, Fig. 8, S23, Paragraph [0050]).
Regarding claim 16, Diederichs, as modified in view of Yang Mao and Kato and Portnoy, discloses The system of claim 15, wherein to collect the second plurality of LiDAR data points and the second plurality of camera data points, the controller is further programmed to:
measure the second plurality of LiDAR data points (Diederichs, Fig. 22B, 2206, Paragraph [0180]), wherein each of the second plurality of LiDAR data points corresponds to a location of one of a plurality of edges of one of the plurality of parked vehicles in the parking lot (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1601, Paragraph [0144]-[0145], Paragraph [0073]; Yang Mao, Fig. 3A, ROI 145, Paragraph [0024]: object can be car), and wherein one or more of the second plurality of LiDAR data points has a location outside of a field-of-view of the camera (Diederichs, Fig. 7, FOV of image 702 and data points 704 are different, Paragraph [0118]);
capture a second plurality of images using the camera (Diederichs, Fig. 22B, 2207, Paragraph [0180]);
generate a second plurality of bounding boxes on each of the second plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145], [0157]), wherein each of the second plurality of bounding boxes identifies one of the plurality of parked vehicles in the second plurality of images (Yang Mao, Fig. 3A, ROI 145, Paragraph [0024]: object can be car);
detect the plurality of edges of each of the plurality of parked vehicles in the second plurality of images (Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0144]-[0145]); and
determine the second plurality of camera data points, wherein each of the second plurality of camera data points corresponds to one of the plurality of edges which does not overlap with any of the second plurality of bounding boxes ((Diederichs, Fig. 15, feature extraction module 1504, Fig. 16, 1602, 1603, Paragraph [0155]-[0157])).
Regarding claim 17, Diederichs, as modified in view of Yang Mao and Portnoy, discloses The system of claim 16, wherein to determine the final LiDAR to camera calibration parameter, the controller is further programmed to:
determine a final spatial transformation necessary to align the second plurality of LiDAR data points with the second plurality of camera data points (Diederichs, Fig. 15, extrinsic calibration module 1505, Paragraph [0158],[0163]; Fig. 22B, 2209, Paragraph [0181]); and
determine the final LiDAR to camera calibration parameter based at least in part on the final spatial transformation (Diederichs, Fig. 22B, 2209 – 2211, Paragraph [0181]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL N NGUYEN whose telephone number is (571)270-5405. The examiner can normally be reached Monday - Friday 8 am - 5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Yuqing Xiao can be reached at (571) 270-3603. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL NGUYEN/Examiner, Art Unit 3645
/YUQING XIAO/Supervisory Patent Examiner, Art Unit 3645
Prosecution Insights