DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 12, and 23-26 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao (2019, “Robust Normal Estimation for 3D LiDAR Point Clouds in Urban Environments”) in view of Dehlinger US 20210263137 A1.
Regarding claim 1, Zhao teaches a method for identifying Z-planes, comprising:
obtaining data from a time-of-flight (TOF) sensor indicating distance between the TOF sensor and a plurality of surfaces (capture of 3D point cloud data, Section 1. Introduction; datasets in section 3.1);
identifying a basis vector representing a peak direction across the point cloud ( x-y-z coordinate system used, section 3.1 Datasets); and
identifying, in the point cloud, at least one Z-plane representing at least one surface of the plurality of surfaces, wherein the at least one Z-plane is substantially orthogonal to the basis vector (detects ground and roofs parallel to ground in Section 3 Experiment Results and Performance Evaluation).
Zhao does not explicitly teach but Dehlinger teaches the TOF data is raw data, applying an averaging filter to the raw data to smooth the raw data for increasing signal-to-noise ratio (SNR) of surfaces of the plurality of surfaces; performing a depth compute process on the raw data, as filtered, to generate distance data, generating a point cloud based on the distance data, the point cloud in a frame of reference of the sensor ([0049, 70-72]; phase data is averaged to reduce noise, then turned into distance data for a 3D point cloud representation of the field of view).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao such that the TOF data is raw data, applying an averaging filter to the raw data to smooth the raw data for increasing signal-to-noise ratio (SNR) of surfaces of the plurality of surfaces; performing a depth compute process on the raw data, as filtered, to generate distance data, generating a point cloud based on the distance data, the point cloud in a frame of reference of the sensor similar to Dehlinger with a reasonable expectation of success. This would have the predictable result of decreasing error in distance estimates (Dehlinger: [0070-72]).
Regarding claim 12, Zhao teaches imaging system comprising:
obtaining data indicating distances between the TOF sensor and a plurality of surfaces in an environment of the TOF sensor (capture of 3D point cloud data, Section 1. Introduction; datasets in section 3.1); and
identify a basis vector representing a peak direction across the point cloud; and
identify, in the point cloud, at least one Z-plane representing at least one surface of the plurality of surfaces, wherein the at least one Z-plane is substantially orthogonal to the basis vector.
Zhao does not explicitly teach but Dehlinger teaches a TOF sensor configured to obtain the TOF data (LIDAR system, Fig. 1, [0046-48]), the TOF data is raw data (phase data, [0070-72]), a processor (one or more processors, [0049]), apply an averaging filter to the raw data to smooth the raw data for increasing signal-to-noise ratio (SNR) of surfaces of the plurality of surfaces; perform a depth compute process on the raw data, as filtered, to generate distance data, generate a point cloud based on the distance data, the point cloud in a frame of reference of the sensor ([0049, 70-72]; phase data is averaged to reduce noise, then turned into distance data for a 3D point cloud representation of the field of view).
Additionally, TOF sensors, raw data collection, processors, data filtering, and point cloud generation are well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao such that the TOF data is raw data, applying an averaging filter to the raw data to smooth the raw data for increasing signal-to-noise ratio (SNR) of surfaces of the plurality of surfaces; performing a depth compute process on the raw data, as filtered, to generate distance data, generating a point cloud based on the distance data, the point cloud in a frame of reference of the sensor similar to Dehlinger with a reasonable expectation of success. This would have the predictable result of decreasing error in distance estimates (Dehlinger: [0070-72]).
Regarding claim 23, Zhao as modified above teaches the imaging system of claim 12,
Zhao does not explicitly teach but Dehlinger teaches wherein the TOF depth sensor comprises a light source to illuminate the environment of the TOF depth sensor and an image sensor to sense reflected light ([0047-49]).
Additionally, lidar light sources and sensors are well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao such that the TOF depth sensor comprises a light source to illuminate the environment of the TOF depth sensor and an image sensor to sense reflected light similar to Dehlinger with a reasonable expectation of success. This would have the predictable result of allowing the sensor to control light emission and enabling functionality of the lidar sensor.
Regarding claim 24, Zhao as modified above teaches the imaging system of claim 12,
Zhao does not explicitly teach but Dehlinger teaches wherein the TOF depth sensor has an image frame, and the raw data is arranged in a plurality of pixels within the image frame (image frames and pixels, [0046-49]).
Additionally, image frames and pixels are well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao such that the TOF depth sensor comprises a light source to illuminate the environment of the TOF depth sensor and an image sensor to sense reflected light similar to Dehlinger with a reasonable expectation of success. This would have the predictable result of allowing the sensor to collect data and enabling functionality of the lidar sensor.
Regarding claim 25, Zhao as modified above teaches the imaging system of claim 24,
Zhao does not explicitly teach but Dehlinger teaches wherein an individual pixel comprises a distance to one of the plurality of surfaces in the environment of the TOF depth sensor, and the individual pixel has an associated ray direction describing a direction from the TOF depth sensor to the surface (image frames and pixels, [0046-49]; Fig. 1).
Additionally, pixel functions of lidar detectors are well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao such that an individual pixel comprises a distance to one of the plurality of surfaces in the environment of the TOF depth sensor, and the individual pixel has an associated ray direction describing a direction from the TOF depth sensor to the surface similar to Dehlinger with a reasonable expectation of success. This would have the predictable result of allowing the sensor to collect data and enabling functionality of the lidar sensor.
Regarding claim 26, Zhao as modified above teaches the imaging system of claim 25,
Zhao does not explicitly teach but Dehlinger teaches wherein, to generate the point cloud, the processor multiplies the ray direction for the individual pixel by the distance to the one of the plurality of surfaces for the individual pixel ([0045-49]).
Additionally, point cloud generation from pixel data is well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao such that to generate the point cloud, the processor multiplies the ray direction for the individual pixel by the distance to the one of the plurality of surfaces for the individual pixel similar to Dehlinger with a reasonable expectation of success. This would have the predictable result of providing data for objection detection analysis which can help improve safety and efficiency of operations.
Claims 27-28 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao (2019, “Robust Normal Estimation for 3D LiDAR Point Clouds in Urban Environments”) in view of Dehlinger US 20210263137 A1 and further in view of Inoue US 20220146818 A1.
Regarding claim 27, Zhao as modified above teaches the imaging system of claim 12,
Zhao does not explicitly teach but Inoue teaches further comprising a camera to capture an image of the environment of the TOF depth sensor ([0163]).
Additionally, use of multiple kinds of sensors is well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao to include a camera to capture an image of the environment of the TOF depth sensor similar to Inoue with a reasonable expectation of success. This would have the predictable result of providing additional information to improve safety and efficiency of operations.
Regarding claim 28, Zhao as modified above teaches the imaging system of claim 27,
Zhao does not explicitly teach further comprising a display screen, the processor to display, on the display screen, the image captured by the camera and a visual indication of the identified Z-plane.
Inoue teaches use of LiDAR to indicate objects on a camera image display ([0163]).
Additionally, indicating detected objects is well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao to include a display screen, the processor to display, on the display screen, the image captured by the camera and a visual indication of the identified Z-plane similar to Inoue with a reasonable expectation of success. This would have the predictable result of providing additional information to improve safety and efficiency of operations.
Claim 29 is rejected under 35 U.S.C. 103 as being unpatentable over Zhao (2019, “Robust Normal Estimation for 3D LiDAR Point Clouds in Urban Environments”) in view of Dehlinger US 20210263137 A1 and further in view of Chen US 20190281202 A1.
Regarding claim 29, Zhao as modified above teaches the imaging system of claim 12,
Zhao does not explicitly teach but Chen teaches further comprising a light sensor for detecting sunlight in the environment of the TOF depth sensor, wherein the processor applies the filter to the raw data in response to detecting at least a threshold level of sunlight ([0028, 83]).
Additionally, ambient light detectors and filters are well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhao to include a light sensor for detecting sunlight in the environment of the TOF depth sensor, wherein the processor applies the filter to the raw data in response to detecting at least a threshold level of sunlight similar to Inoue with a reasonable expectation of success. This would have the predictable result of decreasing noise and protecting sensors from potentially harmful levels of light.
Allowable Subject Matter
Claims 2-11 and 13-22 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The prior art of record does not explicitly teach nor render obvious the method of claim 2 or system of claim 13, specifically including: identifying the at least one Z-plane includes: generating a height map of the point cloud; generating a profile representation of the height map; and identifying the at least one Z-plane respectively corresponding to at least one peak in the profile representation, wherein each peak of the at least one peak indicates a collection of adjacent points in the point cloud having a same surface normal estimate, and wherein the collection achieves a threshold size.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEPH C FRITCHMAN whose telephone number is (571)272-5533. The examiner can normally be reached M-F 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Isam Alsomiri can be reached on 571-272-6970. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.C.F./Examiner, Art Unit 3645
/ISAM A ALSOMIRI/Supervisory Patent Examiner, Art Unit 3645
Prosecution Insights