DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-9 are all the claims pending in the application.
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of Application No. KR10-2022-0034737 and KR10-2023-0020082, filed in Korea on 03/21/2022 and 02/15/2023.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/30/2024 was considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. In particular, Claim 9 is directed to a computer-readable recording medium. The United States Patent and Trademark Office (USPTO) is obliged to give claims their broadest reasonable interpretation consistent with the specification during proceedings before the USPTO. See In re Zletz, 893F.2d 319 (Fed. Cir. 1989) (during patent examination the pending claims must be interpreted as broadly as their terms reasonably allow). The broadest reasonable interpretation of Claim 9 drawn to a computer-readable recording medium covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer-readable media. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). Applicants are advised to amend Claim 9 to recite “A non-transitory computer-readable recording medium…” in order to overcome the rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5, and 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. (U.S. Patent App. Pub No. 2021/0150771 A1, hereafter referred as Huang) in view of David Keilaf et al. (U.S. Patent App. Pub No. 2022/0050203 A1, hereafter referred as Keilaf).
Regarding Claim 1:
Huang teaches a method performed by a lidar point-cloud decoding device for decoding a lidar point cloud (Huang: Par. [0024]; the vehicle computing system can decode the stored LIDAR data), the method comprising: reconstructing a lidar frame from a bitstream by using a video decoding method (Huang: Par. [0047]; to decode, the same entropy model can be used in the arithmetic coder's decoding algorithm; a tree-based data structure can then be built from the decompressed bitstream and used to reconstruct the point cloud, Par. [0060-0061]; the data encoding system can use the intensity entropy model to compress extraneous intensities tied to each spatial point coordinate; the conditioning on the occupancies can emphasize that intensity decoding occurs after the point spatial coordinates have already been reconstructed);
Huang fails to teach post-processing a reconstructed lidar frame; constructing, from the post-processed lidar frame, the lidar point cloud with a converted coordinate system; and reconstructing the lidar point cloud by inversely converting the converted coordinate system of the lidar point cloud.
Keilaf, like Huang, is directed to lidar point cloud coding. Keilaf does teach post-processing a reconstructed lidar frame (Keilaf: Par. [0554]; would detect that LiDAR #2 is non-functional and would designate another LiDAR in the system to compensate for loss of coverage; LiDAR #7 is designated to operate in a backup mode and extend its field of view in order to cover up for LIDAR #2 field of view; increasing the scanning range of LIDAR #7 may occur at the expense of some of its capabilities, decreased total range, resolution or frame rate); constructing, from the post-processed lidar frame, the lidar point cloud with a converted coordinate system; and reconstructing the lidar point cloud by inversely converting the converted coordinate system of the lidar point cloud (Keilaf: Par. [0225]; the at least two differing range measurements may be derived from detection information acquired by two or more detectors of the sensor(s) (e.g., two or more independently sampled SiPM detectors), the at least two differing range measurements may be associated with two differing directions with respect to the LIDAR system, a first range measurement detected by a first detector of the sensor(s) may be converted to a first detection location (e.g., in spherical coordinates), and the second range measurement detected from reflections of the same beam spot by a second detector of the sensor(s) may be converted to a second detection location (e.g., in spherical coordinates), where any combination of at least two pairs of coordinates differ between the first detection location and the second detection location).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to utilize the coordinate conversion, as taught by Keilaf, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Keilaf, the proposed modification would account for a frequency phase-shift measurement and/or a modulation phase shift measurement that may differ from the difference in distance and/or angle of incidence with respect to the different detectors (Keilaf: Par. [0224]).
In regards to Claim 2, Huang as modified by Keilaf further teaches the method of claim 1, wherein post-processing the reconstructed lidar frame includes: removing padding or scaling applied by a lidar point-cloud encoding device (Keilaf: Par. [0554]; would detect that LiDAR #2 is non-functional and would designate another LiDAR in the system to compensate for loss of coverage; LiDAR #7 is designated to operate in a backup mode and extend its field of view in order to cover up for LIDAR #2 field of view; increasing the scanning range of LIDAR #7 may occur at the expense of some of its capabilities, decreased total range, resolution or frame rate).
In regards to Claim 3, Huang as modified by Keilaf further teaches the method of claim 1, wherein the lidar frame has a longitudinal length based on a number of lidar sensors, and a transverse length based on 360 degrees divided by a sampling angle (Keilaf: Par. [0123]; that LIDAR system 100 may include any number of scanning units 104 arranged in any manner, each with an 80° to 120° field of view or less, depending on the number of units employed, a 360-degree horizontal field of view may be also obtained by mounting a multiple LIDAR systems 100 on vehicle 110, each with a single scanning unit 104).
Regarding Claim 5:
Huang as modified by Keilaf further teaches a method performed by a lidar point-cloud encoding device for encoding a lidar point cloud (Huang: Par. [0018]; methods for generating a compressed and encoded representation of point cloud (e.g., LIDAR) data), the method comprising: converting a coordinate system of geometric information of the lidar point cloud; generating a lidar frame from the lidar point cloud converted by the coordinate system (Keilaf: Par. [0225]; the at least two differing range measurements may be derived from detection information acquired by two or more detectors of the sensor(s) (e.g., two or more independently sampled SiPM detectors), the at least two differing range measurements may be associated with two differing directions with respect to the LIDAR system, a first range measurement detected by a first detector of the sensor(s) may be converted to a first detection location (e.g., in spherical coordinates), and the second range measurement detected from reflections of the same beam spot by a second detector of the sensor(s) may be converted to a second detection location (e.g., in spherical coordinates), where any combination of at least two pairs of coordinates differ between the first detection location and the second detection location); preprocessing the lidar frame (Keilaf: Par. [0554]; would detect that LiDAR #2 is non-functional and would designate another LiDAR in the system to compensate for loss of coverage; LiDAR #7 is designated to operate in a backup mode and extend its field of view in order to cover up for LIDAR #2 field of view; increasing the scanning range of LIDAR #7 may occur at the expense of some of its capabilities, decreased total range, resolution or frame rate); and encoding a preprocessed lidar frame by using a video encoding method (Huang: Par. [0005]; method can include generating, by the computing system, a compressed bitstream by sequentially ordering a plurality of compressed representations associated with the plurality of symbols, Par. [0060-0061]; the data encoding system can use the intensity entropy model to compress extraneous intensities tied to each spatial point coordinate; the conditioning on the occupancies can emphasize that intensity decoding occurs after the point spatial coordinates have already been reconstructed).
In regards to Claim 8, Huang as modified by Keilaf further teaches the method of claim 5, wherein preprocessing the lidar frame includes applying padding, or scaling, to the lidar frame to make the lidar frame suitable for encoding the preprocessed lidar frame (Keilaf: Par. [0554]; would detect that LiDAR #2 is non-functional and would designate another LiDAR in the system to compensate for loss of coverage; LiDAR #7 is designated to operate in a backup mode and extend its field of view in order to cover up for LIDAR #2 field of view; increasing the scanning range of LIDAR #7 may occur at the expense of some of its capabilities, decreased total range, resolution or frame rate).
Regarding Claim 9:
Huang as modified by Keilaf further teaches a computer-readable recording medium storing a bitstream generated by a method for encoding a lidar point cloud (Huang: Par. [0018]; methods for generating a compressed and encoded representation of point cloud (e.g., LIDAR) data), the method comprising: converting a coordinate system of geometric information of the lidar point cloud; generating a lidar frame from the lidar point cloud converted by the coordinate system (Keilaf: Par. [0225]; the at least two differing range measurements may be derived from detection information acquired by two or more detectors of the sensor(s) (e.g., two or more independently sampled SiPM detectors), the at least two differing range measurements may be associated with two differing directions with respect to the LIDAR system, a first range measurement detected by a first detector of the sensor(s) may be converted to a first detection location (e.g., in spherical coordinates), and the second range measurement detected from reflections of the same beam spot by a second detector of the sensor(s) may be converted to a second detection location (e.g., in spherical coordinates), where any combination of at least two pairs of coordinates differ between the first detection location and the second detection location); preprocessing the lidar frame (Keilaf: Par. [0554]; would detect that LiDAR #2 is non-functional and would designate another LiDAR in the system to compensate for loss of coverage; LiDAR #7 is designated to operate in a backup mode and extend its field of view in order to cover up for LIDAR #2 field of view; increasing the scanning range of LIDAR #7 may occur at the expense of some of its capabilities, decreased total range, resolution or frame rate); and encoding a preprocessed lidar frame by using a video encoding method (Huang: Par. [0005]; method can include generating, by the computing system, a compressed bitstream by sequentially ordering a plurality of compressed representations associated with the plurality of symbols, Par. [0060-0061]; the data encoding system can use the intensity entropy model to compress extraneous intensities tied to each spatial point coordinate; the conditioning on the occupancies can emphasize that intensity decoding occurs after the point spatial coordinates have already been reconstructed).
Claims 4 and 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. (U.S. Patent App. Pub No. 2021/0150771 A1, hereafter referred as Huang) in view of David Keilaf et al. (U.S. Patent App. Pub No. 2022/0050203 A1, hereafter referred as Keilaf) and Milioto et al. (NPL: RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation, hereafter referred as Milioto).
In regards to Claim 4, Huang as modified by Keilaf fails to further teach the method of claim 3, wherein constructing the lidar point cloud includes: constructing the lidar point cloud with the converted coordinate system (Milioto: 1. Introduction: achieve this by operating on a spherical projection of the input point cloud, i.e., a 2D image representation, similar to a range image, and therefore exploit the way the points are detected by a rotating LiDAR sensor), by using a distance map and a reflectance map contained in the lidar frame; wherein the distance map and the reflectance map are generated by a lidar point-cloud encoding device by sampling the lidar point cloud based on an index and a sampling angle of the lidar sensors (Milioto: 3. Our Approach: (A) a transformation of the input point cloud into a range image representation, (B) a 2D fully convolutional semantic segmentation, (C) a semantic transfer from 2D to 3D that recovers all points from the original point cloud, regardless of the used range image discretization, and (D) an efficient range image based 3D post-processing to clean the point cloud from undesired discretization and inference artifacts, using a fast, GPU-based kNN-search operating on all points), and then projecting distance values and reflection coefficients of sampled points to an index and rotation angle plane of the lidar sensors (Milioto: 3. Our Approach; our first step is to convert each de-skewed point cloud into a range representation, convert each point via a mapping to spherical coordinates and finally to image coordinates, as defined by eq. 1 where (u, v) are said image coordinates, (h, w) are the height and width of the desired range image representation, f = fup + fdown is the vertical field-of-view of the sensor, and r = ||pi||2 is the range of each point, this procedure results in a list of (u, v) tuples containing a pair of image coordinates for each pi, which we use to generate our proxy representation, using these indexes, we extract for each pi, its range r, its x, y, and z coordinates, and its remission, and we store them in the image, creating a [5 × h × w] tensor, because of the de-skewing of the scan, the assignment of each points to its corresponding (u, v) is done in a descending range order, to ensure that all points rendered in the image are in the current field of view of the sensor).
Milioto, like Huang, is directed to lidar point cloud coding. Milioto does teach the method of claim 3, wherein constructing the lidar point cloud includes: constructing the lidar point cloud with the converted coordinate system, by using a distance map and a reflectance map contained in the lidar frame; wherein the distance map and the reflectance map are generated by a lidar point-cloud encoding device by sampling the lidar point cloud based on an index and a sampling angle of the lidar sensors, and then projecting distance values and reflection coefficients of sampled points to an index and rotation angle plane of the lidar sensors
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to utilize the conversion technique, as taught by Milioto, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Milioto, the proposed modification would obtain a more accurate semantic segmentation of the full LiDAR point cloud (Milioto: 3. Our Approach).
In regards to Claim 6, Huang as modified by Keilaf and Milioto further teaches the method of claim 5, wherein generating the lidar frame includes, when the lidar point cloud uses a spherical coordinate system (Milioto: 1. Introduction: achieve this by operating on a spherical projection of the input point cloud, i.e., a 2D image representation, similar to a range image, and therefore exploit the way the points are detected by a rotating LiDAR sensor), generating the lidar frame by distinguishing between a distance map and a reflectance map by sampling the lidar point cloud based on an index and a sampling angle of lidar sensors (Milioto: 3. Our Approach: (A) a transformation of the input point cloud into a range image representation, (B) a 2D fully convolutional semantic segmentation, (C) a semantic transfer from 2D to 3D that recovers all points from the original point cloud, regardless of the used range image discretization, and (D) an efficient range image based 3D post-processing to clean the point cloud from undesired discretization and inference artifacts, using a fast, GPU-based kNN-search operating on all points), and then projecting distance values and reflection coefficients of sampled points to an index and rotation angle plane of the lidar sensors (Milioto: 3. Our Approach; our first step is to convert each de-skewed point cloud into a range representation, convert each point via a mapping to spherical coordinates and finally to image coordinates, as defined by eq. 1 where (u, v) are said image coordinates, (h, w) are the height and width of the desired range image representation, f = fup + fdown is the vertical field-of-view of the sensor, and r = ||pi||2 is the range of each point, this procedure results in a list of (u, v) tuples containing a pair of image coordinates for each pi, which we use to generate our proxy representation, using these indexes, we extract for each pi, its range r, its x, y, and z coordinates, and its remission, and we store them in the image, creating a [5 × h × w] tensor, because of the de-skewing of the scan, the assignment of each points to its corresponding (u, v) is done in a descending range order, to ensure that all points rendered in the image are in the current field of view of the sensor).
In regards to Claim 7, Huang as modified by Keilaf and Milioto further teaches the method of claim 6, wherein the lidar frame has a longitudinal length based on a number of the lidar sensors and a transverse length based on 360 degrees divided by the sampling angle (Keilaf: Par. [0123]; that LIDAR system 100 may include any number of scanning units 104 arranged in any manner, each with an 80° to 120° field of view or less, depending on the number of units employed, a 360-degree horizontal field of view may be also obtained by mounting a multiple LIDAR systems 100 on vehicle 110, each with a single scanning unit 104).
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yuan et al. (U.S. Patent App. Pub No. 2021/0350147 A1) teaches a system identifies a road to be navigated by an ADV, the road being captured by one or more point clouds from one or more LIDAR sensors.
Smolyanskiy et al. (U.S. Patent App. Pub No. 2021/0342609 A1) teaches a deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RENAE BITOR whose telephone number is (703)756-5563. The examiner can normally be reached Monday to Friday: 8:00 - 5:30 but off the 1st Friday of the biweek.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, GREG MORSE can be reached on (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RENAE A BITOR/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights