DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office action is in response to amendment filed 2/26/2026.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1 and 11, the phrase “any” renders the claims vague and indefinite because it is unclear whether the requirement is for the fourth point cloud to have higher resolution than at least one, or all, of the listed point clouds. In addition, the newly amended claimed limitations “(thereby increasing point density)” in claims 1 and “(including at least one of Chamfer Distance, Earth Mover's Distance, or F1-score)” in claim 20 render the claims vague and indefinite because it is unclear whether the limitations recited within the parentheses are part of the claim or not.
Claims 2-10 and 12-19 are also rejected because of depending on claims 1 and 11, respectively, containing the same deficiency.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over Amirloo Abolfathi et al. (US 2020/0158869 A1, hereinafter Amirloo) in view of Liang et al. (US 2020/0025931 A1, hereinafter Liang).
Regarding claim 1, Amirloo discloses a computer-implemented method for generating a high-resolution point cloud ([0008], method of generating a high resolution and high accuracy point cloud), the method comprising: generating a first point cloud based on first sensor data (figure 4A, 404 and [0079], receives a first point cloud obtained by a first sensor system); generating a second point cloud based on at least one of a second sensor data and a third sensor data (figure 4A and [0101], the pre-processing module 530 to generate a high resolution and high accuracy point cloud in real-time or near real-time using the low resolution LiDAR system 510 and the camera system 320); combining with the second point cloud, to result in a third point cloud ([0133], generates a corrected first point cloud and optionally a corrected second point cloud based on a respectively pre-trained correction function for camera system 320 or low resolution LiDAR system); and generating a fourth point cloud ([0136], generates a representation of the environment, typically in 3D such as a 3D map, based on the result/output of one or more point cloud application algorithms). Although Amirloo discloses a system, device and method of generating high resolution and high accuracy point cloud, Amirloo does not explicitly disclose the steps of generating a semantic occupancy grid based on the first point cloud; combining the semantic occupancy grid with the second point cloud, to result in a third point cloud; and generating a fourth point cloud by an image synthesis neural network; configured to generate additional 3D points by increasing point density, based on the third point cloud, such that resolution of the fourth point cloud is higher than resolution of any one of the first point cloud, the second point cloud and the third point cloud. However, Liang teaches a an object detection system including the steps of generating a semantic occupancy grid based on the first point cloud ([0044], generating a LIDAR three-dimensional grid with occupancy and intensity features based on the binary occupancy feature maps and the intensity feature map); combining the semantic occupancy grid with the second point cloud, to result in a third point cloud ([0102], fusion system 208 can be further configured to execute at a machine-learned neural network within BEV system 226, one or more continuous convolutions to fuse the image features from the first data stream (e.g., image stream 222) with the LIDAR features from the second data stream (e.g., BEV stream 224) to generate a feature map 228 that includes the fused image features and LIDAR features); and generating a fourth point cloud by an image synthesis neural network ([0103], the detector system 210 can include a machine-learned detector model 230 configured to receive the map-modified LIDAR data and/or feature map as input and, in response to receiving the map-modified LIDAR data, to generate as output a plurality of detector outputs 232); configured to generate additional 3D points by increasing point density, based on the third point cloud, such that resolution of the fourth point cloud is higher than resolution of any one of the first point cloud, the second point cloud and the third point cloud ([0182]-[0183], fusing information at 656 from the plurality of source data points in the one or more fusion layers to generate an output feature at the target data point can include concatenating a plurality of LIDAR features associated with the LIDAR point cloud data at the plurality of source data points, and each multi-layer perceptron can be configured to encode an offset between each of the source data points associated with the LIDAR point cloud data and the target data point associated with the image data such that the target data point has higher resolution) in order to improve object detection for autonomous driving applications ([0036]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Amirloo in having the steps of generating a semantic occupancy grid based on the first point cloud; combining the semantic occupancy grid with the second point cloud, to result in a third point cloud; and generating a fourth point cloud by an image synthesis neural network; configured to generate additional 3D points by increasing point density, based on the third point cloud, such that resolution of the fourth point cloud is higher than resolution of any one of the first point cloud, the second point cloud and the third point cloud, as per teaching of Liang, in order to improve object detection for autonomous driving applications.
Regarding claim 2, Amirloo discloses that each of the first sensor data, the second sensor data and the third sensor data is output by a respective type of sensor device ([0065], sensor data 182 may comprise image data from the cameras 112, 3D data from the LiDAR units 114, RADAR data from the SAR units 116, IMU data from the IMU 118, compass data from the electronic compass 119, and other sensor data from other vehicle sensors 120).
Regarding claims 3-4, Amirloo discloses that the first sensor data is output by a radar sensor, wherein the first sensor data is 3D radar data ([0049], one or more radar units such as synthetic aperture radar (SAR) units 116).
Regarding claim 5, Amirloo discloses that the second sensor data is output by a LiDAR sensor ([0049], one or more light detection and ranging (LiDAR) units 114) .
Regarding claim 6, Amirloo discloses that the third sensor data is output by a camera (figure 3, 320).
Regarding claim 7, Amirloo does not specifically disclose that generating the semantic occupancy grid comprises generating an intermediate occupancy grid based on the first point cloud, and generating the semantic occupancy grid using a first classification neural network, based that operates on the intermediate occupancy grid to assign per-cell semantic labels. However, Liang teaches to generate a LIDAR three-dimensional grid with occupancy and intensity features based on the binary occupancy feature maps and the intensity feature map, and configure to extract a semantic road region mask from a high definition map to rasterize the semantic road region mask onto the bird's eye view representation of the LIDAR point cloud data as a binary road mask channel ([0058]) in order to improve object detection for autonomous driving applications. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Amirloo in generating the semantic occupancy grid comprises generating an intermediate occupancy grid based on the first point cloud, and generating the semantic occupancy grid using a first classification neural network, based that operates on the intermediate occupancy grid to assign per-cell semantic labels, as per teaching of Liang, in order to improve object detection for autonomous driving applications.
Regarding claim 8, Amirloo discloses that each point is defined by its position in space represented by x, y, and z coordinates and its color characteristics when point cloud is a camera-based point cloud ([0080]) such that result in the second point cloud as a 3D colored point cloud. Amirloo differs from the claimed invention in not specifically disclosing the steps of generating the second point cloud comprises generating a semantic mask using a second classification neural network, based on the third sensor data, and projecting the semantic mask in the first point cloud into 3D and associating it with LiDAR points. However, Liang teaches to rasterize the semantic road region mask onto the bird's eye view representation of the LIDAR point cloud data as a binary road mask channel and to concatenate the binary road mask channel with the LIDAR three-dimensional grid along a z-axis in order to improve in a manner that yields more accurate and robust detection performance ([0058]-[0061]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Amirloo in generating the second point cloud comprises generating a semantic mask using a second classification neural network, based on the third sensor data, and projecting the semantic mask in the first point cloud into 3D and associating it with LiDAR points, as per teaching of Liang, in order to improve in a manner that yields more accurate and robust detection performance.
Regarding claim 9, Amirloo discloses that the second point cloud is a colored point cloud comprising per-point semantic attributes ([0058], each point is defined by its position in space represented by x, y, and z coordinates and its color characteristics).
Regarding claim 10, Amirloo differs from the claimed invention in not specifically disclosing the steps of combining the semantic occupancy grid with the second point cloud comprises painting each datapoint in the second point cloud with information from its corresponding data point voxel in the semantic occupancy grid, to produce an enhanced 3D colored point cloud. However, Liang teaches to generate a LIDAR three-dimensional grid with occupancy and intensity features based on the binary occupancy feature maps and the intensity feature map, and the fusion system can be configured to modify the discretized representation of the LIDAR point cloud data based on the semantic road prior data by extracting a semantic road region mask from a high definition map and rasterizing the semantic road region mask onto the LIDAR point cloud data as a binary road mask channel ([0100]) such that a bird's eye view (BEV) representation as this provides a compact representation that enables efficient inference (figure 5 and [0116]-[0119]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Amirloo in combining the semantic occupancy grid with the second point cloud comprises painting each datapoint in the second point cloud with information from its corresponding data point voxel in the semantic occupancy grid, to produce an enhanced 3D colored point cloud, as per teaching of Liang, in order to generate increased density BEV features.
Regarding claim 11, the limitations of the claim are rejected as the same reasons as set forth in claim 1.
Regarding claim 12, the limitations of the claim are rejected as the same reasons as set forth in claim 2.
Regarding claims 13-14, the limitations of the claim are rejected as the same reasons as set forth in claims 3-4.
Regarding claim 15, the limitations of the claim are rejected as the same reasons as set forth in claim 5.
Regarding claim 16, the limitations of the claim are rejected as the same reasons as set forth in claim 7.
Regarding claim 17, the limitations of the claim are rejected as the same reasons as set forth in claim 8.
Regarding claim 18, the limitations of the claim are rejected as the same reasons as set forth in claim 9.
Regarding claim 19, the limitations of the claim are rejected as the same reasons as set forth in claim 10.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Liang et al. (US 2020/0025931 A1, hereinafter Liang) in view of Amirloo Abolfathi et al. (US 2020/0158869 A1, hereinafter Amirloo).
Regarding claim 20, Liang discloses a method for training an image synthesis neural network (figure 2, 200) comprising a first network (figure 2, 218) and a second network (figure 2, 224), the method comprising: creating a mask using feature values of an input training point cloud ([0099], map estimation system 218 is configured to provide LIDAR point cloud data 256 as input to machine-learned map estimation model 250 and the machine-learned map estimation model 250 is configured to generate one or more map estimation outputs 260 in response to receipt of the LIDAR point cloud data 256 received as input); applying the mask to the input training point cloud, resulting in a modified training point cloud ([0100], fusion system 208 can be configured to modify the discretized representation of the LIDAR point cloud data based on the semantic road prior data by extracting a semantic road region mask from a high definition map and rasterizing the semantic road region mask onto the LIDAR point cloud data as a binary road mask channel); inputting the modified training point cloud to the first network to generate a coarse training point cloud ([0101], the machine-learned neural network can include one or more fusion layers that are configured to fuse image features from image data with LIDAR features from LIDAR point cloud data); inputting the coarse training point cloud to the second network, to generate an output training point cloud ([0102], BEV stream 224 can be determined from BEV input 225, which can correspond to either the LIDAR point cloud data 214 from LIDAR system 202 or map-modified LIDAR point cloud data from map fusion system 220, and the fusion system 208 can be further configured to execute at a machine-learned neural network within BEV system 226, one or more continuous convolutions to fuse the image features from the first data stream with the LIDAR features from the second data stream to generate a feature map 228 that includes the fused image features and LIDAR features). Liang differs from the claimed invention in not specifically teaching the steps of comparing the output training point cloud to a target high-resolution training point cloud; and adjusting weights of the second network based on a point-cloud similarity loss computed between the output training point cloud and the input target high-resolution training point cloud, while not updating weights of the first network. However, Amirloo teaches a method of generating high resolution and high accuracy point cloud having the steps of comparing the output training point cloud to a target high-resolution training point cloud ([0110], the pre-processing module 530 generates a mathematical representation of the training error between two point clouds using one of the two point cloud as the ground truth); and adjusting weights of the second network based on a point-cloud similarity loss computed between the output training point cloud and the input target high-resolution training point cloud, while not updating weights of the first network ([0111], the pre-processing module 530 determines whether the training error is less than an error threshold, i.e., the correction function may be recalculated by backpropagating the training error through the neural network by updating parameters, such as weights, of the neural network to minimize the training error, when the training error is greater than or equal than the error threshold, processing proceeds to operation 618 at which the correction function is recalculated) in order to provide an efficient solution for generating high resolution and high accuracy point clouds. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liang in comparing the output training point cloud to a target high-resolution training point cloud; and adjusting weights of the second network based on a point-cloud similarity loss computed between the output training point cloud and the input target high-resolution training point cloud, while not updating weights of the first network, as per teaching of Amirloo, in order to provide an efficient solution for generating high resolution and high accuracy point clouds.
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Heo et al. (US 2026/0028041 A1) discloses a method enables obtaining fused grid map for precise environmental detection, comprising: generating, based on a segmentation model processing point cloud data, a first semantic grid map; generating, based on an object detection model, a second semantic grid map; adjusting a probability regarding whether occupancy exists for an element included in each grid of the first semantic grid map and the second semantic grid map; and generating a fused grid map by determining, as a representative label, at least one label corresponding to a highest value among final probabilities of the at least one label, thus improving performance and reliability of the autonomous driving system (abstract and figure 2).
Li et al. (US 2024/0087222 A1) discloses a process to generate three-dimensional semantic information for the scene further comprises: generating 3D point cloud data based on the depth map; generating a first binary voxel grid occupancy map at a first resolution based on the 3D point cloud data; and converting, via a depth correction network, the first binary voxel grid occupancy map at the first resolution to a second binary voxel grid occupancy map at a second resolution that is lower than the first resolution (abstract and [0005]-[0025]).
Widjaja et al. (US 12,131,562 B2) discloses methods for enhanced semantic labeling in mapping with a semantic labeling system (abstract and figure 8).
Ho et al. (US 11,580,328 B1) discloses a system selecting multiple images of a space captured from different location and/or at different times to process for providing multiple views of the points in a point cloud, reduces computing resource consumption relative to processing the available images and assures that same types of variations from projection process are experienced in training and inference by same process for training and inference so as to improve performance of a two-dimensional convolution neural network for semantic segmentation (abstract).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEORGE ENG whose telephone number is (571)272-7495. The examiner can normally be reached Flex M to F, 7 am to 3 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James A Kramer can be reached at (571) 272-6783. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEORGE ENG/Supervisory Patent Examiner, Art Unit 2699
Prosecution Insights