DETAILED ACTION
Claims 1-20 have been presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings received on 03 January 2023 are accepted.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to judicial exception (i.e. abstract idea) without significantly more.
Step 1: Claims 1-7 are directed to a system, which is a machine, which is a statutory category of invention. Claims 8-14 are directed to a method, which is a process, which is a statutory category of invention. Claims 15-20 are directed to a non-transitory computer readable media which is an article of manufacture, which is a statutory category of invention. Therefore, claims 1-20 are directed to patent eligible categories of invention.
Step 2A, Prong 1: Claims 1, 8, and 15 recite the abstract idea of simulating beam divergence for Light Detection and Ranging (LiDAR) sensors used by autonomous vehicles, constituting an abstract idea based on Mathematical Concepts including mathematical formulas or equations as well as calculations or alternatively Mental Processes based on concepts performed in the human mind, or with the aid of pencil and paper. The limitations of “determine one or more intensity parameters associated with one or more virtual beam receptions by the LiDAR sensor, wherein the one or more virtual beam receptions correspond to one or more rays from the plurality of rays that are reflected from one or more virtual objects;” and “determine at least one object intensity parameter for each of the one or more virtual objects based on the one or more modified intensity parameters” cover mathematical concepts including utilizing an underlying algorithm or statistical model, in this case the beam divergence model, being used to determine parameters for the virtual beam and objects of the simulation. Alternatively, the limitations of “determine one or more intensity parameters associated with one or more virtual beam receptions by the LiDAR sensor, wherein the one or more virtual beam receptions correspond to one or more rays from the plurality of rays that are reflected from one or more virtual objects;” and “determine at least one object intensity parameter for each of the one or more virtual objects based on the one or more modified intensity parameters” cover mental process including analyzing the virtual beam receptions and virtual objects of a simulation and determining intensity parameters based on that analysis. Additionally, the limitation of “adjust the one or more intensity parameters based on one or more transmission intensity weights corresponding to the one or more rays to yield one or more modified intensity parameters” covers mathematical concepts including utilizing an underlying algorithm or statistical model, in this case the beam divergence model, being used to determine adjusted parameters for the simulation. Alternatively, the limitation of “adjust the one or more intensity parameters based on one or more transmission intensity weights corresponding to the one or more rays to yield one or more modified intensity parameters” covers mental process including analyzing the transmission intensity weights of the virtual beam receptions and adjusting intensity parameters based on that analysis. Thus, the claims recite the abstract idea of a mental process performed in the human mind, or with the aid of pencil and paper.
Dependent claims 2-7, 9-14, and 16-20 further narrow the abstract ideas, identified in the independent claims.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application. The limitation of “generate, within a simulation environment, at least one virtual beam transmission from a Light Detection and Ranging (LiDAR) sensor using a beam divergence model, wherein the at least one virtual beam transmission includes a plurality of rays” in claims 1, 8, and 15, merely uses a computer device as a tool to perform the abstract idea. (MPEP 2106.05(f)) The generation of a virtual transmission from a LiDAR sensor are mere instructions to apply an exception as set forth in MPEP2106.05(f). Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not integrate a judicial exception into a practical application. (MPEP 2106.05(f)(2)) The background of the specification discloses that “a LiDAR sensor can be used to determine ranges (variable distance) of one or more targets by directing a laser to a surface of an entity (e.g., a person, an object, a structure, an animal, etc.) and measuring the time for light reflected from the surface to return to the LiDAR.” (See [0002]) Therefore, the judicial exception is not integrated into a practical application.
Step 2B: Claims 1, 8, and 15 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The limitation of “generate, within a simulation environment, at least one virtual beam transmission from a Light Detection and Ranging (LiDAR) sensor using a beam divergence model, wherein the at least one virtual beam transmission includes a plurality of rays” in claims 1, 8, and 15, merely uses a computer device as a tool to perform the abstract idea. (MPEP 2106.05(f)) The generation of a virtual transmission from a LiDAR sensor are mere instructions to apply an exception as set forth in MPEP2106.05(f). Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not integrate a judicial exception into a practical application. (MPEP 2106.05(f)(2)) Therefore, the claim as a whole does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, when considered alone or in combination, do not amount to significantly more than the judicial exception. As stated in Section I.B. of the December 16, 2014 101 Examination Guidelines, “[t]o be patent-eligible, a claim that is directed to a judicial exception must include additional features to ensure that the claim describes a process or product that applies the exception in a meaningful way, such that it is more than a drafting effort designed to monopolize the exception.”
The dependent claims include the same abstract ideas recited as recited in the independent claims, and merely incorporate additional details that narrow the abstract ideas and fail to add significantly more to the claims.
Dependent claims 2, 9, and 16 are directed to further defining the intensity parameters corresponding to a virtual object and filtering a portion of the model based on the parameters corresponding to that object, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes” or alternatively “Mathematical Concepts.”
Dependent claims 3 and 10 are directed to further defining the model used in the simulation, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes” or alternatively “Mathematical Concepts.”
Dependent claims 4, 11, and 17 are directed to further defining the intensity parameters of the simulation, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes” or alternatively “Mathematical Concepts.”
Dependent claims 5, 12, and 18 are directed to further defining the intensity parameters of the simulation, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes” or alternatively “Mathematical Concepts.”
Dependent claims 6, 13, and 19 are directed to further defining the intensity parameters of the simulation, which further narrows the abstract idea identified in the independent claim, which is directed to “Mental Processes” or alternatively “Mathematical Concepts.”
Dependent claims 7, 14, and 20 are directed to further applying the defined parameters to an autonomous vehicle in the simulation, which narrows the abstract idea identified in the independent claim, which is directed to MPEP2106.05(f).
Accordingly, claims 1-20 are rejected under 35 U.S.C 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without anything significantly more.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4-9, 11-20 are rejected under 35 U.S.C 103 as being unpatentable over U.S. Patent Application 2022/0262072 A1, hereafter M in view of NPL: Lidar Essential Beam Model for Accurate Width Estimation of Thin Poles (Long, 2020), hereafter L.
Regarding Claim 1: M discloses a system comprising:
A memory; and one or more processors coupled to the memory
M [0062] “The LiDAR synthesis computing system 102 includes one or more processors 112 and a memory 114. The one or more processors 112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory devices, flash memory devices, etc., and combinations thereof.”
the one or more processors being configured to:
generate, within a simulation environment, at least one virtual beam transmission from a Light Detection and Ranging (LiDAR) sensor …, wherein the at least one virtual beam transmission includes a plurality of rays;
M [0034] “Certain existing approaches to LiDAR simulation for autonomous driving focus on employing handcrafted 3D primitives (such as buildings, cars, trees, roads). Graphics engines have been utilized to ray cast the scene and create virtual LiDAR data.”
M [0049] “In some implementations, the computing system can account for the rotary motion of the virtual LiDAR system (also known as “rolling shutter effects”) by compensating for motion of the virtual system along the trajectory during the simulated LiDAR sweep. In particular, performing the ray casting can include determining, for each of a plurality of rays, a ray casting location and a ray casting direction based at least in part on the trajectory.”
M [0039] “In particular, aspects of the present disclosure are directed to systems and methods that use a machine-learned model to make an initial three-dimensional point cloud generated using a physics-based approach more realistic. In particular, the machine-learned model can learn to modify the geometry of point clouds (e.g., as exhibited by ray dropouts) generated through ray casting and/or other physics-based approaches to better match ground truth counterparts that were physically collected by LiDAR systems in the real world.”
determine one or more intensity parameters associated with one or more virtual beam receptions by the LiDAR sensor, wherein the one or more virtual beam receptions correspond to one or more rays from the plurality of rays that are reflected from one or more virtual objects;
M [0044] “The computing system can convert the aggregate LiDAR point cloud to a surface element-based three-dimensional mesh. For example, the computing system can perform voxel-based downsampling and normal estimation to perform the conversion. In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation).”
M [0086] “The illustrated process focuses on simulating a scanning LiDAR system. One example system that can be simulated is the Velodyne HDL-64E which has 64 emitter-detector pairs vertically arranged, each of which uses light pulses to measure distance. The basic concept is that each emitter emits a light pulse which travels until it hits a target, and a portion of the light energy is reflected back and received by the detector. Distance is measured by calculating the time of travel and material reflectance is measured through the intensity of the returned pulse. The entire optical assembly rotates on a base to provide a 360-degree azimuth field of view at around 10 Hz with each full “sweep” providing approximately 70 k returns.”
adjust the one or more intensity parameters … to yield one or more modified intensity parameters;
M [0155] “At 614, the computing system can modify one or more values of one or more parameters of the machine-learned model based at least in part on the objective function. For example, the objective function can be backpropagated through the model and the values of the parameters can be updated based on a gradient of the objective function.”
and determine at least one object intensity parameter for each of the one or more virtual objects based on the one or more modified intensity parameters.
M [0044] “The computing system can convert the aggregate LiDAR point cloud to a surface element-based three-dimensional mesh. For example, the computing system can perform voxel-based downsampling and normal estimation to perform the conversion. In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation).”
M doesn’t disclose using a beam divergence model or one or more transmission intensity weights corresponding to the one or more rays.
However, L discloses using a beam divergence model and one or more transmission intensity weights corresponding to the one or more rays.
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected.”
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected. That is, the EB radius specifies a disk which, if it overlaps the edge of an object, will generate a return.” Examiner notes that the specification states that one or more transmission intensity weights can be based on a distance of each of the plurality of rays from a center of the at least one virtual beam transmission (See [0088]).
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 2: M in view of L discloses the system of claim 1 wherein the one or more processors are further configured to: determine that the at least one object intensity parameter corresponding to a first virtual object from the one or more virtual objects is less than …; and disregard a portion of virtual beam receptions from the one or more virtual beam receptions, wherein the portion of virtual beam receptions corresponds to reflections of the one or more rays from the first virtual object.
M [0125] “The output of the model is a ray dropout probability that predicts, for each element in the array, if it returns or not (e.g., with some probability). In some implementations, to simulate LiDAR noise, the computing system can sample from the probability mask to generate the output LiDAR point cloud. Sampling of the probability mask instead of doing direct thresholding has the following benefits: (1) Raydrop can be learned with cross-entropy loss, meaning the estimated probabilities may not be well calibrated. Sampling helps mitigate this issue compared to thresholding. (2) Real lidar data is non-deterministic due to additional noises (atmospheric transmittance, sensor bias, etc.) that the proposed approach may not completely model.”
M doesn’t disclose a threshold intensity value.
However, L discloses a threshold intensity value.
L [Page 2281: Introduction] “When a Lidar beam grazes the edge of an object, whether or not a hit is reported depends not only the spatial energy distribution and the beam’s overlap of the object, but also on the internal processing and thresholding mechanism within the Lidar sensor [3]. Rather than selecting our own threshold on beam spread, we prefer to use the Lidar itself to do this based on a series of measurements of pre-defined targets. We call this process Lidar beam calibration, which is introduced in Section IV-C. This motivates us to propose the essential beam (EB) as a beam spread model that incorporates both actual energies spread and also internal sensor processing and thresholding of the reflected beam.”
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 4: M in view of L discloses the system of claim 1, wherein the one or more intensity parameters are based on an angle of incidence between a corresponding ray from the one or more rays and a corresponding virtual object from the one or more virtual objects.
M [0096] “In addition to geometric information, the computing system can record sensory metadata 210 for each surfel to be used for intensity and ray drop simulation. This can include, among other information, the incidence angle, raw intensity, transmitted power level, range value as well as a unique ID per beam.”
Regarding Claim 5: M in view of L discloses the system of claim 1, wherein the one or more intensity parameters are based on one or more reflectivity parameters corresponding to the one or more virtual objects.
M [0086] “The illustrated process focuses on simulating a scanning LiDAR system. One example system that can be simulated is the Velodyne HDL-64E which has 64 emitter-detector pairs vertically arranged, each of which uses light pulses to measure distance. The basic concept is that each emitter emits a light pulse which travels until it hits a target, and a portion of the light energy is reflected back and received by the detector. Distance is measured by calculating the time of travel and material reflectance is measured through the intensity of the returned pulse.”
Regarding Claim 6: M in view of L discloses the system of claim 1.
M doesn’t disclose wherein the one or more transmission intensity weights are based on a distance of each of the plurality of rays from a center of the at least one virtual beam transmission.
However, L discloses wherein the one or more transmission intensity weights are based on a distance of each of the plurality of rays from a center of the at least one virtual beam transmission.
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected. That is, the EB radius specifies a disk which, if it overlaps the edge of an object, will generate a return.”
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 7: M in view of L discloses the system of claim 1, wherein the one or more processors are further configured to: send the one or more modified intensity parameters to a perception stack of an autonomous vehicle operating in the simulation environment.
M [0029] “The simulated LiDAR data can be used, for example, as simulated input for testing autonomous vehicle control systems. The systems and methods of the present disclosure improve both quantitatively and qualitatively the the synthesized LiDAR data over solely physics-based rendering. The improved quality of the synthesized LiDAR point cloud demonstrates the potential of this LiDAR simulation approach and application to generating realistic sensor data, which will ultimately improve the safety an autonomous vehicle.”
Regarding Claim 8: M discloses a method comprising:
generating, within a simulation environment, at least one virtual beam transmission from a Light Detection and Ranging (LiDAR) sensor …, wherein the at least one virtual beam transmission includes a plurality of rays;
M [0034] “Certain existing approaches to LiDAR simulation for autonomous driving focus on employing handcrafted 3D primitives (such as buildings, cars, trees, roads). Graphics engines have been utilized to ray cast the scene and create virtual LiDAR data.”
M [0049] “In some implementations, the computing system can account for the rotary motion of the virtual LiDAR system (also known as “rolling shutter effects”) by compensating for motion of the virtual system along the trajectory during the simulated LiDAR sweep. In particular, performing the ray casting can include determining, for each of a plurality of rays, a ray casting location and a ray casting direction based at least in part on the trajectory.”
M [0039] “In particular, aspects of the present disclosure are directed to systems and methods that use a machine-learned model to make an initial three-dimensional point cloud generated using a physics-based approach more realistic. In particular, the machine-learned model can learn to modify the geometry of point clouds (e.g., as exhibited by ray dropouts) generated through ray casting and/or other physics-based approaches to better match ground truth counterparts that were physically collected by LiDAR systems in the real world.”
determining one or more intensity parameters associated with one or more virtual beam receptions by the LiDAR sensor, wherein the one or more virtual beam receptions correspond to one or more rays from the plurality of rays that are reflected from one or more virtual objects;
M [0044] “The computing system can convert the aggregate LiDAR point cloud to a surface element-based three-dimensional mesh. For example, the computing system can perform voxel-based downsampling and normal estimation to perform the conversion. In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation).”
M [0086] “The illustrated process focuses on simulating a scanning LiDAR system. One example system that can be simulated is the Velodyne HDL-64E which has 64 emitter-detector pairs vertically arranged, each of which uses light pulses to measure distance. The basic concept is that each emitter emits a light pulse which travels until it hits a target, and a portion of the light energy is reflected back and received by the detector. Distance is measured by calculating the time of travel and material reflectance is measured through the intensity of the returned pulse. The entire optical assembly rotates on a base to provide a 360-degree azimuth field of view at around 10 Hz with each full “sweep” providing approximately 70 k returns.”
adjusting the one or more intensity parameters … to yield one or more modified intensity parameters;
M [0155] “At 614, the computing system can modify one or more values of one or more parameters of the machine-learned model based at least in part on the objective function. For example, the objective function can be backpropagated through the model and the values of the parameters can be updated based on a gradient of the objective function.”
and determining at least one object intensity parameter for each of the one or more virtual objects based on the one or more modified intensity parameters.
M [0044] “The computing system can convert the aggregate LiDAR point cloud to a surface element-based three-dimensional mesh. For example, the computing system can perform voxel-based downsampling and normal estimation to perform the conversion. In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation).”
M doesn’t disclose using a beam divergence model or one or more transmission intensity weights corresponding to the one or more rays.
However, L discloses using a beam divergence model and one or more transmission intensity weights corresponding to the one or more rays.
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected.”
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected. That is, the EB radius specifies a disk which, if it overlaps the edge of an object, will generate a return.” Examiner notes that the specification states that one or more transmission intensity weights can be based on a distance of each of the plurality of rays form a center of the at least one virtual beam transmission (See [0088]).
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 9: M in view of L discloses the method of claim 8 further comprising: determining that the at least one object intensity parameter corresponding to a first virtual object from the one or more virtual objects is less than …; and disregarding a portion of virtual beam receptions from the one or more virtual beam receptions, wherein the portion of virtual beam receptions corresponds to reflections of the one or more rays from the first virtual object.
M [0125] “The output of the model is a ray dropout probability that predicts, for each element in the array, if it returns or not (e.g., with some probability). In some implementations, to simulate LiDAR noise, the computing system can sample from the probability mask to generate the output LiDAR point cloud. Sampling of the probability mask instead of doing direct thresholding has the following benefits: (1) Raydrop can be learned with cross-entropy loss, meaning the estimated probabilities may not be well calibrated. Sampling helps mitigate this issue compared to thresholding. (2) Real lidar data is non-deterministic due to additional noises (atmospheric transmittance, sensor bias, etc.) that the proposed approach may not completely model.”
M doesn’t disclose a threshold intensity value.
However, L discloses a threshold intensity value.
L [Page 2281: Introduction] “When a Lidar beam grazes the edge of an object, whether or not a hit is reported depends not only the spatial energy distribution and the beam’s overlap of the object, but also on the internal processing and thresholding mechanism within the Lidar sensor [3]. Rather than selecting our own threshold on beam spread, we prefer to use the Lidar itself to do this based on a series of measurements of pre-defined targets. We call this process Lidar beam calibration, which is introduced in Section IV-C. This motivates us to propose the essential beam (EB) as a beam spread model that incorporates both actual energies spread and also internal sensor processing and thresholding of the reflected beam.”
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 11: M in view of L discloses the method of claim 8, wherein the one or more intensity parameters are based on an angle of incidence between a corresponding ray from the one or more rays and a corresponding virtual object from the one or more virtual objects.
M [0096] “In addition to geometric information, the computing system can record sensory metadata 210 for each surfel to be used for intensity and ray drop simulation. This can include, among other information, the incidence angle, raw intensity, transmitted power level, range value as well as a unique ID per beam.”
Regarding Claim 12: M in view of L discloses the method of claim 8, wherein the one or more intensity parameters are based on one or more reflectivity parameters corresponding to the one or more virtual objects.
M [0086] “The illustrated process focuses on simulating a scanning LiDAR system. One example system that can be simulated is the Velodyne HDL-64E which has 64 emitter-detector pairs vertically arranged, each of which uses light pulses to measure distance. The basic concept is that each emitter emits a light pulse which travels until it hits a target, and a portion of the light energy is reflected back and received by the detector. Distance is measured by calculating the time of travel and material reflectance is measured through the intensity of the returned pulse.”
Regarding Claim 13: M in view of L discloses the method of claim 8.
M doesn’t disclose wherein the one or more transmission intensity weights are based on a distance of each of the plurality of rays from a center of the at least one virtual beam transmission.
However, L discloses wherein the one or more transmission intensity weights are based on a distance of each of the plurality of rays from a center of the at least one virtual beam transmission.
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected. That is, the EB radius specifies a disk which, if it overlaps the edge of an object, will generate a return.”
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 14: M in view of L discloses the method of claim 8, further comprising: sending the one or more modified intensity parameters to a perception stack of an autonomous vehicle operating in the simulation environment.
M [0029] “The simulated LiDAR data can be used, for example, as simulated input for testing autonomous vehicle control systems. The systems and methods of the present disclosure improve both quantitatively and qualitatively the the synthesized LiDAR data over solely physics-based rendering. The improved quality of the synthesized LiDAR point cloud demonstrates the potential of this LiDAR simulation approach and application to generating realistic sensor data, which will ultimately improve the safety an autonomous vehicle.”
Regarding Claim 15: M discloses a non-transitory computer-readable media comprising instructions stored thereon which, when executed are configured to cause a computer or processor to:
generate, within a simulation environment, at least one virtual beam transmission from a Light Detection and Ranging (LiDAR) sensor …, wherein the at least one virtual beam transmission includes a plurality of rays;
M [0034] “Certain existing approaches to LiDAR simulation for autonomous driving focus on employing handcrafted 3D primitives (such as buildings, cars, trees, roads). Graphics engines have been utilized to ray cast the scene and create virtual LiDAR data.”
M [0049] “In some implementations, the computing system can account for the rotary motion of the virtual LiDAR system (also known as “rolling shutter effects”) by compensating for motion of the virtual system along the trajectory during the simulated LiDAR sweep. In particular, performing the ray casting can include determining, for each of a plurality of rays, a ray casting location and a ray casting direction based at least in part on the trajectory.”
M [0039] “In particular, aspects of the present disclosure are directed to systems and methods that use a machine-learned model to make an initial three-dimensional point cloud generated using a physics-based approach more realistic. In particular, the machine-learned model can learn to modify the geometry of point clouds (e.g., as exhibited by ray dropouts) generated through ray casting and/or other physics-based approaches to better match ground truth counterparts that were physically collected by LiDAR systems in the real world.”
determine one or more intensity parameters associated with one or more virtual beam receptions by the LiDAR sensor, wherein the one or more virtual beam receptions correspond to one or more rays from the plurality of rays that are reflected from one or more virtual objects;
M [0044] “The computing system can convert the aggregate LiDAR point cloud to a surface element-based three-dimensional mesh. For example, the computing system can perform voxel-based downsampling and normal estimation to perform the conversion. In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation).”
M [0086] “The illustrated process focuses on simulating a scanning LiDAR system. One example system that can be simulated is the Velodyne HDL-64E which has 64 emitter-detector pairs vertically arranged, each of which uses light pulses to measure distance. The basic concept is that each emitter emits a light pulse which travels until it hits a target, and a portion of the light energy is reflected back and received by the detector. Distance is measured by calculating the time of travel and material reflectance is measured through the intensity of the returned pulse. The entire optical assembly rotates on a base to provide a 360-degree azimuth field of view at around 10 Hz with each full “sweep” providing approximately 70 k returns.”
adjust the one or more intensity parameters … to yield one or more modified intensity parameters;
M [0155] “At 614, the computing system can modify one or more values of one or more parameters of the machine-learned model based at least in part on the objective function. For example, the objective function can be backpropagated through the model and the values of the parameters can be updated based on a gradient of the objective function.”
and determine at least one object intensity parameter for each of the one or more virtual objects based on the one or more modified intensity parameters.
M [0044] “The computing system can convert the aggregate LiDAR point cloud to a surface element-based three-dimensional mesh. For example, the computing system can perform voxel-based downsampling and normal estimation to perform the conversion. In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation).”
M doesn’t disclose using a beam divergence model or one or more transmission intensity weights corresponding to the one or more rays.
However, L discloses using a beam divergence model and one or more transmission intensity weights corresponding to the one or more rays.
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected.”
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected. That is, the EB radius specifies a disk which, if it overlaps the edge of an object, will generate a return.” Examiner notes that the specification states that one or more transmission intensity weights can be based on a distance of each of the plurality of rays form a center of the at least one virtual beam transmission (See [0088]).
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 16: M in view of L discloses the non-transitory computer-readable media of claim 15, comprising further instructions configured to cause the computer or processor to: determine that the at least one object intensity parameter corresponding to a first virtual object from the one or more virtual objects is less than …; and disregard a portion of virtual beam receptions from the one or more virtual beam receptions, wherein the portion of virtual beam receptions corresponds to reflections of the one or more rays from the first virtual object.
M [0125] “The output of the model is a ray dropout probability that predicts, for each element in the array, if it returns or not (e.g., with some probability). In some implementations, to simulate LiDAR noise, the computing system can sample from the probability mask to generate the output LiDAR point cloud. Sampling of the probability mask instead of doing direct thresholding has the following benefits: (1) Raydrop can be learned with cross-entropy loss, meaning the estimated probabilities may not be well calibrated. Sampling helps mitigate this issue compared to thresholding. (2) Real lidar data is non-deterministic due to additional noises (atmospheric transmittance, sensor bias, etc.) that the proposed approach may not completely model.”
M doesn’t disclose a threshold intensity value.
However, L discloses a threshold intensity value.
L [Page 2281: Introduction] “When a Lidar beam grazes the edge of an object, whether or not a hit is reported depends not only the spatial energy distribution and the beam’s overlap of the object, but also on the internal processing and thresholding mechanism within the Lidar sensor [3]. Rather than selecting our own threshold on beam spread, we prefer to use the Lidar itself to do this based on a series of measurements of pre-defined targets. We call this process Lidar beam calibration, which is introduced in Section IV-C. This motivates us to propose the essential beam (EB) as a beam spread model that incorporates both actual energies spread and also internal sensor processing and thresholding of the reflected beam.”
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 17: M in view of L discloses the non-transitory computer-readable media of claim 15, wherein the one or more intensity parameters are based on an angle of incidence between a corresponding ray from the one or more rays and a corresponding virtual object from the one or more virtual objects.
M [0096] “In addition to geometric information, the computing system can record sensory metadata 210 for each surfel to be used for intensity and ray drop simulation. This can include, among other information, the incidence angle, raw intensity, transmitted power level, range value as well as a unique ID per beam.”
Regarding Claim 18: M in view of L discloses the non-transitory computer-readable media of claim 15, wherein the one or more intensity parameters are based on one or more reflectivity parameters corresponding to the one or more virtual objects.
M [0086] “The illustrated process focuses on simulating a scanning LiDAR system. One example system that can be simulated is the Velodyne HDL-64E which has 64 emitter-detector pairs vertically arranged, each of which uses light pulses to measure distance. The basic concept is that each emitter emits a light pulse which travels until it hits a target, and a portion of the light energy is reflected back and received by the detector. Distance is measured by calculating the time of travel and material reflectance is measured through the intensity of the returned pulse.”
Regarding Claim 19: M in view of L discloses the non-transitory computer-readable media of claim 15.
M doesn’t disclose wherein the one or more transmission intensity weights are based on a distance of each of the plurality of rays from a center of the at least one virtual beam transmission.
However, L discloses wherein the one or more transmission intensity weights are based on a distance of each of the plurality of rays from a center of the at least one virtual beam transmission.
L [Page 2282, Section III] “Our model for a Lidar EB is a cone, illustrated in Fig. 1 with divergence angle θ and linearly growing radius r. Differing from the full diverging beam, the EB is the “essential” portion that specifies at what radial distance from an object being grazed, the Lidar point will be reflected. That is, the EB radius specifies a disk which, if it overlaps the edge of an object, will generate a return.”
M and L are analogous because they both pertain to the field of simulating a LiDAR sensor.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of L with M because the integration of the beam divergence model of L would measure the beam width, which would improve the 3D point clouds of M (See L [Page 2281: Introduction].
Regarding Claim 20: M in view of L discloses the non-transitory computer-readable media of claim 15, comprising further instructions configured to cause the computer or processor to: send the one or more modified intensity parameters to a perception stack of an autonomous vehicle operating in the simulation environment.
M [0029] “The simulated LiDAR data can be used, for example, as simulated input for testing autonomous vehicle control systems. The systems and methods of the present disclosure improve both quantitatively and qualitatively the the synthesized LiDAR data over solely physics-based rendering. The improved quality of the synthesized LiDAR point cloud demonstrates the potential of this LiDAR simulation approach and application to generating realistic sensor data, which will ultimately improve the safety an autonomous vehicle.”
Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application 2022/0262072 A1, hereafter M in view of NPL: Lidar Essential Beam Model for Accurate Width Estimation of Thin Poles (Long, 2020), hereafter L further in view of U.S. Patent Application 2002/0075763 A1, hereafter K.
Regarding Claim 3: M and L disclose the system of claim 1.
M and L don’t disclose wherein the beam divergence model corresponds to a gaussian beam divergence model.
However, K discloses the beam divergence model corresponds to a gaussian beam divergence model.
K [0075] “The foregoing simulation was performed on the assumption that the laser beam was applied from the laser aperture 12 uniformly. In the following example, the simulation was performed on the assumption that the laser beam had intensity distribution. In this example, the simulation was performed on the assumption that the laser beam had a Gaussian distribution.”
M, L and K are analogous because they both pertain to the simulation of a virtual beam.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of K with M and L because the light-emitting device for focusing a beam without a lens produces a smaller laser spot diameter, which is a known property of a Gaussian beam, as well a simplified head structure creating improvements in both productivity and costs (See K [0011]).
Regarding Claim 10: M and L disclose the method of claim 8.
M and L don’t disclose wherein the beam divergence model corresponds to a gaussian beam divergence model.
However, K discloses the beam divergence model corresponds to a gaussian beam divergence model.
K [0075] “The foregoing simulation was performed on the assumption that the laser beam was applied from the laser aperture 12 uniformly. In the following example, the simulation was performed on the assumption that the laser beam had intensity distribution. In this example, the simulation was performed on the assumption that the laser beam had a Gaussian distribution.”
M and K are analogous because they both pertain to the simulation of a virtual beam.
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the teachings of K with M and L because the light-emitting device for focusing a beam without a lens produces a smaller laser spot diameter, which is a known property of a Gaussian beam, as well a simplified head structure creating improvements in both productivity and costs (See K [0011]).
Conclusion
All Claims are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
U.S. Patent Application No. 2023/0114731 A1 discloses a method of generating virtual sensor data of a virtual SPAD lidar sensor including generating a 2D lidar array having a plurality of cells.
U.S. Patent Application No. 2021/0208263 A1 discloses a system and methods for calibrating multiple sensors (lidar, camera or other sensors) including obtaining data from the sensors, determining optimized transformation parameters, and transforming the data from one sensor projection plane to the other sensor projection plane.
Aniceto Belmonte, "Feasibility study for the simulation of beam propagation: consideration of coherent lidar performance," Appl. Opt. 39, 5426-5445 (2000) discloses a study on the effects of atmospheric refractive turbulence on coherent lidar performance through the use of simulations of beam propagation in 3D random media.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Scott T. Tran whose telephone number is (571) 272-8533. The examiner can normally be reached on M-F, 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Renee Chavez, can be reached at (571) 270-1104. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300. Informal or draft communication, please label PROPOSED or DRAFT, can be additionally sent to the Examiner’s fax phone number (571) 272-8533.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published a applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
STT
/SCOTT THANH BINH TRAN/Examiner, Art Unit 2186
/RENEE D CHAVEZ/Supervisory Patent Examiner, Art Unit 2186
Prosecution Insights