Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status
This instant application No. 18/459088 has Claims 1-20 pending.
Priority / Filing Date
Applicant did not claim for any domestic or foreign priority. The effective filing date of this application is August 31, 2023.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-7, 12-14 and 19-20 are rejected under 35 U.S.C. 112, second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which applicant regards as the invention.
i) As per claims 5, 12 and 19, the following terms/limits lacks insufficient antecedent basis for this limitation in the claims.
‘the environment’ (line 3);
‘the LiDAR sensor’ (lines 3).
Additionally it is not clear what is meant by the limitations ‘the environment of the LiDAR sensor’-what is meant by the environment here. Is it virtual environment or real environment. Clarification/correction required in the claim language.
As per depended claims, they are rejected for incorporating the above errors from their respective parent claim by dependency.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1-3, 8-10 and 15-17 are rejected under 35 U.S.C. 103 as being obvious over Manivasagamet al. hereafter Manivasagam (Pub. No.: US 2020/0301799 A1), in view of Aghaei et al. hereafter Aghaei (Pub. No.: US 2024/0385300 A1).
Regarding Claim 1, Manivasagam discloses a method of simulating a Light Detection And Ranging (LiDAR) return signal (Manivasagam: abstract), comprising:
creating a model of an object and a LiDAR unit in a virtual environment (Manivasagam: [0007]: using a machine-learned model, the initial three-dimensional point cloud to predict a respective dropout probability for one or more of the plurality of points; [0008]: determining, by the computing system, a trajectory that describes a series of locations of a virtual object relative to the environment over time; [0009]: obtaining a ground truth three-dimensional point cloud collected by a physical LiDAR system as the physical LiDAR system traveled along a trajectory through an environment), wherein:
the LiDAR return signal comprises a return intensity of a reflection of an incident illumination beam by the object (Manivasagam: [0036]: LiDAR point clouds contain intensity returns, which are typically exploited in applications such as lane detection, semantic segmentation and construction detection, as the reflectivity of some materials is very informative); and
the model determines the LiDAR return signal based in part on a range from the LiDAR unit to the object and (Manivasagam: [0044]: In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation; [0116]: The intensity value of a point is influenced by many factors including incidence angle, range, and the beam bias; Also see [0124);
training the model with a set of road data records each comprising a measured range, a measured return intensity, (Manivasagam: [0044]: In addition to the geometric information, sensory metadata (e.g., incidence angle, raw intensity, transmitted power level, range value, unique ID per beam, etc.) can be recorded for each surface element (e.g., to be used for intensity simulation; [0079], [0080]: the model trainer 160 can train a machine-learned model 110 and/or 140 based on a set of training data 162. The training data 162 can include, for example, sets of LiDAR data that were physically collected at various known locations); and
simulating a LiDAR illumination beam emitted by the LiDAR unit toward the object and determining the return intensity of the reflection of an incident portion of the emitted illumination beam (Manivasagam: [0086]-[0087], [0114]-[0116]: the ray casting engine used to generate the initial point cloud is the Unreal engine……the computing system can find its nearest neighbor in cartesian space that did produce a return and use the range value returned from this successful neighbor……The computing system can employ nearest neighbors as the estimator for intensity. To be specific, for each returned ray, the computing system can conduct a nearest neighbor search within a small radius of the hitted surfel where reflectance of the local surface is assumed to be the same).
Manivasagam do not explicitly disclose:
an object label;
Aghaei disclose:
an object label (Aghaei: [0026], [0041]: an output of the perception stack 112 can be a bounding area around a perceived object that can be associated with a semantic label that identifies the type of object that is within the bounding area);
Manivasagam and Aghaei are analogous art because they are from the same field of endeavor. They both relate to LiDAR Simulation.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above synthetic LiDAR data generation using machine learning, as taught by Manivasagam, and incorporating the use of semantic labeling of object, as taught by Aghaei.
One of ordinary skill in the art would have been motivated to do this modification in order to simulating beam-to-beam variation and device-to-device variation for Light Detection and Ranging (LiDAR) sensors used by autonomous vehicles, as suggested by Aghaei (Aghaei: [0001]).
Regarding Claims 8 and 15, the claims recite the same substantive limitations as Claim 1 and are rejected using the same teachings.
Regarding Claim 2, the combinations of Manivasagam and Aghaei further disclose the method of claim 1, wherein:
the object model comprises a surface that is made of a material and has an orientation relative to the LiDAR unit (Manivasagam: [0043]: based on a respective pose ( e.g., location and orientation) of the vehicle at the time of data collection; [0050]: identifying a closest surface element in the three-dimensional map to the ray casting location and along the ray casting direction and generating one of the plurality of points with its respective depth based at least in part on a distance from the ray casting location to the closest surface element; Also see [0174], [0175]); the object label is associated with the material (Aghaei: [0026], [0041]: an output of the perception stack 112 can be a bounding area around a perceived object that can be associated with a semantic label that identifies the type of object that is within the bounding area);
a portion of the illumination beam is reflected by the surface (Manivasagam: [0086]: a portion of the light energy is reflected back); and
the determination of the return intensity is based in part on at least one of the material and the orientation of the surface (Manivasagam: [0036]: Intensity returns are very difficult to simulate as they depend on many factors including incidence angle, material reflectivity,).
Regarding Claims 9 and 16, the claims recite the same substantive limitations as Claim 2 and are rejected using the same teachings.
Regarding Claim 3, the combinations of Manivasagam and Aghaei further disclose the method of claim 2, wherein:
the object model further comprises a finish of the surface (Manivasagam: [0132]: the three-dimensional map can be a map that includes a plurality of surface elements (which may, in some instances, be referred to as "surfels") that indicate the respective surfaces of various objects (e.g., buildings, road surfaces, curbs, trees, etc.) within the environment. Metadata such as surface normal and/or other surface information can be associated with each surface element); and
the determination of the return intensity is based in part on the finish (Manivasagam: [0096]: the computing system can record sensory metadata 210 for each surfel to be used for intensity and ray drop simulation).
Regarding Claims 10 and 17, the claims recite the same substantive limitations as Claim 3 and are rejected using the same teachings.
5. Claims 4-7, 11-14 and 18-20 are rejected under 35 U.S.C. 103 as being obvious over Manivasagamet al. hereafter Manivasagam (Pub. No.: US 2020/0301799 A1), in view of Aghaei et al. hereafter Aghaei (Pub. No.: US 2024/0385300 A1), further in view of Scott Dylewski hereafter Dylewski (Pub. No.: US 2019/0369212 A1).
Regarding Claim 4, the combinations of Manivasagam and Aghaei disclose the method of claim 3.
However, the combinations of Manivasagam and Aghaei do not explicitly disclose wherein:
the object model further comprises a diffuse-specular reflection parameter; and
the determination of the return intensity is based in part on the diffuse-specular reflection parameter.
Dylewski discloses:
the object model further comprises a diffuse-specular reflection parameter (Dylewski: [0032]: reflectivity values based on intensity return values measured by the Lidar system 118 to determine diffuse and specular reflectivity characteristics of objects and surfaces); and
the determination of the return intensity is based in part on the diffuse-specular reflection parameter (Dylewski: [0032]: reflectivity values based on intensity return values measured by the Lidar system 118 to determine diffuse and specular reflectivity characteristics of objects and surfaces).
Manivasagam, Aghaei and Dylewski are analogous art because they are from the same field of endeavor. All of them relate to LiDAR systems and simulation.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above synthetic LiDAR data generation using machine learning, as taught by the combinations of Manivasagam, and Aghaei and incorporating the use of diffuse-specular reflection parameter, as taught by Dylewski.
One of ordinary skill in the art would have been motivated to do this modification in order to updating a motion plan for the AV system based on the specular reflectivity characteristics of the target object., as suggested by Dylewski (Dylewski: abstract).
Regarding Claims 11 and 18, the claims recite the same substantive limitations as Claim 4 and are rejected using the same teachings.
Regarding Claim 5, the combinations of Manivasagam and Aghaei disclose the method of claim 2.
Manivasagam further disclose, wherein:
an environmental parameter associated with the environment of the LiDAR sensor at a time that the range and intensity were measured (Manivasagam: [0038]:; sensor noise [0125]: atmospheric transmittance, sensor bias, etc);
a finish of the observed object (Manivasagam: [0132]: the three-dimensional map can be a map that includes a plurality of surface elements (which may, in some instances, be referred to as "surfels") that indicate the respective surfaces of various objects (e.g., buildings, road surfaces, curbs, trees, etc.) within the environment. Metadata such as surface normal and/or other surface information can be associated with each surface element);
However, the combinations of Manivasagam and Aghaei do not explicitly disclose:
a diffuse-specular reflection parameter associated with the observed object.
Dylewski discloses:
a diffuse-specular reflection parameter associated with the observed object (Dylewski: [0032]: reflectivity values based on intensity return values measured by the Lidar system 118 to determine diffuse and specular reflectivity characteristics of objects and surfaces).
Manivasagam, Aghaei and Dylewski are analogous art because they are from the same field of endeavor. All of them relate to LiDAR systems and simulation.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above synthetic LiDAR data generation using machine learning, as taught by the combinations of Manivasagam, and Aghaei and incorporating the use of diffuse-specular reflection parameter, as taught by Dylewski.
One of ordinary skill in the art would have been motivated to do this modification in order to updating a motion plan for the AV system based on the specular reflectivity characteristics of the target object., as suggested by Dylewski (Dylewski: abstract).
Regarding Claims 12 and 19, the claims recite the same substantive limitations as Claim 5 and are rejected using the same teachings.
Regarding Claim 6, the combinations of Manivasagam, Aghaei and Dylewski further disclose the method of claim 5, wherein:
the object label (Aghaei: [0026], [0041]: an output of the perception stack 112 can be a bounding area around a perceived object that can be associated with a semantic label that identifies the type of object that is within the bounding area)is associated with the finish (Manivasagam: [0132]: the three-dimensional map can be a map that includes a plurality of surface elements (which may, in some instances, be referred to as "surfels") that indicate the respective surfaces of various objects (e.g., buildings, road surfaces, curbs, trees, etc.) within the environment. Metadata such as surface normal and/or other surface information can be associated with each surface element); and
the environmental parameter (Manivasagam: [0038]: sensor noise; [0125]: atmospheric transmittance, sensor bias, etc.).
However, the combinations of Manivasagam and Aghaei do not explicitly disclose:
the diffuse-specular reflection parameter (Dylewski: [0032]: reflectivity values based on intensity return values measured by the Lidar system 118 to determine diffuse and specular reflectivity characteristics of objects and surfaces).
Regarding Claims 13 and 20, the claims recite the same substantive limitations as Claim 6 and are rejected using the same teachings.
Regarding Claim 7, the combinations of Manivasagam, Aghaei and Dylewski further disclose the method of claim 6, wherein:
one or more of the material, the finish, the diffuse-specular reflection parameter, and the environmental parameter are manually selected (Manivasagam: [0164]: Additionally, or alternatively, the operating mode of the vehicle 805 can be manually selected via one or more interfaces located onboard the vehicle 805 (e.g., key switch, button, etc.) and/or associated with a computing device proximate to the vehicle 805 ( e.g., a tablet operated by authorized personnel located near the vehicle 805));.
Regarding Claim 14, the claim recites the same substantive limitations as Claim 7 and is rejected using the same teachings.
Conclusion
6. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yang Yang (Pub. No.: US 2024/0219536 A1) teaches Systems and techniques are for characterizing LiDAR sensors that are used on autonomous vehicles on a beam-by-beam basis by the position of the LiDAR device that can be based on an elevation angle that corresponds to the LiDAR channel that is under test.
Ranganath et al. (Pub. No.: US 2024/0177412 A1) teaches A LiDAR method and system for training and using a machine learning model, where in generating a LiDAR scene by placing the at least one point cloud representation within the map; and training the machine learning model using the generated LiDAR scene.
Muckenhuber et al. (Automotive Lidar Modelling Approach Based on Material Properties and Lidar Capabilities,2020, MDPI, pp 1-25) conceptually presents a new lidar modelling approach that takes material properties and corresponding lidar capabilities into account. The considered material property is the incidence angle dependent reflectance of the illuminated material in the infrared spectrum and the considered lidar property its capability to detect a material with a certain reflectance up to a certain range.
Zhao et al. (Mapping with Reflection - Detection and Utilization of Reflection in 3D Lidar Scans, 2020, IEEE, pp 1-7) presents a method to detect reflection of 3D light detection and ranging (Lidar) scans and uses it to classify the points and also map objects outside the line of sight.
7. Examiner’s Remarks: Examiner has cited particular columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. In the case of amending the claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
Correspondence Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IFTEKHAR A KHAN whose telephone number is (571)272-5699. The examiner can normally be reached on M-F from 9:00AM-6:00PM (CST). If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emerson Puente can be reached on (571)272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/IFTEKHAR A KHAN/Primary Examiner, Art Unit 2187
Prosecution Insights