DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Examiner acknowledges amendment to the Claims and withdraws the Claim objections and rejections under 35 U.S.C. 112.
Response to Arguments
Applicant’s arguments with respect to Claims 1, 2, and 12 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Kummerle (Kummerle et al., “Automatic Calibration of Multiple Cameras and Depth Sensors with a Spherical Target”, https://ieeexplore.ieee.org/abstract/document/8593955, (2018).) in view of Groh (US 2018/0060725 A1).
Regarding Claim 1, Kummerle discloses a method for recognizing a target object in a target object recognizing apparatus (Page 5584, Column 1: “Today, robots are deployed in more and more complex environments to support us in daily life. To perceive their environment they are equipped with a variety of sensors. By fusing data from multiple sensors, better robustness and performance can be achieved in tasks like object detection or localization.”), the method comprising:
irradiating laser light (Figure 10, Page 5589, Column 2, Section B discloses real experiments with Velodyne LiDARs, which emit laser light) to a reference target object (Page 5585, Column 2, discloses a calibration target: “For our experiments we choose an off-the-shelf white Styrofoam sphere that can be purchased in a regular hardware store.”);
acquiring LiDAR data generated based on a reflection signal reflected from the reference target object (Figure 10, Page 5589, Column 2, Section B discloses real experiments with Velodyne LiDARs, which receive reflected laser light from the calibration target);
generating a reference map and virtual LiDAR data based on the LiDAR data for the reference target object (Page 5585: Column 2, B: “We detect depth discontinuities within each scan line to find free standing segments. Segments that are significantly longer than half of the sphere’s perimeter are filtered out. Next, segments which are close together are associated and form a point cluster.” The segments and scan lines are here taken to constitute a reference map.) and determining a weight (Page 5586, Column 2, D: “We solve the minimization problem (Eq. 2) by a numeric solver [19]. The initialization of all sensor poses is random.” These learned calibration parameters are therefore taken to be weights) for recognizing a target object by performing the deep learning based on the virtual LiDAR data (Under the broadest reasonable interpretation, the processed data from this step based on the LiDAR data is taken to be the virtual LiDAR data); and
recognizing a new target object by applying the weight when new LiDAR data with respect to the new target object is acquired (Page 5584, Column 1: “Today, robots are deployed in more and more complex environments to support us in daily life. To perceive their environment they are equipped with a variety of sensors. By fusing data from multiple sensors, better robustness and performance can be achieved in tasks like object detection or localization. For successful sensor fusion an accurate calibration of the sensor setup is essential. Sensor fusion is especially effective with diverse sensors such as cameras and depth sensors that measure in different domains.” This indicates that a recognition step is performed after the calibration is successfully completed.).
Kummerle does not teach and Groh does teach wherein the generating the reference map ([0051]: “After application of the training data 42, the machine learning module 72 may provide one or more ultrasonic mappings 88 and/or radar mappings 90 from three-dimensional modeling information to reflection values.”; [0059]) comprises performing modeling based on range data and intensity data ([0036]: “Such data may include intensity, phase, polarity, and/or Doppler shift information and may be resolved into different range bins.”) included in the LiDAR data ([0035]), and the generating the virtual LiDAR data comprises generating the virtual LiDAR data with respect to a virtual target object ([0031]) based on the reference map ([0068]: “Once the three-dimensional model is extended to a physical model simulating reflections from radar and/or ultrasonic waves, it may be used to develop, test, and evaluate algorithms 132 for processing radar data 68 and/or ultrasonic data 40 by utilizing virtual radar and/or ultrasonic data gathered from the simulated environment and/or simulated objects. Acquisition of such virtual datasets may be acquired by placing a virtual vehicle 34c within the simulation environment 130b coupled to the virtual sensor and capable of traveling a path through the simulation environment 130b.”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Groh to generate virtual LiDAR data from a reference map into the method of Kummerle. Groh notes in [0028] while referring to the virtual data that “This reflection data may be used as a dataset with which to develop, test, and/or evaluate a perception algorithm able to generate control signals to a vehicle 34 based on data from a radar sensor 38.” Testing a control or perception algorithm in a virtual environment offers a safer environment for validation than an actual road test. Thus innovative or highly experimental algorithms can be initially tested without risk of damage to a vehicle or LiDAR under test.
Regarding Claim 2, which depends from rejected Claim 1, Kummerle further disclose wherein, the LiDAR data with respect to a spherical photoreceptive surface reference target object is acquired and the LiDAR data is acquired based on a reflectance of the reflection signal (Page 5585, Column 2: “A spherical target meets these requirements. For our experiments we choose an off-the-shelf white Styrofoam sphere that can be purchased in a regular hardware store.”) excluding a signal which is scattered from the spherical photoreceptive surface reference target object (The geometry of the setup (Figs. 1 and 5) indicates that most light that undergoes diffuse or Lambertian reflection off of the spherical photoreceptor will not be collected by the LiDAR).
Regarding Claim 12, Kummerle discloses a target object recognizing apparatus (Page 5584, Column 1: “Today, robots are deployed in more and more complex environments to support us in daily life. To perceive their environment they are equipped with a variety of sensors. By fusing data from multiple sensors, better robustness and performance can be achieved in tasks like object detection or localization.”), comprising:
a transmitting unit which irradiates laser light (Figure 10, Page 5589, Column 2, Section B discloses real experiments with Velodyne LiDARs, which emit laser light);
a receiving unit which receives a reflection signal with respect to the laser light (Figure 10, Page 5589, Column 2, Section B discloses real experiments with Velodyne LiDARs, which receive reflected laser light from the calibration target);
one or more processors; and a memory in which one or more programs executed by the processors are stored, wherein when the programs are executed by one or more processors, the programs allow one or more processors to perform operations including (One or more processors and memory are implied by the calculations and ranging disclosed within Kummerle):
acquiring LiDAR data generated based on a reflection signal obtained by reflecting the irradiated laser light from the reference target object (Figure 10, Page 5589, Column 2, Section B discloses real experiments with Velodyne LiDARs, which receive reflected laser light from the calibration target);
generating a reference map and virtual LiDAR data based on the LiDAR data for the reference target object (Page 5585: Column 2, B: “We detect depth discontinuities within each scan line to find free standing segments. Segments that are significantly longer than half of the sphere’s perimeter are filtered out. Next, segments which are close together are associated and form a point cluster.” The segments and scan lines are here taken to constitute a reference map.) and determining a weight (Page 5586, Column 2, D: “We solve the minimization problem (Eq. 2) by a numeric solver [19]. The initialization of all sensor poses is random.” These learned calibration parameters are therefore taken to be weights) for recognizing a target object by performing deep learning based on the virtual LiDAR data (Under the broadest reasonable interpretation, the processed data from this step based on the LiDAR data is taken to be the virtual LiDAR data); and
recognizing a new target object by applying the weight when new LiDAR data with respect to the new target object is acquired (Page 5584, Column 1: “Today, robots are deployed in more and more complex environments to support us in daily life. To perceive their environment they are equipped with a variety of sensors. By fusing data from multiple sensors, better robustness and performance can be achieved in tasks like object detection or localization. For successful sensor fusion an accurate calibration of the sensor setup is essential. Sensor fusion is especially effective with diverse sensors such as cameras and depth sensors that measure in different domains.” This indicates that a recognition step is performed after the calibration is successfully completed.).
Kummerle does not teach and Groh does teach wherein the generating the reference map ([0051]: “After application of the training data 42, the machine learning module 72 may provide one or more ultrasonic mappings 88 and/or radar mappings 90 from three-dimensional modeling information to reflection values.”; [0059]) comprises performing modeling based on range data and intensity data ([0036]: “Such data may include intensity, phase, polarity, and/or Doppler shift information and may be resolved into different range bins.”) included in the LiDAR data ([0035]), and the generating the virtual LiDAR data comprises generating the virtual LiDAR data with respect to a virtual target object ([0031]) based on the reference map ([0068]: “Once the three-dimensional model is extended to a physical model simulating reflections from radar and/or ultrasonic waves, it may be used to develop, test, and evaluate algorithms 132 for processing radar data 68 and/or ultrasonic data 40 by utilizing virtual radar and/or ultrasonic data gathered from the simulated environment and/or simulated objects. Acquisition of such virtual datasets may be acquired by placing a virtual vehicle 34c within the simulation environment 130b coupled to the virtual sensor and capable of traveling a path through the simulation environment 130b.”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Groh to generate virtual LiDAR data from a reference map into the method of Kummerle. Groh notes in [0028] while referring to the virtual data that “This reflection data may be used as a dataset with which to develop, test, and/or evaluate a perception algorithm able to generate control signals to a vehicle 34 based on data from a radar sensor 38.” Testing a control or perception algorithm in a virtual environment offers a safer environment for validation than an actual road test. Thus innovative or highly experimental algorithms can be initially tested without risk of damage to a vehicle or LiDAR under test.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Oh (US 2019/0392254 A1) in view of Groh.
Regarding Claim 15, Oh discloses a mobile object ([0046]: “self-driving vehicle”, comprising:
a target object recognizing apparatus which irradiates laser light, acquires LiDAR data based on a reflection signal obtained by reflecting the laser light ([0107]: “LiDAR”), and calculates position information of the target object ([0116]: “analyzes three-dimensional point cloud data or image data acquired from various sensors or the external devices, generate position data and attribute data for the three-dimensional points,”) by applying a previously trained learning result ([0336]: “the artificial intelligence model before mounted on the artificial intelligence moving agent may extract a feature vector for the training object. The feature vector may indicate an illuminance of the image, a shape of the object, a color of the object, a position of the object, a relationship between the object and a background, or the like.”); and
a moving apparatus which moves the mobile object based on a position of the target object ([0046]: “self-driving car”), wherein the target object recognizing apparatus calculates position information of the target object based on LiDAR data of the target object ([0111]: “The self-driving vehicle 100b may use at least one of the map data, the object information detected from the sensor information, or the object information acquired from the external apparatus to determine the travel route and the travel plan, and may control the driving unit such that the self-driving vehicle 100b travels along the determined travel route and travel plan.”; [0112]: “map data may include object identification information about various objects arranged in the space (for example, road) in which the self-driving vehicle 100b travels.”; [0116], [0117]) by applying a weight determined based on the learning result by a reference target object to move the mobile object ([0337]: “the artificial intelligence model may output (classify) the type information of the training object by using the feature vector extracted from the training object.”; [0335]: “In the artificial intelligence model before mounted on the artificial intelligence moving agent, a parameter (at least one of weight or bias) of the artificial intelligence model may be determined by the training object.”),
and the moving apparatus moves the mobile object based on the position information ([0208]: “When the current position does not match the position on the map or when the current position is not able to be identified, the position recognition unit 5240 may recognize the current position and restore the current position of the robot cleaner 51. When the current position is restored, the travel control unit 5230 may allow the travel driving portion to move to a specified area based on the current position.”; [0225]).
Oh does not teach and Groh does teach performing deep learning based on virtual LiDAR data ([0051]: “After application of the training data 42, the machine learning module 72 may provide one or more ultrasonic mappings 88 and/or radar mappings 90 from three-dimensional modeling information to reflection values.”; [0059]) generated with respect to a virtual target object ([0031]) based on a reference map modeled from range data and intensity data ([0036]: “Such data may include intensity, phase, polarity, and/or Doppler shift information and may be resolved into different range bins.”) included in LiDAR data ([0035]) for a reference target object ([0068]: “Once the three-dimensional model is extended to a physical model simulating reflections from radar and/or ultrasonic waves, it may be used to develop, test, and evaluate algorithms 132 for processing radar data 68 and/or ultrasonic data 40 by utilizing virtual radar and/or ultrasonic data gathered from the simulated environment and/or simulated objects. Acquisition of such virtual datasets may be acquired by placing a virtual vehicle 34c within the simulation environment 130b coupled to the virtual sensor and capable of traveling a path through the simulation environment 130b.”).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Kummerle in view of Groh and further in view of Di Chele (US 2020/0150243 A1).
Regarding Claim 3, which depends from rejected Claim 2, Kummerle does not teach the LiDAR data is acquired for each of the plurality of reference target objects and the LiDAR data is sequentially acquired by changing a surface condition of the spherical photoreceptor ([0150]: “The different remittance values may be provided e.g. by a different surface finishing and/or by a different colour of the reference target 58 or of a portion(s) thereof.” The different colors imply that there are a plurality of photoreceptors).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kummerle in view of Groh with the teaching of Di Chele to use reference targets of different colors. It is well-known in the lidar arts that different materials have different properties in terms of reflectance, which can potentially bias position retrievals. A worker skilled in the arts would know that including reference targets with different properties such as color would yield a predictable result, namely, better and more robust calibrations which promote safety and adherence to policy.
Claims 4-6, 8, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Kummerle in view of Groh and further in view of Levinson (US 2017/0123428 A1).
Regarding Claim 4, which depends from rejected Claim 2, Kummerle does not teach and Levinson does teach wherein the learning step includes:
a reference generating step of generating a reference map by performing modeling based on the range data and the intensity data included in the LiDAR data ([0054]: “As such, the localizer may use acquired sensor data, such as sensor data associated with surfaces of buildings 115 and 117, which can be compared against reference data, such as map data (e.g., 3D map data, including reflectance data) to determine a local pose.:);
a virtual data generating step of generating virtual LiDAR data with respect to a virtual target object based on the reference map ([0117]: “reference data including three dimensional map data is received into a simulator. Dynamic object data defining motion patterns for a classified object may be retrieved at 2904. At 2906, a simulated environment is formed based on at least three dimensional (“3D”) map data and the dynamic object data.”); and
a weight determining step of determining a weight for recognizing a target object using the virtual LiDAR data ([0116]: “Simulator 2840 may also be used to determine vehicle dynamics properties and for calibration purposes. Further, simulator 2840 may be used to explore the space of applicable controls and resulting trajectories so as to effect learning by self-simulation.” [0077] teaches that the simulator simulates sensors by producing “simulated sensor data” which is identified here with virtual LiDAR data.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Levinson to utilize simulated LiDAR data into the method of Kummerle in view of Groh. Levinson notes in [0117] that “simulated vehicle commands are evaluated to determine whether the simulated autonomous vehicle behaved consistent with expected behaviors (e.g., consistent with a policy).” Therefore, incorporating these teachings is advantageous as it provides better vehicle compliance with existing road policies (laws) and thus improved safety.
Regarding Claim 5, which depends from rejected Claim 4, Kummerle does not teach and Levinson does teach wherein in the reference generating step, reference data is generated by modeling a plurality of data included in the LiDAR data and the reference map according to a surface condition of the reference target object is generated based on the reference data ([0054]: “As such, the localizer may use acquired sensor data, such as sensor data associated with surfaces of buildings 115 and 117, which can be compared against reference data, such as map data (e.g., 3D map data, including reflectance data) to determine a local pose.”; [0119]: “Further, logic 3144 receives sensor data and pose graph data 3145 to generate 3D map data 3120 (or other map data, such as 4D map data). In some examples, logic 3144 may implement a truncated sign distance function (“TSDF”) to fuse sensor data and/or map data to form optimal three-dimensional maps. Further, logic 3144 is configured to include texture and reflectance properties.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Levinson to incorporate surface condition data into the map generation into the method of Kummerle in view of Groh and further in view of Levinson. The incorporation of such information can result in better results from perception and classification engines, thereby improving vehicle compliance with local laws and safety.
Regarding Claim 6, which depends from rejected Claim 5, Kummerle further discloses wherein in the reference generating step, the reference data is generated by modeling at least one data of an angle Beam between a LiDAR emitting unit which irradiates laser light and the reference target object, a distance dBeam from the LiDAR emitting unit to a surface of the reference target object (Page 5586, Column 1, B: “For a depth sensor we always represent the observation as a 3D point.” A 3D point in Euclidian space is completely equivalent to a distance and angle formulation of the position), a distance dsph from the LiDAR emitting unit to a center of the reference target object, and a radius rsphof the reference target object, which are included in the LiDAR data (Page 5585, Equation 1 and surrounding text: the distance and radius of the calibration sphere are known or easily calculated and are used in conjunction with the LiDAR data).
Regarding Claim 8, which depends from rejected Claim 5, Kummerle does not teach and Levinson does teach wherein in the reference generating step, the reference map is generated using the reference data for every surface condition for a surface material and a surface color of the reference target object ([0072]: “Vehicle data controller 408 can cause map updater 406 to update reference data within repository 405 and facilitate updates to 2D, 3D, and/or 4D map data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Levinson to update a value of the reference data into the method of Kummerle in view of Groh. LiDAR systems regularly collect data about their surroundings, and that data, or derived products such as recognized features, can be incorporated as new reference data throughout the operational period. These updates can improve safety by ensuring that high-quality and improved references are used in the system.
Regarding Claim 13, which depends from rejected Claim 12, Kummerle further discloses wherein in the acquiring step, LiDAR data for a spherical photoreceptor type reference target object is acquired and the LiDAR data is acquired based on a reflectance of the reflection signal (Page 5585, Column 2: “A spherical target meets these requirements. For our experiments we choose an off-the-shelf white Styrofoam sphere that can be purchased in a regular hardware store.”) excluding a signal which is scattered from the spherical photoreceptor (The geometry of the setup (Figs. 1 and 5) indicates that most light that undergoes diffuse or Lambertian reflection off of the spherical photoreceptor will not be collected by the LiDAR).
Kummerle does not teach and Levinson does teach wherein the learning step includes a reference generating step of generating a reference map by performing modeling based on the range data and the intensity data included in the LiDAR data ([0054]: “As such, the localizer may use acquired sensor data, such as sensor data associated with surfaces of buildings 115 and 117, which can be compared against reference data, such as map data (e.g., 3D map data, including reflectance data) to determine a local pose.:);
a virtual data generating step of generating virtual LiDAR data with respect to a virtual target object based on the reference map ([0117]: “reference data including three dimensional map data is received into a simulator. Dynamic object data defining motion patterns for a classified object may be retrieved at 2904. At 2906, a simulated environment is formed based on at least three dimensional (“3D”) map data and the dynamic object data.”); and
a weight determining step of determining a weight for recognizing a target object using the virtual LiDAR data ([0116]: “Simulator 2840 may also be used to determine vehicle dynamics properties and for calibration purposes. Further, simulator 2840 may be used to explore the space of applicable controls and resulting trajectories so as to effect learning by self-simulation.” [0077] teaches that the simulator simulates sensors by producing “simulated sensor data” which is identified here with virtual LiDAR data.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Levinson to utilize simulated LiDAR data into the method of Kummerle in view of Groh. Levinson notes in [0117] that “simulated vehicle commands are evaluated to determine whether the simulated autonomous vehicle behaved consistent with expected behaviors (e.g., consistent with a policy).” Therefore, incorporating these teachings is advantageous as it provides better vehicle compliance with existing road policies (laws) and thus improved safety.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Kummerle in view of Groh and further in view of Levinson as applied to Claim 5 above, and further in view of Di Chele.
Regarding Claim 7, Kummerle in view of Groh and further in view of Levinson does not teach and Di Chele does teach wherein in the reference generating step, the reference map is generated using the reference data for every surface condition for a surface material and a surface color of the reference target object ([0150]: “The different remittance values may be provided e.g. by a different surface finishing and/or by a different colour of the reference target 58 or of a portion(s) thereof.” The different colors imply that there are a plurality of photoreceptors).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kummerle in view of Groh and further in view of Levinson with the teaching of Di Chele to use reference targets of different colors. It is well-known in the lidar arts that different materials have different properties in terms of reflectance, which can potentially bias position retrievals. A worker skilled in the arts would know that including reference targets with different properties such as color would yield a predictable result, namely, better and more robust calibrations which promote safety and adherence to policy.
Claims 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Kummerle in view of Groh and further in view of Levinson as applied to Claim 4 above, and further in view of Kim (2020/0034661 A1).
Regarding Claim 9, which depends from rejected Claim 4, Kummerle in view of Groh and further in view of Levinson does not teach and Kim does teach wherein in the weight determining step, the deep learning is performed based on the virtual LiDAR data and the weight is determined based on the learning result ([0060]; [0176]: “the target AI model may be learned by supervised learning using the generated training data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kummerle in view of Groh and further in view of Levinson with the teaching of Kim to use generated training data (that is in the nomenclature of the instant application, virtual LiDAR data) in the device. It is well-known in the art to use artificial or generated training data to improve instrument function in situations where actual training data are sparse, and a worker skilled in the art would have been able to implement such a function with a reasonable expectation of success.
Regarding Claim 10, which depends from rejected Claim 4, Kummerle in view of Levinson does not teach and Kim does teach wherein in the weight determining step, the deep learning is performed by mixing the virtual LiDAR data and the LiDAR data with respect to the reference target object and the weight is determined based on the learning result ([0060]; [0193]: “the processor 180 may reproduce previous training data corresponding to the previous learning task of the generative model by using the generative model, and learn the generated model by using the reproduced previous training data and the generated training data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kummerle in view of Groh and further in view of Levinson with the teaching of Kim to use both training data and generated training data (that is in the nomenclature of the instant application, LiDAR data and virtual LiDAR data) in the device. It is well-known in the art to use artificial or generated training data to improve instrument function in situations where actual training data are sparse, and a worker skilled in the art would have been able to implement such a function with a reasonable expectation of success.
Regarding Claim 11, which depends from rejected Claim 9, Kummerle further discloses a target recognizing step of recognizing a new target object based on new LiDAR data;
and a recognition result calculating step of calculating position information of the new target object by applying the weight determined based on the learning result (Page 5584, Column 1: “Today, robots are deployed in more and more complex environments to support us in daily life. To perceive their environment they are equipped with a variety of sensors. By fusing data from multiple sensors, better robustness and performance can be achieved in tasks like object detection or localization. For successful sensor fusion an accurate calibration of the sensor setup is essential. Sensor fusion is especially effective with diverse sensors such as cameras and depth sensors that measure in different domains.” This indicates that a recognition step is performed after the calibration is successfully completed. This step necessarily requires calculation of positions as the underlying data used is LiDAR data.)
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kummerle in view of Groh and further in view of Levinson as applied to claim 13 above, and further in view of Kim (2020/0034661 A1).
Regarding Claim 14, which depends from rejected Claim 13, Kummerle in view of Groh does not teach and Levinson does teach wherein in the reference generating step, reference data is generated by modeling a plurality of data included in the LiDAR data and the reference map according to a surface condition of the reference target object is generated based on the reference data ([0054]: “As such, the localizer may use acquired sensor data, such as sensor data associated with surfaces of buildings 115 and 117, which can be compared against reference data, such as map data (e.g., 3D map data, including reflectance data) to determine a local pose.”; [0119]: “Further, logic 3144 receives sensor data and pose graph data 3145 to generate 3D map data 3120 (or other map data, such as 4D map data). In some examples, logic 3144 may implement a truncated sign distance function (“TSDF”) to fuse sensor data and/or map data to form optimal three-dimensional maps. Further, logic 3144 is configured to include texture and reflectance properties.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Levinson to incorporate surface condition data into the map generation into the method of Kummerle in view of Groh and further in view of and Levinson. The incorporation of such information can result in better results from perception and classification engines, thereby improving vehicle compliance with local laws and safety.
Kummerle in view of Levinson does not teach and Kim does teach wherein in the weight determining step, the deep learning is performed based on the virtual LiDAR data, or the deep learning is performed by mixing the virtual LiDAR data and the LiDAR data with respect to the reference target object and the weight is determined based on the learning result ([0060]; [0193]: “the processor 180 may reproduce previous training data corresponding to the previous learning task of the generative model by using the generative model, and learn the generated model by using the reproduced previous training data and the generated training data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kummerle in view of Groh and further in view of Levinson with the teaching of Kim to use both training data and generated training data (that is in the nomenclature of the instant application, LiDAR data and virtual LiDAR data) in the device. It is well-known in the art to use artificial or generated training data to improve instrument function in situations where actual training data are sparse, and a worker skilled in the art would have been able to implement such a function with a reasonable expectation of success.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN WADE CLOUSER whose telephone number is (571)272-0378. The examiner can normally be reached M-F 7:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ISAM ALSOMIRI can be reached at (571) 272-6970. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.W.C./Examiner, Art Unit 3645
/ISAM A ALSOMIRI/Supervisory Patent Examiner, Art Unit 3645
Prosecution Insights