DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
The claims 1-13 and 15-19 are currently pending and have been examined.
Response to Arguments/Amendments
The amendment filed February 5, 2025 has been entered. Claims 1-13 and 15-19 are currently pending in the Application.
Applicant's arguments with respect Claims 1-13 and 15-19 under 35 U.S.C. 103 have been fully considered but they are not persuasive. Applicant’s argument is that the cited combination fails to teach or suggest verifying, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as set forth in amended independent claims 1 and 13. Applicant additionally argues that the cited references do not provide continuous online verification of underlying assumptions and therefore fail to achieve the alleged technical advantages of the claimed invention (See Applicant’s Remarks, pages 5-9). The Examiner respectfully disagrees. Applicant’s arguments do not address the specific findings set forth in the rejection. In particular, the rejection does not rely upon Deusch or Nister for teaching the argued online plausibility determination and statistical hypotheses testing limitations. Rather, Fritz is relied upon for the amended limitations, as set forth in the rejection. Fritz teaches plausibility testing and hypothesis-test-based evaluation of statistical assumptions, including the use of hypothesis tests for assessment of statistical independence (See paragraph [0347].), evaluation of statistical assumptions (See paragraph [0377].), and significance/probability threshold analysis (See paragraph [0379], [0549].). Accordingly, the Examiner maintains that the combined teachings of Deusch, Nister, and Fritz teach the argued limitations. Applicant’s arguments regarding continuous online verification and the alleged advantages of the claimed invention are not persuasive because the claims do not recite the particular advantages alleged by the Applicant. Furthermore, Applicant has no provided objective evidence demonstrating that any alleged results would have been unexpected relative to the teachings of the prior art. Attorney argument alone is insufficient to overcome the prima facie case of obviousness. With respect to claims 16 and 18, Applicant argues that Fritz fails to teach reporting deviations and integration with other modules. The Examiner respectfully disagrees. As set forth in the rejection, Fritz is relied upon for the argued limitations. The Applicant has not identified error in the Examiner’s findings regarding Fritz and instead merely asserts that Fritz fails to teach the claimed subject matter. With respect to claims 17 and 19, the Applicant argues that Sadeghi fails to teach detection probability continuously during operation. The Examiner respectfully disagrees. As set forth in the rejection, Sadeghi teaches iterative determination of uncertainty estimation parameters over multiple iterations, where parameter values are repeatedly updated based on prior determinations (See paragraph [0030]-[0032].). Sadeghi further teaches confidence/probability measures associated with measurement correctness (See paragraph [0153].), and covariance-based uncertainty estimation during interference (See paragraph [0317].). Accordingly, Sadeghi teaches continual estimation of measurement-model parameters, including probability-related parameters, during operation. The Applicant has not identified error in the Examiner’s findings and merely asserts that Sadeghi fails to teach the claimed subject matter. No separate arguments have been presented with respect to the remaining rejected claims. Accordingly, the rejections under 35 U.S.C. 103 are maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-5, 7, 9, 13, 16, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Deusch, ("Random Finite Set-Based Localization and SLAM for Highly Automated Vehicles”) in view of Nister (US 20190243371 A1) and Fritz (US 20110307217 A1).
Regarding Claim 1, Deusch teaches A map validation method, comprising: receiving sensor data of an at least semi-autonomous robot for depicting at least one detected element, wherein the at least one detected element represents an environmental element of the at least semi-autonomous robot as detected by an environmental sensor of the at least semi-autonomous robot (See at least section 3.1.1, “The vehicle’s default on-board sensory equipment comprises a wide range of different sensors: five radars, two cameras and many more smaller sensors of various kinds. The data of some of these sensors is used within the algorithms presented in this work… Three types of sensors that are of great importance for the work presented in this thesis have been added to the vehicle: First of all, three laser scanners were integrated into the front bumper as described in Section 3.1.3. Further, three additional cameras were installed, see Section 3.1.4. And finally, three rearward facing radars (3.1.5) were added. Another important source of measurement data is the Real Time Kinematic (System) (RTK) which has an integrated module that facilitates using Differential Global Positioning System (DGPS) (see 3.1.6). This system was used to generate ground truth localization data”, section 3.3, Fig. 8, “First of all, the storing system has to provide the requested landmarks to the localization algorithm as fast as possible, in order to facilitate a high update rate of the pose estimate. Thus, it makes sense to use a system that facilitates geospatial queries as this reduces the complexity to find the requested landmarks…Secondly, the data storage system has to be flexible so that it can handle different types of landmarks found on entirely different routes. Further, it should be relatively easy to update the landmarks in order to be able to quickly react to changes in the environment or route layout. Another requirement is that the system is not limited to storing landmarks, instead it has to be usable by other modules as well, e.g., by the trajectory planning algorithm, which needs a globally defined route to plan the trajectory of the vehicle.”); receiving map data depicting a map with at least one map element, wherein the at least one map element represents an environment element of the at least semi-autonomous robot as plotted on a predetermined map (See at least section 4.2.2, “Thus, all detections that originate from stationary objects possessing a reflectivity within a certain range are used as landmark candidates. Typical examples for those kind of objects within the vehicular environment are: traffic lights, lamp posts, traffic signs and other metallic objects… A sophisticated filtering algorithm has to be used in order to cope with the false positive rate of the raw radar measurements, the relatively high measurement uncertainty and the non-trivial association between landmark candidates and measurements” and section 4.3, “Although this chapter focuses on the localization of a highly automated vehicle, generating a map prior to localizing is inevitable…Generating a map that contains the types of landmarks presented before is a relatively easy process. The whole map can be generated by simply driving the route to be mapped once while localizing the vehicle with a precise DGPS. The vehicle that has been used to test and to evaluate the proposed localization algorithm possesses such a system, the ADMA (cf. Section 3.1.6).”); receiving localization data indicating a position of the at least semi-autonomous robot on the map (See at least section 4.4, “This grid map is an input to the MSER detector which then provides the detected features as an input to the particle filter’s weight update process. Besides these features and the detections of the radar, also the MSER features that are obtained directly from the camera images, are fed into the update process (cf. Section 4.2.1). Obviously, the filtering algorithm not only needs the current feature detections but also requires a map of landmarks in order to assess the hypotheses about the vehicle’s pose represented by the particles. This map is provided by the database component (cf. Section 3.3).”); initializing an existence probability for the at least one map element with an initial value (See at least section 5.6.2, “The position and spatial uncertainty of this new landmark can be extracted from the spatial distribution p(l) of the track labeled l. On the other hand, the existence probability in the database pex should not be a copy of the existence probability of the track r(l), although all tracks possess one, as they are represented by an LMB-RFS, cf. Section 5.4.1. The reason for this is that the information stored in the database should represent “long-term knowledge” and the output of the RB-LMB-SLAM can only represent “short-term knowledge”. Thus, setting or updating existence probabilities in the database should resemble a low-pass filter behavior. For example, a landmark that was found by the SLAM algorithm with a high existence probability will not necessarily be there the next time the vehicle passes, which again a high pex would imply. Hence, the initial pex should be set to a relatively low value (which still may depend on the tracks existence probability).”); updating the existence probability of the at least one map element using the map data, the sensor data, the localization data, and the data uncertainties (See at least section 5.1, "Further, the measurement model that basically describes how landmarks are perceived by the vehicle’s sensor can also be regarded as a probabilistic function. In this case, measuring a feature depends on the current state of the vehicle xk and the actual landmark map Mk", section 5.3.1, "Thereby, the likelihood that the vehicle’s sensor measures z based on m and xk is given by g(z\m, xk). The feature measurement RFS therefore directly incorporates the handling of spatial uncertainty (through g(z | m, xk)) and the uncertainty of detection [AMV13]. Thus, in combination with the consideration of clutter measurements (by Ck(xk)) this approach obviously enables the integrated handling of all kinds of measurement uncertainties", and section 5.6.2: "After this update, it has to be checked if the existence probability of the landmarks in the database still is above a certain threshold. If it has fallen below the threshold, the respective landmarks have to be deleted from the database."); determining a robot trajectory using the sensor data, the map data, the localization data; and the existence probability of the at least one map element (See at least section 4.5, “The proposed localization scheme was evaluated on three different scenarios: on an urban road, on a rural road and on a route that combines both kinds of roads. The latter was also driven by a highly automated vehicle, without any driver intervention using the proposed localization algorithm. All scenarios were evaluated with respect to the overall achieved pose accuracy. This accuracy is described by the mean error averaged over the whole route and the respective standard deviation. The ground truth pose was provided by the ADMA (see Section 3.1.6). Further, all runs that exhibited a position error above terr = 5m at any time were marked as “failed” and were excluded from the calculation of the mean error... Further, only the central laser scanner (cf. Section 3.1.3), the forward facing camera (cf. Section 3.1.4), and the two rearward facing mid-range radars (cf. Section 3.1.3) were used.”).
Deusch does not explicitly disclose, however, Nister, in the same field of endeavor, teaches determining a data uncertainty, that comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty (See at least paragraph [0059], “The autonomous vehicle 102 may include a sensor manager 108. The sensor manager 108 may manage and/or abstract sensor data from sensors of the vehicle 102. For example, and with reference to FIG. 11C, the sensor data may be generated (e.g., perpetually, at intervals, based on certain conditions) by global navigation satellite system (GNSS) sensor(s) 1158, RADAR sensor(s) 1160, ultrasonic sensor(s) 1162, LIDAR sensor(s) 1164, inertial measurement unit (IMU) sensor(s) 1166, microphone(s) 1196, stereo camera(s) 1168, wide-view camera(s) 1170, infrared camera(s) 1172, surround camera(s) 1174, long range and/or mid-range camera(s) 1198, and/or other sensor types”, paragraph [0068], “The planning component(s) 124, which may be part of a planning layer of the autonomous driving software stack or architecture, may include a route planner, a lane planner, a behavior planner, and a behavior selector, among other components, features, and/or functionality. The route planner may use the information from the map perceiver 116, the map manager 118, and/or the localization manger 120, among other information, to generate a planned path that may consist of GNSS waypoints (e.g., GPS waypoints). The waypoints may be representative of a specific distance into the future for the vehicle 102, such as a number of city blocks, a number of kilometers, a number of feet, a number of miles, etc., that may be used as a target for the lane planner” and paragraph [0069], “The lane planner may use the lane graph (e.g., the lane graph from the path perceiver 112), object poses within the lane graph (e.g., according to the localization manager 120), and/or a target point and direction at the distance into the future from the route planner as inputs. The target point and direction may be mapped to the best matching drivable point and direction in the lane graph (e.g., based on GNSS and/or compass direction). A graph search algorithm may then be executed on the lane graph from a current edge in the lane graph to find the shortest path to the target point.” The vehicle depends on various sensors for perception and they are subject to noise and error, accounting for sensor data uncertainty. The system relies on map data that varies with changes in the road or environment, showing map data uncertainty. The process of mapping a target point into the lane graph and performing a graph search requires a current position estimate of the vehicle relative to the map, which is generated by the localization manager depending on localization data uncertainty.); controlling the at least semi-autonomous robot in control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory (See at least paragraph [0053], “The current system may determine states and safety procedures for each object (perceived and unperceived, static and moving) in the environment. The system may then generate virtual representations of the points in space-time the objects will occupy (e.g., object-occupied trajectories) when executing their respective safety procedures. To determine unperceived objects in the environment, the system may use rules that are based on reasonable expectations. For example, when an intersection is occluded in some way, the system may generate an unperceived object as a car traveling at a reasonable speed approaching the intersection from the occluded direction. As another example, if two cars are parked on the side of the street, the system may generate an unperceived object as a person between the two cars. The unperceived objects may be included in the analysis by the system in addition to the perceived objects (e.g., objects identified based on sensor data from sensors of the vehicle)”, paragraph [0054], “The system may then monitor the vehicle-occupied trajectory(ies) in view of the object-occupied trajectories to determine if an intersection or overlap occurs (in some examples, with a safety margin built in). Once it is determined that an intersection occurs, the system may implement a “safety force field” and seek to “repel” the vehicle from the object(s) by implementing the safety procedure or by implementing another set of controls that has been determined to have an equal or lesser associated likelihood and/or imminence of causing an actual collision between the vehicle and any of the objects”, paragraph [0056], “During implementation of the safety procedure or the other action(s), when it is determined that there is no longer an overlap or intersection between the vehicle-occupied trajectory(ies) and the object-occupied trajectories (e.g., the imminence and/or likelihood of a collision is reduced), the system may cease implementing the safety procedure or the other action(s), and give control back to a higher layer of the autonomous driving software stack (e.g., a planning layer and/or control layer) associated with controlling the vehicle according to normal driving protocols (e.g., obeying rules of the road, following the current directions, etc.). In some examples, the action(s) selected may depend on what the higher layer of the stack originally planned to implement (e.g., proposed controls representative of a planned trajectory), and the system may select the closest safe controls and/or safe trajectories to the original planned action(s) (e.g., using one or more metrics)”, and paragraph [0069], “The lane planner may use the lane graph (e.g., the lane graph from the path perceiver 112), object poses within the lane graph (e.g., according to the localization manager 120), and/or a target point and direction at the distance into the future from the route planner as inputs. The target point and direction may be mapped to the best matching drivable point and direction in the lane graph (e.g., based on GNSS and/or compass direction). A graph search algorithm may then be executed on the lane graph from a current edge in the lane graph to find the shortest path to the target point.” Determining safety procedures to avoid hazards corresponds to the preventative safety mode and casing operation and overriding control when overlap is detected corresponds to the safety stop mode.).
Deusch and Nister do not explicitly disclose, however, Fritz, in the same field of endeavor, teaches and verifying, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level (See at least paragraph [0347], “A total of three different methods were investigated for assessment of statistical independence. In contrast to the calculation of the autocorrelation coefficient (AKK), the successive difference scatter and the correlation test according to R. A. Fischer relate to hypothesis tests”, paragraph [0377], “If the assumptions relating to independence and normal distribution are severely infringed, then this leads, particularly in the case of correlated data, to an incorrect calculation of the control limits”, paragraph [0379], “the probability of occurrence of one of the described cases, being p=0.0027, should not be considered random, but should be considered to be a fault since, in the case of normally distributed data, 99.73% of the data considered will be within .+-.3s”, and paragraph [0549], “The test variable t.sub.fault isolation is calculated using the approach that every plausibility method produces a binary test result E.sub.bin, method. The method result E.sub.method is calculated by multiplying the binary method result E.sub.bin, method by the method separation sharpness p.sub.method (equation 1.2).” The system performs online plausibility determination using hypothesis tests to evaluate whether statistical assumptions remain valid under predefined significance thresholds, thereby determining whether assumed parameters deviate from observed data beyond acceptable limits.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister and Fritz such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), and to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].).
Regarding Claim 2, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch teaches further comprising: projecting the at least one map element into a sensor space of the environmental sensor (See at least section 4.4.3, “The weight update basically assesses the particle states based on the obtained measurements. Thus, the main objective is to find a sensible implementation of the measurement likelihood function: g(Zk | x (i) k), i.e., the likelihood of obtaining the set of measurements Zk given the state x (i) k . If one takes a closer look at the concrete case of MCL that means: The goal is to find the likelihood of having measured a set of landmarks given the hypothesis of the vehicle pose that is represented by the particle state. This pose directly translates to a set of landmarks of the a-priori constructed map M (cf. Section 4.3) which contains all landmarks that were in the FOV of the vehicle’s sensors, if the vehicle’s pose actually matched the pose described by the particle’s state. This set of landmarks is then transformed from the global UTM coordinate frame _utmto the local coordinate frame of the vehicle _veh(cf. Section 3.2.2): M ∩ FOV (xk (i) )ζveh−−→ M¯ F oV(xk (i) ). Given this set of transformed landmarks within the FOV of the respective particle, assessing the particle’s weight essentially boils down to finding a multi-object likelihood: ω(i)∼g(Zk | M¯ F oV(xk (i))). When using this multi-object likelihood description, the similarities to the approaches discussed in Section 2.3 become obvious. Basically, the predicted multi-object state X+ of equation (2.32) is replaced by an expected map state given the vehicle’s pose.” Projecting into a space is changing the reference frame, global UTM coordinate frame to the local coordinate frame, of a map element to align with sensor measurements. The map elements are the pre-existing set of known landmarks used for localization from the a-priori constructed map M. The sensor space of the environmental sensor is the FOV of the vehicle’s sensors.).
Regarding Claim 3, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch teaches further comprising: assigning the at least one map element to the at least one detected element (See at least section 4.4.3, “The weight update basically assesses the particle states based on the obtained measurements. Thus, the main objective is to find a sensible implementation of the measurement likelihood function: g(Zk | x (i) k), i.e., the likelihood of obtaining the set of measurements Zk given the state x (i) k . If one takes a closer look at the concrete case of MCL that means: The goal is to find the likelihood of having measured a set of landmarks given the hypothesis of the vehicle pose that is represented by the particle state. This pose directly translates to a set of landmarks of the a-priori constructed map M (cf. Section 4.3) which contains all landmarks that were in the FOV of the vehicle’s sensors, if the vehicle’s pose actually matched the pose described by the particle’s state. This set of landmarks is then transformed from the global UTM coordinate frame _utmto the local coordinate frame of the vehicle _veh(cf. Section 3.2.2): M ∩ FOV (xk (i) )ζveh−−→ M¯ F oV(xk (i) ). Given this set of transformed landmarks within the FOV of the respective particle, assessing the particle’s weight essentially boils down to finding a multi-object likelihood: ω(i)∼g(Zk | M¯ F oV(xk (i))). When using this multi-object likelihood description, the similarities to the approaches discussed in Section 2.3 become obvious. Basically, the predicted multi-object state X+ of equation (2.32) is replaced by an expected map state given the vehicle’s pose.” Assigning is assessing the particle’s weight using the likelihood function to determine how well an observed element, sensor data, matches a known map element. The map elements are the pre-existing set of known landmarks used for localization from the a-priori constructed map M. The detected elements are the set of real-world landmarks observed by the sensor.).
Regarding Claim 4, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch teaches further comprising: evaluating the existence probability of the at least one map element, wherein the evaluating comprises one of confirming the map element, disproving the map element, potentially new map element, or no possible statement (See at least section 5.4, “Since all tracks provide an existence probability r(l), one can simply apply a thresholding mechanism on that value, so that the set of extracted tracks is described by M = {(mˆ , l)|rl >ϑ}, given the threshold ϑ. Further, also a hysteresis can be used in order to enable a continuous output of tracks that already have been confirmed earlier but have then been missed during the last few consecutive updates. To this end, the maximum existence probability r(l) max of any track is stored as well. If the current probability of existence r(l) is above a lower threshold ϑ1 and the existence probability has previously exceeded an upper threshold ϑu, the track will be part of the set of tracks that constitute the output of the filtering algorithm (See equation 5.76) Consequently, such a threshold can not only be used to prevent tracks from contributing to the output, but also to enable a pruning scheme, which deletes all tracks that have an existence probability below a further threshold ϑd.” A map element is confirmed when the probability existence is above the upper threshold and the element is retained. If below the threshold, the element is removed. New tracks are created when measurements are not associated with existing tracks. If the probability existence falls in an uncertain range between thresholds, no decision is made.).
Regarding Claim 5, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch teaches wherein the updating the existence probability comprises a random finite set approach or a logit approach (See at least section 5.3, “As with FastSLAM, the vehicle transition density is chosen as the proposal density in RB-PHD-SLAM [MVAV11b]. Therefore, the weight of particle i can similarly be calculated by:ωk(i) = g(Zk|Z0:k−1, x 0:k (i) )ω(i)k−1. The actual calculation of the particle weights nevertheless differs severely from the weight calculation in FastSLAM since it is not possible to approximate the measurement likelihood under an EKF-Framework when using random finite sets…The following three strategies are the most common and have been published in [MVAV11b] and [LIA13] respectively: The empty map strategy…The single-feature strategy…The multi-feature strategy” and section 5.4, “To achieve this, the probability that a measurement z e Z(i), (Z(i) contains the measurements being in group i, cf. the paragraph about grouping below), has been assigned to an existing track is calculated by (cf. [Reu14] See equation 5.48)… Based on this finding and equation (5.48), the birth existence probability rb(l) (with l e B) can be determined by (cf. [Reu14] See equation 5.49)… where the expected number of new born objects is lambdaB. The minimum operator ensures that a maximum birth existence probability rmax B e [0, 1] is not exceeded.” Updating the probability of existence using is the probability that a measurement z has been assigned to an existing track using sensor data. The existence probability, equation 5.49, represents the likelihood a detected object, landmark, truly exists. The random finite set approach includes Poisson distributed clutter and multi-feature strategy approaches that model sensor measurements as possible clutter and apply probabilistic updates.).
Regarding Claim 7, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch teaches wherein updating the existence probability of the at least one map element is repeated in a temporal interval (See at least section 5.6.2, figures 5.5, 5.6, “For example, a landmark that was found by the SLAM algorithm with a high existence probability will not necessarily be there the next time the vehicle passes, which again a high pex would imply. Hence, the initial pex should be set to a relatively low value (which still may depend on the tracks existence probability). The second case, where existing landmarks have to be updated by the output of the SLAM algorithm can be processed as follows: First, those landmarks which have been passed to the SLAM component (cf. Figure 5.9) and which have a significant spatial likelihood of corresponding to the track labeled l are selected. Then their positions and existence probabilities are updated using the data from the track (p(l) and r(l)) and the learning rate n. This learning rate is a function of the likelihood of the track corresponding to the landmark being updated and of the number of times this landmark has already been updated, i.e., the more likely a track corresponds to the landmark, the higher the learning rate and the older the landmark (high number of updates) the lower the learning rate.” The learning rate n is the temporal interval.).
Regarding Claim 9, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch teaches wherein the data uncertainty is used to determine the visibility of the at least one map element (See at least section 4.4.3, “The weight update basically assesses the particle states based on the obtained measurements. Thus, the main objective is to find a sensible implementation of the measurement likelihood function: g(Zk | x (i) k), i.e., the likelihood of obtaining the set of measurements Zk given the state x (i) k . If one takes a closer look at the concrete case of MCL that means: The goal is to find the likelihood of having measured a set of landmarks given the hypothesis of the vehicle pose that is represented by the particle state. This pose directly translates to a set of landmarks of the a-priori constructed map M (cf. Section 4.3) which contains all landmarks that were in the FOV of the vehicle’s sensors, if the vehicle’s pose actually matched the pose described by the particle’s state. This set of landmarks is then transformed from the global UTM coordinate frame _utmto the local coordinate frame of the vehicle _veh(cf. Section 3.2.2): M ∩ FOV (xk (i) )ζveh−−→ M¯ F oV(xk (i) ). Given this set of transformed landmarks within the FOV of the respective particle, assessing the particle’s weight essentially boils down to finding a multi-object likelihood: ω(i)∼g(Zk | M¯ F oV(xk (i))). When using this multi-object likelihood description, the similarities to the approaches discussed in Section 2.3 become obvious. Basically, the predicted multi-object state X+ of equation (2.32) is replaced by an expected map state given the vehicle’s pose.” The visibility is when the landmarks are within the vehicle’s estimated field of view.).
Regarding Claim 13, Deusch teaches A map validation system, comprising: at least one computer processing system which is configured to perform procedures comprising: receiving sensor data of an at least semi-autonomous robot for depicting at least one detected element, wherein the at least one detected element represents an environmental element of the at least semi-autonomous robot as detected by an environmental sensor of the at least semi-autonomous robot (See at least section 3.1.1, “The vehicle’s default on-board sensory equipment comprises a wide range of different sensors: five radars, two cameras and many more smaller sensors of various kinds. The data of some of these sensors is used within the algorithms presented in this work… Three types of sensors that are of great importance for the work presented in this thesis have been added to the vehicle: First of all, three laser scanners were integrated into the front bumper as described in Section 3.1.3. Further, three additional cameras were installed, see Section 3.1.4. And finally, three rearward facing radars (3.1.5) were added. Another important source of measurement data is the Real Time Kinematic (System) (RTK) which has an integrated module that facilitates using Differential Global Positioning System (DGPS) (see 3.1.6). This system was used to generate ground truth localization data”, section 3.3, Fig. 8, “First of all, the storing system has to provide the requested landmarks to the localization algorithm as fast as possible, in order to facilitate a high update rate of the pose estimate. Thus, it makes sense to use a system that facilitates geospatial queries as this reduces the complexity to find the requested landmarks…Secondly, the data storage system has to be flexible so that it can handle different types of landmarks found on entirely different routes. Further, it should be relatively easy to update the landmarks in order to be able to quickly react to changes in the environment or route layout. Another requirement is that the system is not limited to storing landmarks, instead it has to be usable by other modules as well, e.g., by the trajectory planning algorithm, which needs a globally defined route to plan the trajectory of the vehicle.”); receiving map data depicting a map with at least one map element, wherein the at least one map element represents an environment element of the at least semi- autonomous robot as plotted on a predetermined map (See at least section 4.2.2, “Thus, all detections that originate from stationary objects possessing a reflectivity within a certain range are used as landmark candidates. Typical examples for those kind of objects within the vehicular environment are: traffic lights, lamp posts, traffic signs and other metallic objects… A sophisticated filtering algorithm has to be used in order to cope with the false positive rate of the raw radar measurements, the relatively high measurement uncertainty and the non-trivial association between landmark candidates and measurements” and section 4.3, “Although this chapter focuses on the localization of a highly automated vehicle, generating a map prior to localizing is inevitable…Generating a map that contains the types of landmarks presented before is a relatively easy process. The whole map can be generated by simply driving the route to be mapped once while localizing the vehicle with a precise DGPS. The vehicle that has been used to test and to evaluate the proposed localization algorithm possesses such a system, the ADMA (cf. Section 3.1.6).”); receiving localization data, the localization data indicating a position of the at least semi-autonomous robot on the map (See at least section 4.4, “This grid map is an input to the MSER detector which then provides the detected features as an input to the particle filter’s weight update process. Besides these features and the detections of the radar, also the MSER features that are obtained directly from the camera images, are fed into the update process (cf. Section 4.2.1). Obviously, the filtering algorithm not only needs the current feature detections but also requires a map of landmarks in order to assess the hypotheses about the vehicle’s pose represented by the particles. This map is provided by the database component (cf. Section 3.3).”); determining a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and/or a localization data uncertainty (See at least section 4.4.3, “The goal is to find the likelihood of having measured a set of landmarks given the hypothesis of the vehicle pose that is represented by the particle state. This pose directly translates to a set of landmarks of the a-priori constructed map M (cf. Section 4.3) which contains all landmarks that were in the FOV of the vehicle’s sensors, if the vehicle’s pose actually matched the pose described by the particle’s state…All associations have to be unique, since each and every landmark generates at most one measurement and all measurements may originate from at most one of the landmarks. Further, more than one landmark may be assigned to the measurement “0”, which represents a missed detection. The Munkres algorithm [Mun57] is used to find those n most optimal assignments. In order to find these assignments, the likelihoods for all possible measurement to landmark associations have to be calculated and also the missed detections have to be considered. Since the Munkres algorithm works on costs rather than likelihoods, the costs corresponding to the likelihoods have to be determined.”); initializing an existence probability for the at least one map element with an initial value (See at least section 5.6.2, “The position and spatial uncertainty of this new landmark can be extracted from the spatial distribution p(l) of the track labeled l. On the other hand, the existence probability in the database pex should not be a copy of the existence probability of the track r(l), although all tracks possess one, as they are represented by an LMB-RFS, cf. Section 5.4.1. The reason for this is that the information stored in the database should represent “long-term knowledge” and the output of the RB-LMB-SLAM can only represent “short-term knowledge”. Thus, setting or updating existence probabilities in the database should resemble a low-pass filter behavior. For example, a landmark that was found by the SLAM algorithm with a high existence probability will not necessarily be there the next time the vehicle passes, which again a high pex would imply. Hence, the initial pex should be set to a relatively low value (which still may depend on the tracks existence probability).”); updating the existence probability of the at least one map element using the map data, the sensor data, the localization data, and the data uncertainties (See at least section 5.1, "Further, the measurement model that basically describes how landmarks are perceived by the vehicle’s sensor can also be regarded as a probabilistic function. In this case, measuring a feature depends on the current state of the vehicle xk and the actual landmark map Mk", section 5.3.1, "Thereby, the likelihood that the vehicle’s sensor measures z based on m and xk is given by g(z\m, xk). The feature measurement RFS therefore directly incorporates the handling of spatial uncertainty (through g(z | m, xk)) and the uncertainty of detection [AMV13]. Thus, in combination with the consideration of clutter measurements (by Ck(xk)) this approach obviously enables the integrated handling of all kinds of measurement uncertainties", and section 5.6.2: "After this update, it has to be checked if the existence probability of the landmarks in the database still is above a certain threshold. If it has fallen below the threshold, the respective landmarks have to be deleted from the database.") determining a robot trajectory using the sensor data, the map data, the localization data; and the existence probability of the at least one map element (See at least section 4.5, “The proposed localization scheme was evaluated on three different scenarios: on an urban road, on a rural road and on a route that combines both kinds of roads. The latter was also driven by a highly automated vehicle, without any driver intervention using the proposed localization algorithm. All scenarios were evaluated with respect to the overall achieved pose accuracy. This accuracy is described by the mean error averaged over the whole route and the respective standard deviation. The ground truth pose was provided by the ADMA (see Section 3.1.6). Further, all runs that exhibited a position error above terr = 5m at any time were marked as “failed” and were excluded from the calculation of the mean error... Further, only the central laser scanner (cf. Section 3.1.3), the forward facing camera (cf. Section 3.1.4), and the two rearward facing mid-range radars (cf. Section 3.1.3) were used.”).
Deusch does not explicitly disclose, however, Nister, in the same field of endeavor, teaches and controlling the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory (See at least paragraph [0053], “The current system may determine states and safety procedures for each object (perceived and unperceived, static and moving) in the environment. The system may then generate virtual representations of the points in space-time the objects will occupy (e.g., object-occupied trajectories) when executing their respective safety procedures. To determine unperceived objects in the environment, the system may use rules that are based on reasonable expectations. For example, when an intersection is occluded in some way, the system may generate an unperceived object as a car traveling at a reasonable speed approaching the intersection from the occluded direction. As another example, if two cars are parked on the side of the street, the system may generate an unperceived object as a person between the two cars. The unperceived objects may be included in the analysis by the system in addition to the perceived objects (e.g., objects identified based on sensor data from sensors of the vehicle)”, paragraph [0054], “The system may then monitor the vehicle-occupied trajectory(ies) in view of the object-occupied trajectories to determine if an intersection or overlap occurs (in some examples, with a safety margin built in). Once it is determined that an intersection occurs, the system may implement a “safety force field” and seek to “repel” the vehicle from the object(s) by implementing the safety procedure or by implementing another set of controls that has been determined to have an equal or lesser associated likelihood and/or imminence of causing an actual collision between the vehicle and any of the objects”, paragraph [0056], “During implementation of the safety procedure or the other action(s), when it is determined that there is no longer an overlap or intersection between the vehicle-occupied trajectory(ies) and the object-occupied trajectories (e.g., the imminence and/or likelihood of a collision is reduced), the system may cease implementing the safety procedure or the other action(s), and give control back to a higher layer of the autonomous driving software stack (e.g., a planning layer and/or control layer) associated with controlling the vehicle according to normal driving protocols (e.g., obeying rules of the road, following the current directions, etc.). In some examples, the action(s) selected may depend on what the higher layer of the stack originally planned to implement (e.g., proposed controls representative of a planned trajectory), and the system may select the closest safe controls and/or safe trajectories to the original planned action(s) (e.g., using one or more metrics)”, and paragraph [0069], “The lane planner may use the lane graph (e.g., the lane graph from the path perceiver 112), object poses within the lane graph (e.g., according to the localization manager 120), and/or a target point and direction at the distance into the future from the route planner as inputs. The target point and direction may be mapped to the best matching drivable point and direction in the lane graph (e.g., based on GNSS and/or compass direction). A graph search algorithm may then be executed on the lane graph from a current edge in the lane graph to find the shortest path to the target point.” Determining safety procedures to avoid hazards corresponds to the preventative safety mode and casing operation and overriding control when overlap is detected corresponds to the safety stop mode.).
Deusch and Nister do not explicitly disclose, however, Fritz, in the same field of endeavor, teaches and verifying, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level (See at least paragraph [0347], “A total of three different methods were investigated for assessment of statistical independence. In contrast to the calculation of the autocorrelation coefficient (AKK), the successive difference scatter and the correlation test according to R. A. Fischer relate to hypothesis tests”, paragraph [0377], “If the assumptions relating to independence and normal distribution are severely infringed, then this leads, particularly in the case of correlated data, to an incorrect calculation of the control limits”, paragraph [0379], “the probability of occurrence of one of the described cases, being p=0.0027, should not be considered random, but should be considered to be a fault since, in the case of normally distributed data, 99.73% of the data considered will be within .+-.3s”, and paragraph [0549], “The test variable t.sub.fault isolation is calculated using the approach that every plausibility method produces a binary test result E.sub.bin, method. The method result E.sub.method is calculated by multiplying the binary method result E.sub.bin, method by the method separation sharpness p.sub.method (equation 1.2).” The system performs online plausibility determination using hypothesis tests to evaluate whether statistical assumptions remain valid under predefined significance thresholds, thereby determining whether assumed parameters deviate from observed data beyond acceptable limits.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister and Fritz such that the localization apparatus of Deusch is further configured to control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0069].), and to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].).
Regarding Claim 16, Deusch, Nister, and Fritz teach The method according to claim 1, as set forth in the obviousness rejection above. Deusch and Nister do not explicitly disclose, however, Fritz, in the same field of endeavor, teaches wherein the online plausibility determination further comprises reporting a deviation to further modules if the online plausibility determination is no longer plausible (See at least paragraph [0556], “The term 6(E.sub.GC+E.sub.LC) is justified by the fact that one channel must always be isolated as being faulty in the event of a fault message in the limit check or appliance check” and paragraph [0559], “In order to initiate at least one warning with the low setting, at least the following individual results must occur.” The system isolates a channel as being faulty and a fault message is generated when plausibility checks such as a limit check or appliance check indicate implausible conditions, thereby reporting a deviation to other system modules when the online plausibility determination is no longer plausible.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister and Fritz such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), and to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549], [0556], [0559].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].).
Regarding Claim 18, Deusch, Nister, and Fritz teach The map validation system according to claim 13, as set forth in the obviousness rejection above. Deusch and Nister do not explicitly disclose, however, Fritz, in the same field of endeavor, teaches wherein the online plausibility determination further comprises reporting a deviation to further modules if the online plausibility determination is no longer plausible (See at least paragraph [0556], “The term 6(E.sub.GC+E.sub.LC) is justified by the fact that one channel must always be isolated as being faulty in the event of a fault message in the limit check or appliance check” and paragraph [0559], “In order to initiate at least one warning with the low setting, at least the following individual results must occur.” The system isolates a channel as being faulty and a fault message is generated when plausibility checks such as a limit check or appliance check indicate implausible conditions, thereby reporting a deviation to other system modules when the online plausibility determination is no longer plausible.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister and Fritz such that the localization apparatus of Deusch is further configured to control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0069].), and to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549], [0556], [0559].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Deusch, ("Random Finite Set-Based Localization and SLAM for Highly Automated Vehicles”) in view of Nister (US 20190243371 A1), Fritz (US 20110307217 A1), and Van der Laan (US 20120271791 A1).
Regarding Claim 6, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Van der Laan, in the same field of endeavor, teaches wherein the existence probability is initialized with an initial value of 50% (See at least paragraph [0025], “Maximum likelihood estimation involves determining a parameter that describes the density estimator making the observed data most probable. Generally, given a set of data, the likelihood of a given density estimator or parameter is equal to or proportional to the probability that the data occurred (e.g., experimental results occurred) given the density estimator or parameter. Hence, the density estimator or parameter with the maximum likelihood represents that density estimator or parameter estimate that makes the observed data most probable, i.e., the density estimator or parameter (e.g., probability) associated with the highest probability of the observed data. For the purposes of the present discussion, a parameter may be any value, function, or a combination of one or more values and/or one or more functions” and paragraph [0027], “Suppose a coin is tossed 10 times (n=10), and 6 heads (h=6) and 4 tails are observed. This represents a dataset of 10 observations (n=10, h=6). The likelihood of obtaining 6 (h=6) heads when tossing a coin 10 times, and the coin is such that the probability on head is 50%, (p=0.5), is thus given by the formula (see equation 2).”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Van der Laan such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), and to initialize the existence probability with an initial value of 50%, as taught by Van der Laan (See paragraph [0027].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be increased probability distribution accuracy and overall data correctness, as taught by Van der Laan (See paragraph [0027].).
Claim(s) 8 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Deusch, ("Random Finite Set-Based Localization and SLAM for Highly Automated Vehicles”) in view of Nister (US 20190243371 A1), Fritz (US 20110307217 A1), and Hamperl (US 20210003420 A1).
Regarding Claim 8, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Hamperl, in the same field of endeavor, teaches further comprising: determining a visibility of the at least one map element, wherein the visibility of the at least one map element is determined using a field of view of the ambient sensor and an occlusion of the map element; and determining a detection probability using the visibility of the at least one map element (See at least paragraph [0101], “Furthermore, a capture unit 13 (for example a camera) is located in the center of the vehicle 4. The field of view of the capture unit 13 is symbolized by the two obliquely running dashed lines. The opening angle or the field of view can vary depending on the capture unit 13 used. Furthermore, a plurality of rectangles in the field of view of the capture unit 13 are illustrated. These define the plurality of three-dimensional segments”, paragraph [0106], “A vehicle 4 which moves along a route, an object 31 to be recognized and a concealment 21 are present in FIG. 6. In the first section a), the vehicle 4 can recognize the object 31 since the view is clear, but the probability of correct recognition is still low on account of the distance. For example, the visibility of the object 31 may be 100% and the probability of correct recognition may be 10%. In the second section b), the vehicle has already moved further on the route in the direction of the object 31. Furthermore, the object 31 is recognized and the probability of correct recognition is increased since the distance between the vehicle 4 and the object 31 becomes shorter. For example, the visibility of the object 31 may still be 100% and the probability of correct is 30%”, and paragraph [0107], “In section c), the vehicle 4 has advanced even further on the route, with the result that the object 31 is concealed by the concealment 21. The concealment 21 is between the object 31 and the vehicle 4. For example, the visibility of the object 31 is now 0% and the probability of correct recognition of the object 31 would be 75%.” The ambient sensor is the capture unit 13. The occlusion is the concealment 21 of the object.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Hamperl such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), and to determine the visibility and detection probability of the map element using a field of view of the ambient sensor and occlusion, as taught by Hamperl (See paragraphs [0101], [0106], [0107].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be increased navigation safety and object recognition, as taught by Hamperl (See paragraphs [0101], [0106], [0107].).
Regarding Claim 10, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Hamperl, in the same field of endeavor, teaches wherein the existence probability of the at least one map element having a detection probability below a predetermined threshold is not updated (See at least paragraph [0049], “The evaluation unit of the object recognition device would therefore recognize concealment of the object since the probability of correct recognition is below the predefined threshold value, and the confidence value of the object in the backend would not be adjusted. In other words, adjustment of the confidence value can be omitted if the object, its position or its content has not been recognized with a probability of correct recognition above a threshold value.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Hamperl such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), and to determine the existence probability of the map element has a detection probability below a threshold and is not updated, as taught by Hamperl (See paragraph [0049].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be increased navigation safety and object recognition, as taught by Hamperl (See paragraph [0049].).
Claim(s) 11 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Deusch, ("Random Finite Set-Based Localization and SLAM for Highly Automated Vehicles”) in view of Nister (US 20190243371 A1), Fritz (US 20110307217 A1), and Wood, ("SAFETY FIRST FOR AUTOMATED DRIVING").
Regarding Claim 11, Deusch, Nister, and Fritz teach The map validation method according to claim 1, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Wood, in the same field of endeavor, teaches further comprising verifying a validity of the existence probability (See at least section 2.2.2.4, “Generally, input checks that determine the plausibility of individual sensor data, fusing multiple weighted input sources, and accumulating sensor data are possible strategies. Hardware and software diversity for the implementation of functionalities with the highest required error robustness should be considered.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Wood such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), and to verify the validity of the existence probability, as taught by Wood (See section 2.2.2.4) with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be enhanced sensor monitoring and reducing error, as taught by Wood (See section 2.2.2.4).
Regarding Claim 12, Deusch, Nister, Fritz, and Wood teach The map validation method according to claim 11, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Wood, in the same field of endeavor, teaches further comprising comparing the sensor data from different sensors of the at least semi- autonomous robot are compared to each other for verifying the validity of the existence probability (See at least section 2.2.2.4, “Generally, input checks that determine the plausibility of individual sensor data, fusing multiple weighted input sources, and accumulating sensor data are possible strategies. Hardware and software diversity for the implementation of functionalities with the highest required error robustness should be considered. While individual sensors can provide information about their current detection capabilities and range, sensor fusion can add substantial value in determining the current horizon of full sensor cluster perception, which may help to monitor the actual sensor performance. Regarded as a cross-referencing mechanism, sensor fusion can enable the detection of individual sensor limitations that are not detectable by the individual sensor itself.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Wood such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), and to verify the validity of the existence probability, as taught by Wood (See section 2.2.2.4), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be enhanced sensor monitoring and reducing error, as taught by Wood (See section 2.2.2.4).
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Deusch, ("Random Finite Set-Based Localization and SLAM for Highly Automated Vehicles”) in view of Nister (US 20190243371 A1), Fritz (US 20110307217 A1), and Zhang (US 20200217666 A1).
Regarding Claim 15, Deusch, Nister, and Fritz teach The method according to claim 1, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Zhang, in the same field of endeavor, teaches further comprising: determining a control mode using the sensor data, the map data, the localization data, and the existence probability of the at least one map element; and controlling the at least semi-autonomous robot based on the determined control mode (See at least paragraph [0018], “a method comprises retrieving a map of a 3D geometry of an environment the map comprising a plurality of non-spatial attribute values each corresponding to one of a plurality of non-spatial attributes and indicative of a plurality of non-spatial sensor readings acquired throughout the environment, receiving a plurality of sensor readings from a device within the environment wherein each of the sensor readings corresponds to at least one of the non-spatial attributes, matching the plurality of received sensor readings to at least one location in the map to produce a determined sensor location, retrieving additional sensor data from the determined sensor location and comparing the additional sensor data to data derived from the map”, paragraph [0064], “The SLAM system can build and maintain a point cloud in real time as a user is moving through an environment, such as when walking, biking, driving, flying, and combinations thereof. A map is constructed in real time as the mapper progresses through an environment. The SLAM system can track thousands of features as points. As the mapper moves, the points are tracked to allow estimation of motion. Thus, the SLAM system operates in real time and without dependence on external location technologies, such as GPS”, paragraph [0067], “The modularized mapping system, disclosed below, is configured to process data from range, vision, and inertial sensors for motion estimation and mapping by using a multi-layer optimization structure. The modularized mapping system may achieve high accuracy, robustness, and low drift by incorporating features which may include: an ability to dynamically reconfigure the computational modules; an ability to fully or partially bypass failure modes in the computational modules, and combine the data from the remaining modules in a manner to handle sensor and/or sensor data degradation, thereby addressing environmentally induced data degradation and the aggressive motion of the mobile mapping system; and an ability to integrate the computational module cooperatively to provide real-time performance”, paragraph [0068], “Disclosed herein is a mapping system for online ego-motion estimation with data from a 3D laser, a camera, and an IMU. The estimated motion further registers laser points to build a map of the traversed environment. In many real-world applications, ego-motion estimation and mapping must be conducted in real-time. In an autonomous navigation system, the map may be crucial for motion planning and obstacle avoidance, while the motion estimation is important for vehicle control and maneuver,” and paragraph [0163], “In embodiments, a SLAM system may be integrated with various external systems, such as vehicle navigation systems (such as for unmanned aerial vehicles, drones, mobile robots, unmanned underwater vehicles, self-driving vehicles, semi-automatic vehicles, and many others). In embodiments, the SLAM system may be used to allow a vehicle to navigate within its environments, without reliance on external systems like GPS.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Zhang such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), and to determine a control mode using sensor data, map data, localization data, and the existence probability of a map element, and controlling the semi-autonomous robot based on the determined control mode, as taught by Zhang (See paragraphs [0064], [0067], [0068].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be increased navigation information and optimizing path finding, as taught by Zhang (See paragraphs [0064], [0067], [0068].).
Claim(s) 17 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Deusch, ("Random Finite Set-Based Localization and SLAM for Highly Automated Vehicles”) in view of Nister (US 20190243371 A1), Fritz (US 20110307217 A1), and Sadeghi (US 20220101549 A1).
Regarding Claim 17, Deusch, Nister, and Fritz teach The method according to claim 1, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Sadeghi, in the same field of endeavor, teaches wherein the online plausibility determination further comprises estimating relevant parameters of a measurement model that includes (i) a spatial uncertainty of sensor measurements, (ii) a sensor perturbation rate, and (iii) a detection probability, continuously during an operation (See at least paragraph [0030], [0031], “The one or more uncertainty estimation parameters may be one or more perturbation weights that at least partially define the perturbation function, wherein: initial value(s) are may be to the one or more perturbation weights”, paragraph [0032], “in each of multiple training iterations, the likelihood function may be initially sampled using value(s) of the weights determined in the previous iteration or, in the case of the first iteration, the initial value(s), the value(s) of the perturbation weights being determined in each iteration based on an objective function determined by the sampling of the likelihood function in that iteration, the iterations continuing until a termination condition is satisfied”, paragraph [0153], “Methods exist for determining the confidence in a stereo disparity estimate exist—see [17]. These provide a confidence measure, denoting the probability of a given pixel being correct. However, these provide confidence measures cannot be directly transformed into co-variance matrices, so cannot be used with Bayesian fusion methods. These methods can be adapted to create a covariance matrix, e.g. with the application of the Chebyshev inequality, but this requires resampling, which makes the method insufficiently fast”, and paragraph [0317], “This significantly reduces the amount of computation that needs to be performed by the uncertainty estimator 4 at inference, because it means the covariance of the generative noise model P(Ī.sub.d.sub.i|I.sub.d.sub.i) for pixel (j, k) is simply the covariance associated with the corresponding weight ƒ(w, I.sub.d.sub.i).sub.jk. In other words, the uncertainty associated with pixel (j, k) is simply the covariance associated with the weight corresponding to that pixel.” The system estimates parameters of a measurement model including spatial uncertainty, perturbation parameters, and probability of correctness, wherein such parameters are determined iteratively and continuously during system operation.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Sadeghi such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549].), and wherein the online plausibility determination further comprises estimating relevant parameters of a measurement model that includes (i) a spatial uncertainty of sensor measurements, (ii) a sensor perturbation rate, and (iii) a detection probability, continuously during an operation, as taught by Sadeghi (See paragraphs [0030], [0031], [0153], [0317].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be increased safety and appropriateness of driving decisions, as taught by Sadeghi (See paragraph [0005].).
Regarding Claim 19, Deusch, Nister, and Fritz teach The map validation system according to claim 13, as set forth in the obviousness rejection above. Deusch, Nister, and Fritz do not explicitly disclose, however, Sadeghi, in the same field of endeavor, teaches wherein the online plausibility determination further comprises estimating relevant parameters of a measurement model that includes (i) a spatial uncertainty of sensor measurements, (ii) a sensor perturbation rate, and (iii) a detection probability, continuously during an operation (See at least paragraph [0030], [0031], “The one or more uncertainty estimation parameters may be one or more perturbation weights that at least partially define the perturbation function, wherein: initial value(s) are may be to the one or more perturbation weights”, paragraph [0032], “in each of multiple training iterations, the likelihood function may be initially sampled using value(s) of the weights determined in the previous iteration or, in the case of the first iteration, the initial value(s), the value(s) of the perturbation weights being determined in each iteration based on an objective function determined by the sampling of the likelihood function in that iteration, the iterations continuing until a termination condition is satisfied”, paragraph [0153], “Methods exist for determining the confidence in a stereo disparity estimate exist—see [17]. These provide a confidence measure, denoting the probability of a given pixel being correct. However, these provide confidence measures cannot be directly transformed into co-variance matrices, so cannot be used with Bayesian fusion methods. These methods can be adapted to create a covariance matrix, e.g. with the application of the Chebyshev inequality, but this requires resampling, which makes the method insufficiently fast”, and paragraph [0317], “This significantly reduces the amount of computation that needs to be performed by the uncertainty estimator 4 at inference, because it means the covariance of the generative noise model P(Ī.sub.d.sub.i|I.sub.d.sub.i) for pixel (j, k) is simply the covariance associated with the corresponding weight ƒ(w, I.sub.d.sub.i).sub.jk. In other words, the uncertainty associated with pixel (j, k) is simply the covariance associated with the weight corresponding to that pixel.” The system estimates parameters of a measurement model including spatial uncertainty, perturbation parameters, and probability of correctness, wherein such parameters are determined iteratively and continuously during system operation.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing
date to combine the invention of Deusch with the teachings of Nister, Fritz, and Sadeghi such that the localization apparatus of Deusch is further configured to determine a data uncertainty, wherein the data uncertainty comprises a sensor data uncertainty, a map data uncertainty, and a localization data uncertainty, and control the at least semi-autonomous robot in three control modes comprising a normal driving mode, a preventative safety mode, and a safety stop mode based on the determined robot trajectory, as taught by Nister (See paragraphs [0053], [0054], [0056], [0059], [0068], [0069].), and to verify, with an online plausibility determination, a validity of statistical and algorithmic assumptions used for determining the existence probability of the at least one map element, wherein the online plausibility determination comprises performing a statistical hypothesis test to determine whether assumed parameters used for determining the existence probability deviate from online estimated parameters beyond a predefined significance level, as taught by Fritz (See paragraphs [0347], [0377], [0379], [0549], [0556], [0559].), and wherein the online plausibility determination further comprises estimating relevant parameters of a measurement model that includes (i) a spatial uncertainty of sensor measurements, (ii) a sensor perturbation rate, and (iii) a detection probability, continuously during an operation, as taught by Sadeghi (See paragraphs [0030], [0031], [0153], [0317].), with a reasonable expectation of success. The motivation for doing so would be implementing safety procedures to avoid collisions, as taught by Nister (See paragraph [0006].). The motivation for doing so would be ensuring consistency with stringent requirements of modern test panels and monitor multiple measurement channels at once with decreased effort, as taught by Fritz (See paragraph [0051].). The motivation for doing so would be increased safety and appropriateness of driving decisions, as taught by Sadeghi (See paragraph [0005].).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEWEL ASHLEY KUNTZ whose telephone number is (571)270-5542. The examiner can normally be reached M-F 8:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached at (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEWEL A KUNTZ/Examiner, Art Unit 3666
/ANNE MARIE ANTONUCCI/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights