DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in responsive to communication(s): original application filed on 12/21/2023, said application claims a priority filing date of 06/29/2021. Claims 1-20 are pending. Claims 1, 9, and 11 are independent.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: 111 in ¶ [53]; 190 in ¶ [57]; 194 in ¶ [57]; 192 in ¶ [57]; and 140 in ¶ [57]. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: 112 in FIG. 1 and 170 in FIG. 1. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference characters "200" in ¶¶ [59]-[62], [64]-[65], [67], [75], [78]-[80], [84], [94]-[95], [99], and [101] with FIG. 2 and "210" in ¶ [59] have both been used to designate "autonomous system(s)". Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “210” has been used to designate both "computer system" in ¶¶ [59], [62]-[65], [67], [69], [73], [84], [88], and [101] with FIG. 2 and "autonomous systems" in ¶ [59]. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The use of the term "Wi-Fi" in ¶ [56], which is a trade name or a mark used in commerce, has been noted in this application. The term should be accompanied by the generic terminology; furthermore the term should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Claim Objections
Claims 1-3, 6-9, 11, 13, 15, and 18-20 are objected to because of the following informalities:
in Claims 1, 3, 6, 8-9, 11, 15, 18, and 20, it is recommended to change "… if …" to "… when …" which provide more definite condition in scope'
in Claim 1, Page 27, lines 27-28, "… determining that the new dynamic environment property is incoherent if it conflicts with at least one of the static environment properties …" appears to be "… determining that the new dynamic environment property is incoherent when it conflicts with the at least one of the static environment properties …";
in Claim 2, Page 28, lines 4-5, "… wherein generating a new active objective from the candidate objectives based on the new event comprises …" appears to be "… wherein generating the new active objective from the candidate objectives based on the new event comprises …";
in Claim 6, Page 28, line 28, "… coherence checking is further executed on …" appears to be "… the coherence checking is further executed on …" (see also 112 Rejections to Claim 6);
in Claim 7, Page 29, line 4, "… coherence checking is further executed on …" appears to be "… the coherence checking is further executed on …" (see also 112 Rejections to Claim 7);
in Claim 9, Page 29, lines 26-28, "… the dynamic environment properties comprising third computer-readable instructions generated by the autonomous system based on the detected events …" appears to be "… the dynamic environment properties comprising third computer-readable instructions generated by the autonomous system based on the observed events …";
in Claim 9, Page 30, lines 11-12, "… determining that the new dynamic environment property is incoherent if it conflicts with at least one of the static environment properties …" appears to be "… determining that the new dynamic environment property is incoherent when it conflicts with the at least one of the static environment properties …";
in Claim 9, Page 30, line 13, "… in response to determining that the environment property is incoherent …" appears to be "… in response to determining that the new dynamic environment property is incoherent …";
in Claim 11, Page 31, lines 17-19, "… identifying the new dynamic environment property as incoherent if determination is made that the new dynamic environment property conflicts with at least one of the static environment properties" appears to be "… identifying the new dynamic environment property as incoherent when determination is made that the new dynamic environment property conflicts with the at least one of the static environment properties";
in Claim 13, Page 31, line 31 – Page 32, line 2, "… wherein the execution of coherence checking comprises: if determination is made that the new dynamic environment property is conflicting with at least one static environment properties of the first database, marking the new dynamic environment property as incoherent, a conflict between the new dynamic environment property and at least one static environment properties being caused by an opposition of their respective computer- readable instructions …" appears to be "… wherein the execution of the coherence checking comprises: when the determination is made that the new dynamic environment property is conflicting with the at least one of the static environment properties of the first database, marking the new dynamic environment property as incoherent, a conflict between the new dynamic environment property and the at least one of the static environment properties being caused by an opposition of their respective computer- readable instructions …";
in Claim 18, Page 32, line 29, "… coherence checking is further executed on …" appears to be "… the coherence checking is further executed on …" (see also 112 Rejections to Claim 18);
in Claim 19, Page 33, line 6, "… coherence checking is further executed on …" appears to be "… the coherence checking is further executed on …" (see also 112 Rejections to Claim 19).
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "… one or more sensing devices configured to observe events, the events describing characteristics of entities, the entities defining an environment in which the autonomous system is configured to operate … accessing a list of active objectives, each one of the active objectives comprising a set of parameters … accessing a first database populated with static environment properties, the static environment properties comprising second computer-readable instructions defining pre- determined properties of entities and relations between the entities; accessing a second database populated with dynamic environment properties, the dynamic environment properties comprising third computer-readable instructions generated by the processor based on events observed by the one or more sensing devices … entering the new active objective to the list of active objectives; and operating the autonomous system based on active objectives of the list of active objectives" in Page 27, line, which rendering the claim indefinite because (1) it.
Claims 2-8 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 3 recites the limitation "… a selection of a candidate objective from the candidate objectives of the third database is based on the activation condition of the candidate objective" in Page 28, lines 14-15, which rendering the claim indefinite because ".
Claim 4 recites the limitation "… defining general properties of the entities and general relations between the entities, each general property being property of a group of entities and each general relation between the entities being relation between groups of entities" in Page 28, lines 18-21, which rendering the claim indefinite because ".
Claims 5-6 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 6 recites the limitation "…" in Page 28, line 27 - Page 29, line, which rendering the claim indefinite because ".
Claim 7 recites the limitation "… upon entering " in Page 29, lines 3-7, which rendering the claim indefinite because ".
Claim 8 recites the limitation "… wherein " in Page 29, lines 9-13, which rendering the claim indefinite because ".
Claim 9 recites the limitation "... one or more sensing devices configured to observe events, the events describing characteristics of entities, the entities defining an environment in which the autonomous system is configured to operate … the static environment properties comprising second computer-readable instructions defining pre-determined properties of entities and relations between the entities ..." in lines 19-25, which rendering the claim indefinite because (1) it is unclear whether the 1st and 3rd instances of ".
Claim 9 recites the limitation "… a third database comprising candidate objectives … accessing the third database populated with candidate objectives, each one of the candidate objectives comprising … generating a new active objective from the candidate objectives based on the new event " in Page 29, line 2, which rendering the claim indefinite because (1) it is.
Claim 10 is rejected for fully incorporating the deficiency of their respective base claims.
Claim 10 recites the limitation "… defining general properties of the entities and general relations between the entities, each general property being property of a group of entities and each general relation between the entities being relation between groups of entities" in Page , which rendering the claim indefinite because ".
Claim 11 recites the limitation "A computer-implemented method for knowledge-based reasoning to establish a list of active objectives by an autonomous system, the method comprising: accessing a list of active objectives, each one of the active objectives comprising a set of parameters …" in Page 30, lines 29-32, which rendering the claim indefinite because (1).
Claims 12-20 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 14 recites the limitation "… wherein generating a new active objective from the candidate objectives based on the new event comprises: selecting a candidate objective from the candidate objectives of the third database … " in Page 32, lines 5-7, which rendering the claim indefinite because (1) .
Claims 15-18 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 15 recites the limitation "." in Page 32, line 15-16, which rendering the claim indefinite because ".
Claim 16 recites the limitation "… each common-sense rule comprising fifth computer-readable instructions defining general properties of the entities and general relations between the entities, each general property being property of a group of entities and each general relation between the entities being relation between groups of entities" in Page , which rendering the claim indefinite because ".
Claims 17-18 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 18 recites the limitation "…" in Page , which rendering the claim indefinite because .
Claim 19 recites the limitation "… upon entering " in Page , which rendering the claim indefinite because ".
Claim 20 recites the limitation "… wherein " in Page , which rendering the claim indefinite because ".
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Della Penna (US 2019/0220011 A1, pub. date: 07/18/2019), hereinafter Della Penna.
Independent Claim 1
Della Penna discloses a computer-implemented method for operating an autonomous system (Della Penna, ¶ [0017] with FIG. 1: autonomy controller 150 configured to determine trajectories for an autonomous vehicle 120 to facilitate driverless, collision-free navigation via a path of travel based on computed trajectories 122; ¶ [0055] with 402 in FIG. 4: an autonomy controller may compute vehicular drive parameters that may be applied to a vehicle controller to facilitate driverless transit of autonomous vehicle via a path of travel), the autonomous system comprising one or more sensing devices (Della Penna, ¶¶ [0026] and [0033] with 156 and 121 in FIG. 1: event recorder 156 may be configured to "data mine," thereby collecting data and information from a variety of sensors in sensor platform 121, as well as derived data generated by logic, algorithms, or processes of autonomy controller 150, such as localization data, perception data (e.g., object recognition and classification data), trajectory data, and physical vehicle data (e.g., steering angles, braking pressures, etc.); sensor platform 121 may include any number of sensors (not shown) with which to facilitate driverless control of autonomous vehicle 120; examples of sensors include one or more image capture devices (e.g., image sensors or cameras to capture video including high definition, or "HD," cameras), one or more radar devices (e.g., short-range radar, long-range radar, etc.), one or more LID AR devices, one or more sonar devices (or sensors configured to detect ultrasound), one or more global positioning system ("GPS") devices, one or more inertial measurement units ("IMU") devices, and one or more other types of sensors including, but not limited to, gyroscopes, accelerometers, odometry sensors, steering wheel angle sensors, tire angle sensors, throttle sensors, brake pressure sensors, proximity sensors (e.g., in or adjacent to a seat to determine whether occupied by a passenger), etc.; ¶ [0043] with 202-208 and 212-216 in FIG. 2: receive radar sensor data 202, lidar sensor data 204, image/video data 206, and other sensor data 208; receive ultrasound sensor data 212, inertial measurement unit ("IMU") data 214, and other sensor data 216 (e.g., GPS data, wheel or odometry data, gyroscopic data, etc.); ¶¶ [0051]-[0052] with 340-348 in FIG. 3: event recorder 356 is configured to receive data, such as sensor data 340 to 348 and data 370 to 372, each subset of which may be recorded and stored in an event storage repository 357; autonomy controller 350 may be configured to receive sensor data 340 to 348, which may include camera data 340, lidar data 341, radar data 342, GPS data 343, IMU data 344, sonar data 345, and other sensor data 348) configured to observe events, the events describing characteristics of entities, the entities defining an environment in which the autonomous system is configured to operate (Della Penna, ¶¶ [0017]-[0027] with 156 in FIG. 1: autonomy controller 150 may include an event recorder 156 that may be configured to receive data from multiple sources whether internal or external to autonomous vehicle 120, and further configured to identify an interval of time in which to store a subset of received data (e.g., event data) associated with an event in, for example, an event storage repository 157; an example of captured data from the multiple sources includes control data such as steering data, throttle data, braking data transmission shifting data, etc.; another example of captured data from the multiple sources includes computed vehicular drive parameters, such as a degree of wheel angle, an amount of throttle, an amount of brake pressure, a state of transmission, and other computed values of which may be applied to facilitate driverless transit; thus, vehicular drive parameters include parameter data representing steering data (e.g., degree(s) of wheel angle to effect a turn), acceleration data (e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., a state of a transmission subsystem to effect forward motion and reverse motion in one or more states of speed and torque), and the like; yet another example of captured data from multiple sources include derived data (e.g., metadata) calculated as a result of computations or processing other data to determine various states, responses, etc. to facilitate driverless operation, such as a list of determined objects in an environment (e.g., lamp posts, trees, bicycles, cars, signs, pedestrians, cyclists, dogs, fire hydrants, etc.), and a state of an "ESP" flag indicating whether an electronic stability program ("ESP") is activated to provide stability or traction control (e.g., responsive to skidding on ice); other examples of captured data from the multiple sources may include sensor data (e.g., lidar data, radar data, image data, GPS data, wheel or odometry data, accelerometer data, ambient or external air temperature, grade or angle of a roadway surface, etc.); event recorder 156 may be configured to capture steering wheel data, acceleration data, braking data, and the like; static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; autonomy controller 150 and/or event recorder 156 in autonomous vehicle 120 may be configured to monitor various streams of data regarding the performance and control of autonomous vehicle 120 from a variety of sources to detect an event; ¶¶ [0037] and [0045] with FIGS. 1-2: object recognizer 255 may be configured to implement object characterization and classification to identify types and attributes of objects (e.g., whether an object is dynamic or static, whether an object is animate, or living, rather than an inanimate object, etc.); vehicle controller 254 may detect and classify objects to generate an object list 230, which includes a list of objects, such as object ("1") 231, object ("2") 232, object ("3") 233, etc.; the objects may represent detect and/or classified objects detected by one or more sensors; e.g., objects 231, 232, and 233 may include static objects, such as a lamp post, and dynamic objects, such as a person walking; ¶ [0052] with FIG. 3: perception engine 355 may be configured to receive various subsets of data, and further configured to detect and classify objects, such as objects in object list 230 of FIG. 2, based on characteristics of an object (e.g., object characteristics); a classified object may trigger an indication of an event; a classified object may include ice, potholes, traffic cones, signs, etc.; any of which may be determined to trigger an event during which at least a subset of data is a recorded by event recorder 356; perception engine 355 may also be configured to predict locomotive behavior of external objects (e.g., predicting a tree is static or stationary, whereas a cyclist is dynamic and moves); perception engine 355 may transmit object data 370, which includes data describing one or more objects, to event recorder 356; ¶ [0056] with 404 in FIG. 4: data representing control signals originate in one or more control devices in an autonomous vehicle may be monitored; sensor data signals originating on one or more sensors internal or external to an autonomous vehicle may be monitored, whereby at least a subset of values representing sensor data signals may be stored in an event storage repository), and a processor (Della Penna, ¶ [0050]: one or more components of autonomy controller may be implemented as one or more processors, such as one or more graphics processing units ("GPUs"); ¶ [0086] with 804 in FIG. 8: processor 804) configured to execute the computer-implemented method (Della Penna, ¶ [0088] with FIG. 8: computing platform 800 performs specific operations by processor 804 executing one or more sequences of one or more instructions stored in system memory 806), the method comprising:
accessing a list of active objectives, each one of the active objectives comprising a set of parameters and first computer-readable instructions which upon being executed by the processor result in the autonomous system performing a task in accordance with the set of parameters (Della Penna, ¶ [0020]: whether a valid reason or goal was obtained (e.g., assuring safety as a goal); ¶ [0023]: navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; ¶ [0025]: the operation can be safely performed without impacting vehicle 119; ¶¶ [0038]-[0040] and [0042] with FIG. 1: vehicle controller 154 also may be configured to generate trajectories or paths of travel 122 in accordance with a planned route to guide the transiting of autonomous vehicle 120 via lanes 111 and 113 of roadway 126; for a trajectory or path of travel 122, vehicle controller 154 may determine in real-time (or substantially in real-time) a number of path segments constituting a path of travel along roadway 126; to transit along a segment, vehicle controller 154 may compute a number of vehicular drive parameters (i.e., computed vehicular drive parameters) that may be applied incrementally to mechanical drive components (e.g., at a rate of 30 sets of vehicular drive parameters for every second) to cause autonomous vehicle 120 to automatically drive along trajectory-based path segments over roadway 126; vehicle controller 154 may be configured to compute one or more drive parameters in real-time (or substantially in real-time) with which to apply to vehicle control unit 123, including driving control signals to effect propulsion, steering, braking, transmission shifting, lighting (e.g., emergency flashers), sound (e.g., automatic horn alerts, etc.), among other functions; vehicle controller 154 may be configured to calculate a variety of trajectories per unit time (e.g., per second), in real-time or substantially in real-time, that may be used to guide autonomous vehicle along a route from a point of origination to a point of destination, most of which may be calculated to facilitate driverless control; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; route planning (e.g., planning paths of travel relative to roadway 126; a planned route along various paths of travel; ¶ [0046] with FIG. 2: trajectory generator 258 may be configured to generate trajectories or paths of travel in accordance with a planned route to guide the transiting of an autonomous vehicle via a roadway from origination point "A" (not shown) to destination point "B," such as a destination; to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; ¶ [0053] with FIG. 3: decision-making computer 359 may be configured to determine and planned routes by generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.);
accessing a first database populated with static environment properties, the static environment properties comprising second computer-readable instructions defining pre- determined properties of entities and relations between the entities (Della Penna, ¶ [0022] with FIG. 1: static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; ¶¶ [0033] and [0036] with FIG. 1: at least a subset of the aforementioned sensors of sensor platform 121 may be used to localize autonomous vehicle 120 at reference point 127 relative to its environment and objects within the environment (e.g., relative to roadway markings, a lamp post, a tree, and the like), and relative to a position in a global coordinate system (e.g., using GPS coordinates); vehicle controller 154 may determine a pose (e.g., a position and/or orientation) localized at a reference point 127 of autonomous vehicle 120; reference point 127 may be identified relative to external objects and surfaces of an external environment (or scene), and may be correlated to a position on a roadway 126, which may be described in map data 151; vehicle controller 154 may be configured to determine a position of reference point 127 relative to monuments or markers that may be used as known locations or points in a coordinate system to confirm or facilitate localization of autonomous vehicle 120 relative to, for example, roadway 126; ¶¶ [0040] and [0042] with 151 in FIG. 1: map manager 152 may be configured to implement map data 151 to localize and navigate autonomous vehicle 120 relative to roadway 126 or any pathway or route, any of which may be represented as image data; map data 151 may include relatively high resolutions of images of roadway 126 and adjacent objects, such as communication tower 198 and the like; map data 151 may include static or semi-static objects that have a relatively low or negligible probability of moving positions; static objects may be used as monuments or markers; autonomy controller 150 may use map data 151 to identify external imagery to facilitate route planning; map data 151 may include image data representing lane markings as well as data representing lane widths and curbs (e.g., with curb markings, such as "loading zone," etc.); map data 151 may also include any type of map data, such as 2D map data, 3D map data, 4D map data (e.g., includes three-dimensional map data at a particular point in time), or the like; additionally, map data 151 may include route data, such as road network data, including, but not limited to, route network definition file ("RNDF") data (or similar data) and the like; map data 151 may include images in high resolutions that include granular details of an environment or scene in which an autonomous vehicle is driving to ensure relatively accurate and precise localization, object classification, navigation, path of travel generation (e.g., trajectory generation), etc., as well as ensuring accurate and precise customized orientation and positioning when self-parking a vehicle; portions of map data 151 associated with a planned route along various paths of travel may be downloaded (e.g., as adjacent blocks of grid-type HD map data) as an autonomous vehicle travels along the route, thereby preserving resources (e.g., relatively large amount of storage need not be required to store an entire HD map of a particular region, such as a country); ¶ [0044] with FIG. 2: localizer 253 can determine a pose (e.g., a local position and orientation) at any one of number of geographic locations; localizer 253 may use acquired sensor data, such as sensor data associated with lamp posts, trees, or surfaces of buildings (e.g., a garage), which can be compared against reference data, such as map data (e.g., 3D map data, including reflectance data) to determine a local pose; localizer 253 may determine a relative geographic location of an autonomous vehicle relative to, for example, a global coordinate system (e.g., latitude and longitudinal coordinates, etc.));
accessing a second database populated with dynamic environment properties, the dynamic environment properties comprising third computer-readable instructions generated by the processor based on events observed by the one or more sensing devices (Della Penna, ¶¶ [0022] and [0026]-[0032] with 156-157 in FIG.1: examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; also, dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; pattern data stored in event storage 157 may be used by event recorder 156 to determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel; detect via a sensor platform 121 that a vehicle 119 in lane 113 obstructs the new path of travel 199; ¶ [0041] with FIG. 1: map manager 152 may also be configured to generate a dynamic representation of map data 151 by fusing or combining static map data (e.g., image data representing visual characteristics of roadway 126 and static objects, such as road markings, tree 144, stop sign 146, etc.) and dynamic map data to form dynamic map data 151; dynamic map data may include data representing objects detected via image capture (and/or other sensor data, including lidar), whereby an object may have attributes indicative of dynamism, such as a pedestrian or a cyclist);
upon observing, by the one or more sensing devices, a new event in the environment: generating a new dynamic environment property based on the new event; entering the new dynamic environment property to the second database; executing coherence checking on the new dynamic environment property and the static environment properties, the coherence checking comprising comparing the new dynamic environment property with all the static environment properties to assess whether the new dynamic environment property conflicts with at least one of the static environment properties, a conflict being representative of a logical incompatibility between the new dynamic environment property and the at least one of the static environment properties; determining that the new dynamic environment property is incoherent if it conflicts with at least one of the static environment properties (Della Penna, ¶¶ [0017] and [0020]-[0032] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120); autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); identify one or more anomalies associated with an event; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c).; note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶¶ [0077]-[0080] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event); and in response to determining that the new dynamic environment property is incoherent: accessing a third database populated with candidate objectives; generating a new active objective from one of the candidate objectives and based on the new event, the new active objective comprising information about tasks to be performed by the autonomous system; entering the new active objective to the list of active objectives; and operating the autonomous system based on active objectives of the list of active objectives (Della Penna, ¶¶ [0023]-[0035] and [0039] with FIG. 1: an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; generate control data 372 that may be transmitted to vehicle components 366 to effect autonomous driving; ¶¶ [0059]-[0063] with 406-412 in FIG. 4: when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; ¶¶ [0064]-[0074] with FIG. 5: as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0081]-[0082] with FIG. 7: to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 2
Della Penna discloses all the elements as stated in Claim 1 and further discloses wherein generating a new active objective from the candidate objectives based on the new event comprises: selecting a candidate objective from the candidate objectives of the third database; generating new active objective parameters based on the new event; and associating the new active objective parameters to the selected candidate objective (Della Penna, ¶¶ [0023]-[0035] and [0039] with FIG. 1: an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; generate control data 372 that may be transmitted to vehicle components 366 to effect autonomous driving; ¶¶ [0059]-[0063] with 406-412 in FIG. 4: when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; ¶¶ [0064]-[0074] with FIG. 5: as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0081]-[0082] with FIG. 7: to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 3
Della Penna discloses all the elements as stated in Claim 2 and further discloses wherein each candidate objective of the third database comprises an activation condition, the activation condition corresponding to one or more dynamic environment properties and the activation condition being fulfilled if determination is made that the corresponding one or more dynamic environment properties are found in the second database, and a selection of a candidate objective from the candidate objectives of the third database is based on the activation condition of the candidate objective (Della Penna, ¶¶ [0017], [0020]-[0035], and [0039] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119, and initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; thus, path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation and to vehicle components 366 to effect autonomous driving; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; as such, computed tire angles (and associated throttle or brake amounts) may be stored in an event storage repository for subsequent transmission as a function of transmission criteria; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; identify one or more anomalies associated with an event, and to determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c); note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0077]-[0082] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event; to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 4
Della Penna discloses all the elements as stated in Claim 1 further discloses wherein the first database further comprises common-sense rules, each common-sense rule comprising fifth computer-readable instructions defining general properties of the entities and general relations between the entities, each general property being property of a group of entities and each general relation between the entities being relation between groups of entities (Della, ¶¶ [0019]-[0020], [0022], [0024]-[0025], [0029], [0060], [0064]-[0067] with FIG.1: derived data may include computed vehicular drive parameters, may represent a course of action based on one or more rules (e.g., maintaining a lane offset due to detection of cones demarcating a construction zone); autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian (e.g., if the operation can be safely performed without impacting vehicle 119); modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; during "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input (e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, e.g., permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; in another example, illegally parked vehicles or trees may occlude sensing ( e.g., imagery, lidar, radar, etc.) of nearby pedestrians or traffic signs when an autonomous vehicle approaches; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid).
Claim 5
Della Penna discloses all the elements as stated in Claim 4 and further discloses wherein the common-sense rules are populated by an operator of the autonomous system and describe a cultural context of the environment in which the autonomous system is configured to operate (Della, ¶¶ [0019]-[0020], [0022], [0024]-[0025], [0029], [0060], [0064]-[0067] with FIG.1: derived data may include computed vehicular drive parameters, may represent a course of action based on one or more rules (e.g., maintaining a lane offset due to detection of cones demarcating a construction zone); autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian (e.g., if the operation can be safely performed without impacting vehicle 119); modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; during "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input (e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, e.g., permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; in another example, illegally parked vehicles or trees may occlude sensing ( e.g., imagery, lidar, radar, etc.) of nearby pedestrians or traffic signs when an autonomous vehicle approaches; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid).
Claim 6
Della Penna discloses all the elements as stated in Claim 4 and further discloses wherein, upon entering a new dynamic environment property based on a new event in the second database, coherence checking is further executed on the new dynamic environment property and the common-sense rules, and if determination is made that the new dynamic environment property conflicts with at least one of the common-sense rules: accessing the third database populated with candidate objectives; generating a new active objective from the candidate objectives based on the new event; and entering the new active objective to the list of active objectives (Della Penna, ¶¶ [0017], [0020]-[0035], and [0039] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119, and initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; thus, path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation and to vehicle components 366 to effect autonomous driving; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; as such, computed tire angles (and associated throttle or brake amounts) may be stored in an event storage repository for subsequent transmission as a function of transmission criteria; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; identify one or more anomalies associated with an event, and to determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c); note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0077]-[0082] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event; to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 7
Della Penna discloses all the elements as stated in Claim 1 and further discloses wherein upon entering a new dynamic environment property in the second database, coherence checking is further executed on the new dynamic environment property and the list of active objectives, and, if determination is made that the new dynamic environment property conflicts with at least one of the active objectives, removing the at least one of the active objectives from the list of active objectives (Della Penna, ¶¶ [0017], [0020]-[0035], and [0039] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119, and initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; thus, path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶ [0041]: dynamic map data may include temporally-static objects (e.g., semi-static objects), which may be temporally static for a certain duration of time ( e.g., during construction or times of day) and may be added or removed dynamically from a mapped environment; ¶¶ [0045]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation and to vehicle components 366 to effect autonomous driving; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; as such, computed tire angles (and associated throttle or brake amounts) may be stored in an event storage repository for subsequent transmission as a function of transmission criteria; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; identify one or more anomalies associated with an event, and to determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c); note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0077]-[0082] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event; to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 8
Della Penna discloses all the elements as stated in Claim 1 and further discloses wherein one or more dynamic environment properties describing meta-events are generated if determination is made that predefined combinations of dynamic environment properties are found in the second database, each predefined combination of dynamic environment properties corresponding to a meta-event and causing a generation of a corresponding dynamic environment property in the second database (Della Penna, ¶¶ [0017]-[0027] with 156 in FIG. 1: autonomy controller 150 may include an event recorder 156 that may be configured to receive data from multiple sources whether internal or external to autonomous vehicle 120, and further configured to identify an interval of time in which to store a subset of received data (e.g., event data) associated with an event in, for example, an event storage repository 157; an example of captured data from the multiple sources includes control data such as steering data, throttle data, braking data transmission shifting data, etc.; another example of captured data from the multiple sources includes computed vehicular drive parameters, such as a degree of wheel angle, an amount of throttle, an amount of brake pressure, a state of transmission, and other computed values of which may be applied to facilitate driverless transit; thus, vehicular drive parameters include parameter data representing steering data (e.g., degree(s) of wheel angle to effect a turn), acceleration data (e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., a state of a transmission subsystem to effect forward motion and reverse motion in one or more states of speed and torque), and the like; yet another example of captured data from multiple sources include derived data (e.g., metadata) calculated as a result of computations or processing other data to determine various states, responses, etc. to facilitate driverless operation, such as a list of determined objects in an environment (e.g., lamp posts, trees, bicycles, cars, signs, pedestrians, cyclists, dogs, fire hydrants, etc.), and a state of an "ESP" flag indicating whether an electronic stability program ("ESP") is activated to provide stability or traction control (e.g., responsive to skidding on ice); other examples of captured data from the multiple sources may include sensor data (e.g., lidar data, radar data, image data, GPS data, wheel or odometry data, accelerometer data, ambient or external air temperature, grade or angle of a roadway surface, etc.); event recorder 156 may be configured to capture steering wheel data, acceleration data, braking data, and the like; static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; autonomy controller 150 and/or event recorder 156 in autonomous vehicle 120 may be configured to monitor various streams of data regarding the performance and control of autonomous vehicle 120 from a variety of sources to detect an event; ¶¶ [0037] and [0045] with FIGS. 1-2: object recognizer 255 may be configured to implement object characterization and classification to identify types and attributes of objects (e.g., whether an object is dynamic or static, whether an object is animate, or living, rather than an inanimate object, etc.); vehicle controller 254 may detect and classify objects to generate an object list 230, which includes a list of objects, such as object ("1") 231, object ("2") 232, object ("3") 233, etc.; the objects may represent detect and/or classified objects detected by one or more sensors; e.g., objects 231, 232, and 233 may include static objects, such as a lamp post, and dynamic objects, such as a person walking; objects of object list 230 may be described as being represented by computed data or "metadata" that may be used to identify an event as well as responsive action, such as corrective action (e.g., updated software or logic); ¶ [0052] with FIG. 3: perception engine 355 may be configured to receive various subsets of data, and further configured to detect and classify objects, such as objects in object list 230 of FIG. 2, based on characteristics of an object (e.g., object characteristics); a classified object may trigger an indication of an event; a classified object may include ice, potholes, traffic cones, signs, etc.; any of which may be determined to trigger an event during which at least a subset of data is a recorded by event recorder 356; perception engine 355 may also be configured to predict locomotive behavior of external objects (e.g., predicting a tree is static or stationary, whereas a cyclist is dynamic and moves); perception engine 355 may transmit object data 370, which includes data describing one or more objects, to event recorder 356; ¶ [0056] with 404 in FIG. 4: data representing control signals originate in one or more control devices in an autonomous vehicle may be monitored; sensor data signals originating on one or more sensors internal or external to an autonomous vehicle may be monitored, whereby at least a subset of values representing sensor data signals may be stored in an event storage repository; ¶¶ [0022] and [0026]-[0032] with 156-157 in FIG.1: examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; also, dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; pattern data stored in event storage 157 may be used by event recorder 156 to determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel; detect via a sensor platform 121 that a vehicle 119 in lane 113 obstructs the new path of travel 199; ¶ [0041] with FIG. 1: map manager 152 may also be configured to generate a dynamic representation of map data 151 by fusing or combining static map data (e.g., image data representing visual characteristics of roadway 126 and static objects, such as road markings, tree 144, stop sign 146, etc.) and dynamic map data to form dynamic map data 151; dynamic map data may include data representing objects detected via image capture (and/or other sensor data, including lidar), whereby an object may have attributes indicative of dynamism, such as a pedestrian or a cyclist; ¶¶ [0017] and [0020]-[0032] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120); autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); identify one or more anomalies associated with an event; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c).; note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶¶ [0077]-[0080] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event).
Independent Claim 9
Della Penna an autonomous system configured to manage a list of active objectives, each one of the active objectives comprising a set of parameters and first computer-readable instructions which upon being executed by the autonomous system results in the autonomous system performing a task in accordance with the set of parameters (Della Penna, ¶ [0017], [0020], [0023], [0025], [0028], and [0034] with FIG. 1: autonomy controller 150 configured to determine trajectories for an autonomous vehicle 120 to facilitate driverless, collision-free navigation via a path of travel based on computed trajectories 122; whether a valid reason or goal was obtained (e.g., assuring safety as a goal); navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; the operation can be safely performed without impacting vehicle 119; controlling vehicle to transit lane 111 of roadway 126 based on one or more trajectories 122, as determined by control signals originating from control devices (e.g., steering wheel, a throttle pedal, a brake pedal, a transmission shifter, etc.; receive data representing steering data (e.g., degree of wheel angle to effect a turn), acceleration data ( e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., representing a selected gear and/or a direction), and the like; apply changes to at least steering, acceleration and deceleration at a rate of thirty (30) times a second or greater; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; ¶¶ [0038]-[0040] and [0042] with FIG. 1: vehicle controller 154 also may be configured to generate trajectories or paths of travel 122 in accordance with a planned route to guide the transiting of autonomous vehicle 120 via lanes 111 and 113 of roadway 126; for a trajectory or path of travel 122, vehicle controller 154 may determine in real-time (or substantially in real-time) a number of path segments constituting a path of travel along roadway 126; to transit along a segment, vehicle controller 154 may compute a number of vehicular drive parameters (i.e., computed vehicular drive parameters) that may be applied incrementally to mechanical drive components (e.g., at a rate of 30 sets of vehicular drive parameters for every second) to cause autonomous vehicle 120 to automatically drive along trajectory-based path segments over roadway 126; vehicle controller 154 may be configured to compute one or more drive parameters in real-time (or substantially in real-time) with which to apply to vehicle control unit 123, including driving control signals to effect propulsion, steering, braking, transmission shifting, lighting (e.g., emergency flashers), sound (e.g., automatic horn alerts, etc.), among other functions; vehicle controller 154 may be configured to calculate a variety of trajectories per unit time (e.g., per second), in real-time or substantially in real-time, that may be used to guide autonomous vehicle along a route from a point of origination to a point of destination, most of which may be calculated to facilitate driverless control; e.g., vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; route planning (e.g., planning paths of travel relative to roadway 126; a planned route along various paths of travel; ¶ [0046] with FIG. 2: trajectory generator 258 may be configured to generate trajectories or paths of travel in accordance with a planned route to guide the transiting of an autonomous vehicle via a roadway from origination point "A" (not shown) to destination point "B," such as a destination; to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; ¶ [0053] with FIG. 3: decision-making computer 359 may be configured to determine and planned routes by generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.)),
the system comprising: one or more sensing devices (Della Penna, ¶¶ [0026] and [0033] with 156 and 121 in FIG. 1: event recorder 156 may be configured to "data mine," thereby collecting data and information from a variety of sensors in sensor platform 121, as well as derived data generated by logic, algorithms, or processes of autonomy controller 150, such as localization data, perception data (e.g., object recognition and classification data), trajectory data, and physical vehicle data (e.g., steering angles, braking pressures, etc.); sensor platform 121 may include any number of sensors (not shown) with which to facilitate driverless control of autonomous vehicle 120; examples of sensors include one or more image capture devices (e.g., image sensors or cameras to capture video including high definition, or "HD," cameras), one or more radar devices (e.g., short-range radar, long-range radar, etc.), one or more LID AR devices, one or more sonar devices (or sensors configured to detect ultrasound), one or more global positioning system ("GPS") devices, one or more inertial measurement units ("IMU") devices, and one or more other types of sensors including, but not limited to, gyroscopes, accelerometers, odometry sensors, steering wheel angle sensors, tire angle sensors, throttle sensors, brake pressure sensors, proximity sensors (e.g., in or adjacent to a seat to determine whether occupied by a passenger), etc.; ¶ [0043] with 202-208 and 212-216 in FIG. 2: receive radar sensor data 202, lidar sensor data 204, image/video data 206, and other sensor data 208; receive ultrasound sensor data 212, inertial measurement unit ("IMU") data 214, and other sensor data 216 (e.g., GPS data, wheel or odometry data, gyroscopic data, etc.); ¶¶ [0051]-[0052] with 340-348 in FIG. 3: event recorder 356 is configured to receive data, such as sensor data 340 to 348 and data 370 to 372, each subset of which may be recorded and stored in an event storage repository 357; autonomy controller 350 may be configured to receive sensor data 340 to 348, which may include camera data 340, lidar data 341, radar data 342, GPS data 343, IMU data 344, sonar data 345, and other sensor data 348) configured to observe events, the events describing characteristics of entities, the entities defining an environment in which the autonomous system is configured to operate (Della Penna, ¶¶ [0017]-[0027] with 156 in FIG. 1: autonomy controller 150 may include an event recorder 156 that may be configured to receive data from multiple sources whether internal or external to autonomous vehicle 120, and further configured to identify an interval of time in which to store a subset of received data (e.g., event data) associated with an event in, for example, an event storage repository 157; an example of captured data from the multiple sources includes control data such as steering data, throttle data, braking data transmission shifting data, etc.; another example of captured data from the multiple sources includes computed vehicular drive parameters, such as a degree of wheel angle, an amount of throttle, an amount of brake pressure, a state of transmission, and other computed values of which may be applied to facilitate driverless transit; thus, vehicular drive parameters include parameter data representing steering data (e.g., degree(s) of wheel angle to effect a turn), acceleration data (e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., a state of a transmission subsystem to effect forward motion and reverse motion in one or more states of speed and torque), and the like; yet another example of captured data from multiple sources include derived data (e.g., metadata) calculated as a result of computations or processing other data to determine various states, responses, etc. to facilitate driverless operation, such as a list of determined objects in an environment (e.g., lamp posts, trees, bicycles, cars, signs, pedestrians, cyclists, dogs, fire hydrants, etc.), and a state of an "ESP" flag indicating whether an electronic stability program ("ESP") is activated to provide stability or traction control (e.g., responsive to skidding on ice); other examples of captured data from the multiple sources may include sensor data (e.g., lidar data, radar data, image data, GPS data, wheel or odometry data, accelerometer data, ambient or external air temperature, grade or angle of a roadway surface, etc.); event recorder 156 may be configured to capture steering wheel data, acceleration data, braking data, and the like; static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; autonomy controller 150 and/or event recorder 156 in autonomous vehicle 120 may be configured to monitor various streams of data regarding the performance and control of autonomous vehicle 120 from a variety of sources to detect an event; ¶¶ [0037] and [0045] with FIGS. 1-2: object recognizer 255 may be configured to implement object characterization and classification to identify types and attributes of objects (e.g., whether an object is dynamic or static, whether an object is animate, or living, rather than an inanimate object, etc.); vehicle controller 254 may detect and classify objects to generate an object list 230, which includes a list of objects, such as object ("1") 231, object ("2") 232, object ("3") 233, etc.; the objects may represent detect and/or classified objects detected by one or more sensors; e.g., objects 231, 232, and 233 may include static objects, such as a lamp post, and dynamic objects, such as a person walking; ¶ [0052] with FIG. 3: perception engine 355 may be configured to receive various subsets of data, and further configured to detect and classify objects, such as objects in object list 230 of FIG. 2, based on characteristics of an object (e.g., object characteristics); a classified object may trigger an indication of an event; a classified object may include ice, potholes, traffic cones, signs, etc.; any of which may be determined to trigger an event during which at least a subset of data is a recorded by event recorder 356; perception engine 355 may also be configured to predict locomotive behavior of external objects (e.g., predicting a tree is static or stationary, whereas a cyclist is dynamic and moves); perception engine 355 may transmit object data 370, which includes data describing one or more objects, to event recorder 356; ¶ [0056] with 404 in FIG. 4: data representing control signals originate in one or more control devices in an autonomous vehicle may be monitored; sensor data signals originating on one or more sensors internal or external to an autonomous vehicle may be monitored, whereby at least a subset of values representing sensor data signals may be stored in an event storage repository);
a memory (Della Penna, ¶ [0067]: local autonomous vehicle memory; ¶¶ [0086] and [0088]-[0089] with 806 and 808 in FIG. 8: system memory 806 (e.g., RAM, etc.), storage device 808 (e.g., ROM, etc.), an in-memory cache (which may be implemented in RAM 806 or other portions of computing platform 800); ) comprising: a first database populated with static environment properties, the static environment properties comprising second computer-readable instructions defining pre-determined properties of entities and relations between the entities (Della Penna, ¶ [0022] with FIG. 1: static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; ¶¶ [0033] and [0036] with FIG. 1: at least a subset of the aforementioned sensors of sensor platform 121 may be used to localize autonomous vehicle 120 at reference point 127 relative to its environment and objects within the environment (e.g., relative to roadway markings, a lamp post, a tree, and the like), and relative to a position in a global coordinate system (e.g., using GPS coordinates); vehicle controller 154 may determine a pose (e.g., a position and/or orientation) localized at a reference point 127 of autonomous vehicle 120; reference point 127 may be identified relative to external objects and surfaces of an external environment (or scene), and may be correlated to a position on a roadway 126, which may be described in map data 151; vehicle controller 154 may be configured to determine a position of reference point 127 relative to monuments or markers that may be used as known locations or points in a coordinate system to confirm or facilitate localization of autonomous vehicle 120 relative to, for example, roadway 126; ¶¶ [0040] and [0042] with 151 in FIG. 1: map manager 152 may be configured to implement map data 151 to localize and navigate autonomous vehicle 120 relative to roadway 126 or any pathway or route, any of which may be represented as image data; map data 151 may include relatively high resolutions of images of roadway 126 and adjacent objects, such as communication tower 198 and the like; map data 151 may include static or semi-static objects that have a relatively low or negligible probability of moving positions; static objects may be used as monuments or markers; autonomy controller 150 may use map data 151 to identify external imagery to facilitate route planning; map data 151 may include image data representing lane markings as well as data representing lane widths and curbs (e.g., with curb markings, such as "loading zone," etc.); map data 151 may also include any type of map data, such as 2D map data, 3D map data, 4D map data (e.g., includes three-dimensional map data at a particular point in time), or the like; additionally, map data 151 may include route data, such as road network data, including, but not limited to, route network definition file ("RNDF") data (or similar data) and the like; map data 151 may include images in high resolutions that include granular details of an environment or scene in which an autonomous vehicle is driving to ensure relatively accurate and precise localization, object classification, navigation, path of travel generation (e.g., trajectory generation), etc., as well as ensuring accurate and precise customized orientation and positioning when self-parking a vehicle; portions of map data 151 associated with a planned route along various paths of travel may be downloaded (e.g., as adjacent blocks of grid-type HD map data) as an autonomous vehicle travels along the route, thereby preserving resources (e.g., relatively large amount of storage need not be required to store an entire HD map of a particular region, such as a country); ¶ [0044] with FIG. 2: localizer 253 can determine a pose (e.g., a local position and orientation) at any one of number of geographic locations; localizer 253 may use acquired sensor data, such as sensor data associated with lamp posts, trees, or surfaces of buildings (e.g., a garage), which can be compared against reference data, such as map data (e.g., 3D map data, including reflectance data) to determine a local pose; localizer 253 may determine a relative geographic location of an autonomous vehicle relative to, for example, a global coordinate system (e.g., latitude and longitudinal coordinates, etc.)); a second database populated with dynamic environment properties, the dynamic environment properties comprising third computer-readable instructions generated by the autonomous system based on the detected events (Della Penna, ¶¶ [0022] and [0026]-[0032] with 156-157 in FIG.1: examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; also, dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; pattern data stored in event storage 157 may be used by event recorder 156 to determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel; detect via a sensor platform 121 that a vehicle 119 in lane 113 obstructs the new path of travel 199; ¶ [0041] with FIG. 1: map manager 152 may also be configured to generate a dynamic representation of map data 151 by fusing or combining static map data (e.g., image data representing visual characteristics of roadway 126 and static objects, such as road markings, tree 144, stop sign 146, etc.) and dynamic map data to form dynamic map data 151; dynamic map data may include data representing objects detected via image capture (and/or other sensor data, including lidar), whereby an object may have attributes indicative of dynamism, such as a pedestrian or a cyclist)); and a third database comprising candidate objectives (Della Penna, ¶ [0017], [0020], [0023], [0025], [0028], [0034], [0038]-[0039], and [0042], with FIG. 1: autonomy controller 150 configured to determine trajectories for an autonomous vehicle 120 to facilitate driverless, collision-free navigation via a path of travel based on computed trajectories 122; whether a valid reason or goal was obtained (e.g., assuring safety as a goal); navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; the operation can be safely performed without impacting vehicle 119; controlling vehicle to transit lane 111 of roadway 126 based on one or more trajectories 122, as determined by control signals originating from control devices (e.g., steering wheel, a throttle pedal, a brake pedal, a transmission shifter, etc.; receive data representing steering data (e.g., degree of wheel angle to effect a turn), acceleration data ( e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., representing a selected gear and/or a direction), and the like; apply changes to at least steering, acceleration and deceleration at a rate of thirty (30) times a second or greater; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; vehicle controller 154 also may be configured to generate trajectories or paths of travel 122 in accordance with a planned route to guide the transiting of autonomous vehicle 120 via lanes 111 and 113 of roadway 126; for a trajectory or path of travel 122, vehicle controller 154 may determine in real-time (or substantially in real-time) a number of path segments constituting a path of travel along roadway 126; to transit along a segment, vehicle controller 154 may compute a number of vehicular drive parameters (i.e., computed vehicular drive parameters) that may be applied incrementally to mechanical drive components (e.g., at a rate of 30 sets of vehicular drive parameters for every second) to cause autonomous vehicle 120 to automatically drive along trajectory-based path segments over roadway 126; vehicle controller 154 may be configured to calculate a variety of trajectories per unit time (e.g., per second), in real-time or substantially in real-time, that may be used to guide autonomous vehicle along a route from a point of origination to a point of destination, most of which may be calculated to facilitate driverless control; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; map data 151 may include images in high resolutions that include granular details of an environment or scene in which an autonomous vehicle is driving to ensure relatively accurate and precise localization, object classification, navigation, path of travel generation (e.g., trajectory generation), etc., as well as ensuring accurate and precise customized orientation and positioning when self-parking a vehicle; ¶¶ [0046] and [0053] with FIGS. 2-3: trajectory generator 258 may be configured to generate trajectories or paths of travel in accordance with a planned route to guide the transiting of an autonomous vehicle via a roadway from origination point "A" (not shown) to destination point "B," such as a destination; to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; hence, trajectory generator 258 may be configured to compute one or more vehicular drive parameters in real-time (or substantially in real-time) with which to apply to event recorder 256 or vehicle control unit 123, including driving control signals to effect propulsion, steering, braking, transmission shifting, lighting (e.g., emergency flashers), sound (e.g., automatic horn alerts, etc.), among other functions; decision-making computer 359 may be configured to determine and planned routes by generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; decision-making computer 359 may include one or more functionalities associated with trajectory generator 258 of FIG. 2; ¶¶ [0061] and [0064]: an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object ( e.g., cyclist), thereby possibly causing a collision; an autonomy controller may be further configured to predict the path of displacement of a cyclist ( e.g., a dynamic object) to predict a probability that a trajectory of an autonomous vehicle may intersect the path of displacement; human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules); and
a processor (Della Penna, ¶ [0050]: one or more components of autonomy controller may be implemented as one or more processors, such as one or more graphics processing units ("GPUs"); ¶ [0086] with 804 in FIG. 8: processor 804) operably coupled to the memory and the one or more sensing devices, and configured to execute instructions that, when executed, results in operations (Della Penna, ¶¶ [0085]-[0088] with FIG. 8: computing platform 800 or any portion (e.g., any structural or functional portion) can be disposed in any device, such as a computing device 890a, autonomous vehicle 890b, and/or a processing circuit in forming structures and/or functions of an autonomy controller 820a; computing platform 800 includes a bus 802 or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor 804, system memory 806 (e.g., RAM, etc.), storage device 808 (e.g., ROM, etc.), an in-memory cache (which may be implemented in RAM 806 or other portions of computing platform 800), a communication interface 813 (e.g., an Ethernet or wireless controller, a Bluetooth controller, NFC logic, etc.) to facilitate communications via a port on communication link 821 to communicate, for example, with a computing device; computing platform 800 exchanges data representing inputs and outputs via input-and-output devices 801; computing platform 800 performs specific operations by processor 804 executing one or more sequences of one or more instructions stored in system memory 806; ¶ [0033] with FIG. 1: autonomous vehicle 120 is shown to include a sensor platform 121, a vehicle control unit 123, and an autonomy controller 150) comprising:
upon observing, by the one or more sensing devices, a new event in the environment: generating a new dynamic environment property based on the new event; entering the new dynamic environment property to the second database; executing coherence checking on the new dynamic environment property and the static environment properties, the coherence checking comprising comparing the new dynamic environment property with all the static environment properties to assess whether the new dynamic environment property conflicts with at least one of the static environment properties, a conflict being representative of a logical incompatibility between the new dynamic environment property and the at least one of the static environment properties; and determining that the new dynamic environment property is incoherent if it conflicts with at least one of the static environment properties (Della Penna, ¶¶ [0017] and [0020]-[0032] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120); autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); identify one or more anomalies associated with an event; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c).; note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶¶ [0077]-[0080] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event); and in response to determining that the environment property is incoherent: accessing the third database populated with candidate objectives, each one of the candidate objectives comprising fifth computer- readable instructions which upon being executed by the autonomous system result in generating the first computer- readable instructions; generating a new active objective from the candidate objectives based on the new event; and entering the new active objective to the list of active objectives (Della Penna, ¶¶ [0023]-[0035] and [0039] with FIG. 1: an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; generate control data 372 that may be transmitted to vehicle components 366 to effect autonomous driving; ¶¶ [0059]-[0063] with 406-412 in FIG. 4: when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; ¶¶ [0064]-[0074] with FIG. 5: as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0081]-[0082] with FIG. 7: to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 10
Della Penna discloses all the elements as stated in Claim 9 and further discloses wherein the first database further comprises common-sense rules, each common-sense rule comprising fourth computer-readable instructions defining general properties of the entities and general relations between the entities, each general property being property of a group of entities and each general relation between the entities being relation between groups of entities (Della, ¶¶ [0019]-[0020], [0022], [0024]-[0025], [0029], [0060], [0064]-[0067] with FIG.1: derived data may include computed vehicular drive parameters, may represent a course of action based on one or more rules (e.g., maintaining a lane offset due to detection of cones demarcating a construction zone); autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian (e.g., if the operation can be safely performed without impacting vehicle 119); modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; during "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input (e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, e.g., permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; in another example, illegally parked vehicles or trees may occlude sensing ( e.g., imagery, lidar, radar, etc.) of nearby pedestrians or traffic signs when an autonomous vehicle approaches; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid).
Independent Claim 11
Della Penna discloses a computer-implemented method for knowledge-based reasoning to establish a list of active objectives by an autonomous system, the method comprising: accessing a list of active objectives, each one of the active objectives comprising a set of parameters and first computer-readable instructions which upon being executed by the autonomous system result in the autonomous system performing a task in accordance with the set of parameters (Della Penna, ¶ [0017], [0020], [0023], [0025], [0028], and [0034] with FIG. 1: autonomy controller 150 configured to determine trajectories for an autonomous vehicle 120 to facilitate driverless, collision-free navigation via a path of travel based on computed trajectories 122; whether a valid reason or goal was obtained (e.g., assuring safety as a goal); navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; the operation can be safely performed without impacting vehicle 119; controlling vehicle to transit lane 111 of roadway 126 based on one or more trajectories 122, as determined by control signals originating from control devices (e.g., steering wheel, a throttle pedal, a brake pedal, a transmission shifter, etc.; receive data representing steering data (e.g., degree of wheel angle to effect a turn), acceleration data ( e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., representing a selected gear and/or a direction), and the like; apply changes to at least steering, acceleration and deceleration at a rate of thirty (30) times a second or greater; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; ¶¶ [0038]-[0040] and [0042] with FIG. 1: vehicle controller 154 also may be configured to generate trajectories or paths of travel 122 in accordance with a planned route to guide the transiting of autonomous vehicle 120 via lanes 111 and 113 of roadway 126; for a trajectory or path of travel 122, vehicle controller 154 may determine in real-time (or substantially in real-time) a number of path segments constituting a path of travel along roadway 126; to transit along a segment, vehicle controller 154 may compute a number of vehicular drive parameters (i.e., computed vehicular drive parameters) that may be applied incrementally to mechanical drive components (e.g., at a rate of 30 sets of vehicular drive parameters for every second) to cause autonomous vehicle 120 to automatically drive along trajectory-based path segments over roadway 126; vehicle controller 154 may be configured to compute one or more drive parameters in real-time (or substantially in real-time) with which to apply to vehicle control unit 123, including driving control signals to effect propulsion, steering, braking, transmission shifting, lighting (e.g., emergency flashers), sound (e.g., automatic horn alerts, etc.), among other functions; vehicle controller 154 may be configured to calculate a variety of trajectories per unit time (e.g., per second), in real-time or substantially in real-time, that may be used to guide autonomous vehicle along a route from a point of origination to a point of destination, most of which may be calculated to facilitate driverless control; e.g., vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; route planning (e.g., planning paths of travel relative to roadway 126; a planned route along various paths of travel; ¶ [0046] with FIG. 2: trajectory generator 258 may be configured to generate trajectories or paths of travel in accordance with a planned route to guide the transiting of an autonomous vehicle via a roadway from origination point "A" (not shown) to destination point "B," such as a destination; to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; ¶ [0053] with FIG. 3: decision-making computer 359 may be configured to determine and planned routes by generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.));
accessing a first database populated with static environment properties, the static environment properties comprising second computer-readable instructions defining properties of entities and relations between the entities, the entities and the relations between the entities defining an environment in which the autonomous system is configured to operate (Della Penna, ¶ [0022] with FIG. 1: static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; ¶¶ [0033] and [0036] with FIG. 1: at least a subset of the aforementioned sensors of sensor platform 121 may be used to localize autonomous vehicle 120 at reference point 127 relative to its environment and objects within the environment (e.g., relative to roadway markings, a lamp post, a tree, and the like), and relative to a position in a global coordinate system (e.g., using GPS coordinates); vehicle controller 154 may determine a pose (e.g., a position and/or orientation) localized at a reference point 127 of autonomous vehicle 120; reference point 127 may be identified relative to external objects and surfaces of an external environment (or scene), and may be correlated to a position on a roadway 126, which may be described in map data 151; vehicle controller 154 may be configured to determine a position of reference point 127 relative to monuments or markers that may be used as known locations or points in a coordinate system to confirm or facilitate localization of autonomous vehicle 120 relative to, for example, roadway 126; ¶¶ [0040] and [0042] with 151 in FIG. 1: map manager 152 may be configured to implement map data 151 to localize and navigate autonomous vehicle 120 relative to roadway 126 or any pathway or route, any of which may be represented as image data; map data 151 may include relatively high resolutions of images of roadway 126 and adjacent objects, such as communication tower 198 and the like; map data 151 may include static or semi-static objects that have a relatively low or negligible probability of moving positions; static objects may be used as monuments or markers; autonomy controller 150 may use map data 151 to identify external imagery to facilitate route planning; map data 151 may include image data representing lane markings as well as data representing lane widths and curbs (e.g., with curb markings, such as "loading zone," etc.); map data 151 may also include any type of map data, such as 2D map data, 3D map data, 4D map data (e.g., includes three-dimensional map data at a particular point in time), or the like; additionally, map data 151 may include route data, such as road network data, including, but not limited to, route network definition file ("RNDF") data (or similar data) and the like; map data 151 may include images in high resolutions that include granular details of an environment or scene in which an autonomous vehicle is driving to ensure relatively accurate and precise localization, object classification, navigation, path of travel generation (e.g., trajectory generation), etc., as well as ensuring accurate and precise customized orientation and positioning when self-parking a vehicle; portions of map data 151 associated with a planned route along various paths of travel may be downloaded (e.g., as adjacent blocks of grid-type HD map data) as an autonomous vehicle travels along the route, thereby preserving resources (e.g., relatively large amount of storage need not be required to store an entire HD map of a particular region, such as a country); ¶ [0044] with FIG. 2: localizer 253 can determine a pose (e.g., a local position and orientation) at any one of number of geographic locations; localizer 253 may use acquired sensor data, such as sensor data associated with lamp posts, trees, or surfaces of buildings (e.g., a garage), which can be compared against reference data, such as map data (e.g., 3D map data, including reflectance data) to determine a local pose; localizer 253 may determine a relative geographic location of an autonomous vehicle relative to, for example, a global coordinate system (e.g., latitude and longitudinal coordinates, etc.)); accessing a second database populated with dynamic environment properties, the dynamic environment properties comprising third computer-readable instructions generated by the autonomous system based on events having been observed by the autonomous system, the events having occurred during operation of the autonomous system in the environment (Della Penna, ¶¶ [0022] and [0026]-[0032] with 156-157 in FIG.1: examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; also, dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; pattern data stored in event storage 157 may be used by event recorder 156 to determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel; detect via a sensor platform 121 that a vehicle 119 in lane 113 obstructs the new path of travel 199; ¶ [0041] with FIG. 1: map manager 152 may also be configured to generate a dynamic representation of map data 151 by fusing or combining static map data (e.g., image data representing visual characteristics of roadway 126 and static objects, such as road markings, tree 144, stop sign 146, etc.) and dynamic map data to form dynamic map data 151; dynamic map data may include data representing objects detected via image capture (and/or other sensor data, including lidar), whereby an object may have attributes indicative of dynamism, such as a pedestrian or a cyclist) (Della Penna, ¶¶ [0017]-[0027] with 156 in FIG. 1: autonomy controller 150 may include an event recorder 156 that may be configured to receive data from multiple sources whether internal or external to autonomous vehicle 120, and further configured to identify an interval of time in which to store a subset of received data (e.g., event data) associated with an event in, for example, an event storage repository 157; an example of captured data from the multiple sources includes control data such as steering data, throttle data, braking data transmission shifting data, etc.; another example of captured data from the multiple sources includes computed vehicular drive parameters, such as a degree of wheel angle, an amount of throttle, an amount of brake pressure, a state of transmission, and other computed values of which may be applied to facilitate driverless transit; thus, vehicular drive parameters include parameter data representing steering data (e.g., degree(s) of wheel angle to effect a turn), acceleration data (e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., a state of a transmission subsystem to effect forward motion and reverse motion in one or more states of speed and torque), and the like; yet another example of captured data from multiple sources include derived data (e.g., metadata) calculated as a result of computations or processing other data to determine various states, responses, etc. to facilitate driverless operation, such as a list of determined objects in an environment (e.g., lamp posts, trees, bicycles, cars, signs, pedestrians, cyclists, dogs, fire hydrants, etc.), and a state of an "ESP" flag indicating whether an electronic stability program ("ESP") is activated to provide stability or traction control (e.g., responsive to skidding on ice); other examples of captured data from the multiple sources may include sensor data (e.g., lidar data, radar data, image data, GPS data, wheel or odometry data, accelerometer data, ambient or external air temperature, grade or angle of a roadway surface, etc.); event recorder 156 may be configured to capture steering wheel data, acceleration data, braking data, and the like; static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; autonomy controller 150 and/or event recorder 156 in autonomous vehicle 120 may be configured to monitor various streams of data regarding the performance and control of autonomous vehicle 120 from a variety of sources to detect an event; ¶¶ [0037] and [0045] with FIGS. 1-2: object recognizer 255 may be configured to implement object characterization and classification to identify types and attributes of objects (e.g., whether an object is dynamic or static, whether an object is animate, or living, rather than an inanimate object, etc.); vehicle controller 254 may detect and classify objects to generate an object list 230, which includes a list of objects, such as object ("1") 231, object ("2") 232, object ("3") 233, etc.; the objects may represent detect and/or classified objects detected by one or more sensors; e.g., objects 231, 232, and 233 may include static objects, such as a lamp post, and dynamic objects, such as a person walking; ¶ [0052] with FIG. 3: perception engine 355 may be configured to receive various subsets of data, and further configured to detect and classify objects, such as objects in object list 230 of FIG. 2, based on characteristics of an object (e.g., object characteristics); a classified object may trigger an indication of an event; a classified object may include ice, potholes, traffic cones, signs, etc.; any of which may be determined to trigger an event during which at least a subset of data is a recorded by event recorder 356; perception engine 355 may also be configured to predict locomotive behavior of external objects (e.g., predicting a tree is static or stationary, whereas a cyclist is dynamic and moves); perception engine 355 may transmit object data 370, which includes data describing one or more objects, to event recorder 356; ¶ [0056] with 404 in FIG. 4: data representing control signals originate in one or more control devices in an autonomous vehicle may be monitored; sensor data signals originating on one or more sensors internal or external to an autonomous vehicle may be monitored, whereby at least a subset of values representing sensor data signals may be stored in an event storage repository);
upon observing, by the autonomous system, a new event in the environment: generating a new dynamic environment property based on the new event; entering the new dynamic environment property to the second database; executing coherence checking on the new dynamic environment property and the static environment properties, the coherence checking comprising comparing the new dynamic environment property with the static environment properties to assess whether the new dynamic environment property conflicts with at least one of the static environment properties; and identifying the new dynamic environment property as incoherent if determination is made that the new dynamic environment property conflicts with at least one of the static environment properties (Della Penna, ¶¶ [0017] and [0020]-[0032] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120); autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); identify one or more anomalies associated with an event; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c).; note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶¶ [0077]-[0080] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event).
Claim 12
Della Penna discloses all the elements as stated in Claim 11 and further discloses determining whether the new dynamic environment property is incoherent and, if the new dynamic environment property is incoherent: accessing a third database populated with candidate objectives, each one of the candidate objectives comprising fourth computer-readable instructions which upon being executed by the autonomous system result in generating the first computer-readable instructions; generating a new active objective from the candidate objectives based on the new event; entering the new active objective to the list of active objectives (Della Penna, ¶¶ [0023]-[0035] and [0039] with FIG. 1: an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; generate control data 372 that may be transmitted to vehicle components 366 to effect autonomous driving; ¶¶ [0059]-[0063] with 406-412 in FIG. 4: when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; ¶¶ [0064]-[0074] with FIG. 5: as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0081]-[0082] with FIG. 7: to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 13
Della Penna discloses all the elements as stated in Claim 11 and further discloses wherein the execution of coherence checking comprises: if determination is made that the new dynamic environment property is conflicting with at least one static environment properties of the first database, marking the new dynamic environment property as incoherent, a conflict between the new dynamic environment property and at least one static environment properties being caused by an opposition of their respective computer- readable instructions; and identifying the new dynamic environment property as coherent otherwise (Della Penna, ¶¶ [0017] and [0020]-[0032] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120); autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); identify one or more anomalies associated with an event; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c).; note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶¶ [0077]-[0080] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event).
Claim 14
Della Penna discloses all the elements as stated in Claim 11 and further discloses wherein generating a new active objective from the candidate objectives based on the new event comprises: selecting a candidate objective from the candidate objectives of the third database; generating new active objective parameters based on the new event; and associating the new active objective parameters to the selected candidate objective (Della Penna, ¶¶ [0023]-[0035] and [0039] with FIG. 1: an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; generate control data 372 that may be transmitted to vehicle components 366 to effect autonomous driving; ¶¶ [0059]-[0063] with 406-412 in FIG. 4: when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; ¶¶ [0064]-[0074] with FIG. 5: as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0081]-[0082] with FIG. 7: to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 15
Della Penna discloses all the elements as stated in Claim 14 and further discloses wherein each candidate objective of the third database comprises an activation condition, the activation condition corresponding to one or more dynamic environment properties and the activation condition being fulfilled if determination is made that the corresponding one or more dynamic environment properties are found in the second database, and a selection of a candidate objective from the candidate objectives of the third database is based on the activation condition of the candidate objective (Della Penna, ¶¶ [0017], [0020]-[0035], and [0039] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119, and initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; thus, path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation and to vehicle components 366 to effect autonomous driving; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; as such, computed tire angles (and associated throttle or brake amounts) may be stored in an event storage repository for subsequent transmission as a function of transmission criteria; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; identify one or more anomalies associated with an event, and to determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c); note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0077]-[0082] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event; to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 16
Della Penna discloses all the elements as stated in Claim 14 and further discloses wherein the first database further comprises common-sense rules, each common-sense rule comprising fifth computer-readable instructions defining general properties of the entities and general relations between the entities, each general property being property of a group of entities and each general relation between the entities being relation between groups of entities (Della, ¶¶ [0019]-[0020], [0022], [0024]-[0025], [0029], [0060], [0064]-[0067] with FIG.1: derived data may include computed vehicular drive parameters, may represent a course of action based on one or more rules (e.g., maintaining a lane offset due to detection of cones demarcating a construction zone); autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian (e.g., if the operation can be safely performed without impacting vehicle 119); modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; during "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input (e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, e.g., permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; in another example, illegally parked vehicles or trees may occlude sensing ( e.g., imagery, lidar, radar, etc.) of nearby pedestrians or traffic signs when an autonomous vehicle approaches; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid).
Claim 17
Della Penna discloses all the elements as stated in Claim 16 and further discloses wherein the common-sense rules are populated by an operator of the autonomous system and describe a cultural context of the environment in which the autonomous system is configured to operate (Della, ¶¶ [0019]-[0020], [0022], [0024]-[0025], [0029], [0060], [0064]-[0067] with FIG.1: derived data may include computed vehicular drive parameters, may represent a course of action based on one or more rules (e.g., maintaining a lane offset due to detection of cones demarcating a construction zone); autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian (e.g., if the operation can be safely performed without impacting vehicle 119); modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; during "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input (e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, e.g., permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; in another example, illegally parked vehicles or trees may occlude sensing ( e.g., imagery, lidar, radar, etc.) of nearby pedestrians or traffic signs when an autonomous vehicle approaches; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid).
Claim 18
Della Penna discloses all the elements as stated in Claim 16 and further discloses wherein, upon entering a new dynamic environment property based on a new event in the second database, coherence checking is further executed on the new dynamic environment property and the common-sense rules, and if determination is made that the new dynamic environment property conflicts with at least one of the common-sense rules: accessing the third database populated with candidate objectives; generating a new active objective from the candidate objectives based on the new event; and entering the new active objective to the list of active objectives (Della Penna, ¶¶ [0017], [0020]-[0035], and [0039] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119, and initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; thus, path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶¶ [0046]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation and to vehicle components 366 to effect autonomous driving; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; as such, computed tire angles (and associated throttle or brake amounts) may be stored in an event storage repository for subsequent transmission as a function of transmission criteria; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; identify one or more anomalies associated with an event, and to determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c); note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0077]-[0082] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event; to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 19
Della Penna discloses all the elements as stated in Claim 11 and further discloses wherein upon entering a new dynamic environment property in the second database, coherence checking is further executed on the new dynamic environment property and the list of active objectives, and, if determination is made that the new dynamic environment property conflicts with at least one of the active objectives, removing the at least one of the active objectives from the list of active objectives (Della Penna, ¶¶ [0017], [0020]-[0035], and [0039] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; an autonomy controller 150 configured to capture data relevant to events and initiate actions to resolve functional discrepancies and enhance reliability of autonomous logic (e.g., hardware or software, or a combination thereof) implemented in autonomy controller 150, thereby facilitating driverless navigation and propulsion reliably; autonomy controller 150 may be configured to analyze event data and initiate one or more actions (e.g., corrective actions); autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119, and initiate one or more actions (e.g., corrective actions), such as generating updated logic or software; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); upon such an analysis, logic or software of autonomy controller 150 may be updated to generate enhanced or refined rules of operation (e.g., updated autonomy controller logic); updates to logic or software of autonomy controller 150 may be transmitted as data 136 from event-adaptive computing platform 109 to autonomy controller 150 of autonomous vehicle 120; exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; modify a path of travel to a new path of travel 199 into lane 113 to avoid defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120) to identify modifications to autonomy controller 150 or other autonomous-related data to resolve, minimize, or negate similar conflicting vehicular drive parameters (e.g., applied versus computed) relative to defect 140; e.g., event-adaptive computing platform 109 may transmit updated map data 136 to be used for other autonomous vehicles 120, where map data 136 identifies pothole 140 based on above-described Z-axis acceleration data; the updated map data may be used to avoid pothole 140 during subsequent travels over lane 111; event-adaptive computing platform 109 may be configured to modify any portion of logic for implementing autonomy controller 150, or the like, of autonomous vehicle 120 to determine an optimal subset of actions or rules autonomy controller 150 may implement similar subsequent situations relating to, for example, pothole 140; autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; autonomy controller 150 may be configured to maintain a lane offset along path 125 relative to cones 142 to navigate into lane 113 without collision around objects 142; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; thus, path 125 around cones 142 may be initiated sooner (e.g., farther away from the construction zone) so as to provide comfort to the classified user so that they need not feel compelled to assume manual control of autonomous vehicle 120; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; event-adaptive computing platform 109 configured to analyze pre-event data and post-event data to, e.g., predict or identify an event, and to determine an action responsive to the event, including corrective actions; vehicle control unit 123 may receive updates of above-described data (e.g., vehicular drive parameters) to facilitate course corrections or modifications, if any, to ensure autonomous vehicle 120 traverses over path of travel based on one or more trajectories 122; autonomy controller 150 may include logic configured to generate and implement one or more paths of travel, such as paths of travel 122 and 125; vehicle controller 154 may select and implement a trajectory relative to locations of external dynamic and static objects along a sequence of roadways that provides for collision-free travel over the roadways, such as roadway 126; thus, autonomy controller 150 may also be configured to compute vehicular drive parameters based on the calculated trajectories to facilitate transit of autonomous vehicle 120 to a destination geographical location; ¶ [0041]: dynamic map data may include temporally-static objects (e.g., semi-static objects), which may be temporally static for a certain duration of time ( e.g., during construction or times of day) and may be added or removed dynamically from a mapped environment; ¶¶ [0045]-[0048] with FIG. 2: to determine a trajectory-based path of travel, trajectory generator 258 may determine in real-time (or substantially in real-time) a number of path segments to evaluate a collision-free path of travel along a roadway; trajectory generator 258 may implement object list 230 to select trajectories that may avoid collisions with objects 221, 232, and 233; to transit along a segment, trajectory generator 258 may compute a number of vehicular drive parameters that may be applied incrementally to mechanical drive components to cause an autonomous vehicle to traverse along path segments over the roadway without driver input; receive status data 245 which may include state data about one or more components or sub-systems of an autonomous vehicle (e.g., existence of high temperatures in an electrical power plant or in other electronics, a state of power degradation or voltage degradation, etc.); responsive to state data of the one or more components or sub-systems, event recorder 256 may be configured to modify a path of travel associated with a parking spot to, e.g., modify an orientation or position of the vehicle as it travels; event recorder 256 may be configured to capture or record data associated with generated path planning, such as selecting an optimal path of travel that is collision-free based on, e.g., terminating transit in a specialized orientation and position; event recorder 256 may also record computed vehicular drive parameters as (or as part of) command data, such as steering data ("s") 241a, throttle data ("t") 242a, braking data ("b") 243a, or any other data ("o") 244a, such as transmission shifting data (e.g., data describing gear and either a forward or reverse direction), for execution by vehicle control unit 223, which, in tum, may generate low-level commands or control signals for application to actuators or other mechanical or electro-mechanical components to cause changes in steering angles, velocity, etc.; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; perception engine 355 may provide data to decision-making computer 359 to assist in deciding one or more courses of action autonomy controller 350 may undertake to control navigation and propulsion of an autonomous vehicle driverlessly; generating trajectories relative to objects in a surrounding environment, whereby a subset of trajectories may be selected to facilitate collision-free travel; intermediate results or decisions (determined by decision-making computer 359) may also include one or more commands generated by logic in autonomy controller 350, such as commands relating to generating values of steering angles, velocities, braking pressures, etc.; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation and to vehicle components 366 to effect autonomous driving; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; when an autonomous vehicle encounters a slippery surface, such as ice, and begins to skid, an autonomy controller may be configured to compute tire angles to steer the vehicle "into" a skid to optimally recover control; the computed tire angles may constitute data representing a selected course of action based on an object (e.g., patch of ice as identified via capture device) and/or on different angular velocities of tires indicative of a lack of traction for at least one tire; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; as such, computed tire angles (and associated throttle or brake amounts) may be stored in an event storage repository for subsequent transmission as a function of transmission criteria; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an autonomy controller may detect an event and then select data representing a course of action based on the event (e.g., an object in a path of travel may be deemed or determined to be an obstacle); the autonomy controller may classify an object as a traffic cone, a patch of ice, a pothole, or the like; for a particular object, such as a pothole, an autonomy controller can select at least one set of executable instructions to implement a course of action or rule to generate a subset of vehicular drive parameters to address (e.g., avoid) the pothole; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; based on the probability and the object characteristics of an object (e.g., cyclists may travel faster than a pedestrian), an autonomy may be configured to select a course of action to facilitate collision-free travel; event data associated with the cyclist may be transmitted to an event-adaptive computing platform to analyze various aspects of the event (e.g., whether the object was correctly classified as a cyclist, whether another course of action may have been more optimal, etc.); after the analysis, updated software or autonomous logic may be transmitted back to an autonomous vehicle to update the autonomous logic; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; an autonomy controller may be configured to execute instructions to perform vehicle diagnostics to generate data characterizing vehicle anomaly; the characterized vehicle anomaly may be stored for subsequent transmission in association with event data; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); as an example of an exception or deviation from path planning rules, an analysis of data gathered by autonomous vehicle logic at an autonomous vehicle may be transmitted for analysis at event-adaptive computing platform 509; a result may be an update to software that improves the onboard autonomous vehicle logic to, for example, permit a vehicle to move through an intersection when a traffic control authority is detected, no obstacles are present, and, upon authorization from said authority to enter the intersection, or upon other rules or conditions; the aggregated human responses may be correlated with each other and/or with computed vehicle decisions to provide for real-time (or substantially real-time) updates to autonomous vehicle logic and rules so as to optimally navigate such events; identify one or more anomalies associated with an event, and to determine one or more courses of action to resolve such anomalies, including generating updates to data and/or executable instructions; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c); note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event resolver 557 includes a simulator 559, which can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; event resolver 557 may include logic to determine an optimal resolution, such as generating executable instructions to begin braking earlier at a sufficient distance to decelerate at a rate that minimizes skidding in icy conditions; event resolver 537 may be configured to modify map data to include stop sign 546 for future use; update generator 560 may be configured to generate an updated version of autonomy software (e.g., a patch) to download to autonomous vehicles to reduce or eliminate similar events; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶ [0076] with FIG. 6: event adaptation processor 650 is shown to include an event resolver 657, which, in turn, includes a simulator 659; event adaptation processor 650 may be configured to adapt operations of various components or subsystems of an autonomous vehicle 620 to resolve subsequent similar events, thereby enhancing safety, user experience, vehicle performance, etc.; to ensure sufficient time to stop to avoid a collision with vehicle 619 at intersection 699, event resolver 657 may control the simulation so as to test various "corner cases," whereby one or more control signals, sensor data values, and computed vehicular drive parameters may operate beyond normative ranges of operation during an event; simulator 659 can simulate of the application of various values for one or more of the control signals, the sensor data, and the subset of computed vehicular drive parameters to test and resolve the type of event; e.g., application of brake pressure may be implemented at various time intervals preceding detection of stop sign 646 at various distances 660 and at various velocities 682; simulator 659 can modify various parameters, including one or more roadway characteristics, which may include a degree of traction at surface, type of road material (e.g., pavement, asphalt, etc.), whether lane 611 is sloped uphill or downhill (and by how much), a degree of slipperiness during rain or icing conditions, etc.; ¶¶ [0077]-[0082] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event; to resolve the event, one or more values may be modified to identify or determine optimal autonomous vehicle operation; a subset of executable instructions may be generated for transmission or download via a network to update executable instructions in an autonomy controller at the autonomous vehicle responsive to the type of event).
Claim 20
Della Penna discloses all the elements as stated in Claim 11 and further discloses wherein one or more dynamic environment properties describing meta-events are generated if determination is made that predefined combinations of dynamic environment properties are found in the second database, each predefined combination of dynamic environment properties corresponding to a meta-event and causing a generation of a corresponding dynamic environment property in the second database (Della Penna, ¶¶ [0017]-[0027] with 156 in FIG. 1: autonomy controller 150 may include an event recorder 156 that may be configured to receive data from multiple sources whether internal or external to autonomous vehicle 120, and further configured to identify an interval of time in which to store a subset of received data (e.g., event data) associated with an event in, for example, an event storage repository 157; an example of captured data from the multiple sources includes control data such as steering data, throttle data, braking data transmission shifting data, etc.; another example of captured data from the multiple sources includes computed vehicular drive parameters, such as a degree of wheel angle, an amount of throttle, an amount of brake pressure, a state of transmission, and other computed values of which may be applied to facilitate driverless transit; thus, vehicular drive parameters include parameter data representing steering data (e.g., degree(s) of wheel angle to effect a turn), acceleration data (e.g., an amount of throttle or power to apply to a drive train or the like), deceleration data (e.g., an amount of pressure to apply to brakes to reduce velocity), transmission data (e.g., a state of a transmission subsystem to effect forward motion and reverse motion in one or more states of speed and torque), and the like; yet another example of captured data from multiple sources include derived data (e.g., metadata) calculated as a result of computations or processing other data to determine various states, responses, etc. to facilitate driverless operation, such as a list of determined objects in an environment (e.g., lamp posts, trees, bicycles, cars, signs, pedestrians, cyclists, dogs, fire hydrants, etc.), and a state of an "ESP" flag indicating whether an electronic stability program ("ESP") is activated to provide stability or traction control (e.g., responsive to skidding on ice); other examples of captured data from the multiple sources may include sensor data (e.g., lidar data, radar data, image data, GPS data, wheel or odometry data, accelerometer data, ambient or external air temperature, grade or angle of a roadway surface, etc.); event recorder 156 may be configured to capture steering wheel data, acceleration data, braking data, and the like; static objects may also include roadway defects, such as potholes, that may be detected (or may yet to be detected) when autonomy controller 150 identifies a roadway surface having a nonplanar surface portion; autonomy controller 150 and/or event recorder 156 in autonomous vehicle 120 may be configured to monitor various streams of data regarding the performance and control of autonomous vehicle 120 from a variety of sources to detect an event; ¶¶ [0037] and [0045] with FIGS. 1-2: object recognizer 255 may be configured to implement object characterization and classification to identify types and attributes of objects (e.g., whether an object is dynamic or static, whether an object is animate, or living, rather than an inanimate object, etc.); vehicle controller 254 may detect and classify objects to generate an object list 230, which includes a list of objects, such as object ("1") 231, object ("2") 232, object ("3") 233, etc.; the objects may represent detect and/or classified objects detected by one or more sensors; e.g., objects 231, 232, and 233 may include static objects, such as a lamp post, and dynamic objects, such as a person walking; objects of object list 230 may be described as being represented by computed data or "metadata" that may be used to identify an event as well as responsive action, such as corrective action (e.g., updated software or logic); ¶ [0052] with FIG. 3: perception engine 355 may be configured to receive various subsets of data, and further configured to detect and classify objects, such as objects in object list 230 of FIG. 2, based on characteristics of an object (e.g., object characteristics); a classified object may trigger an indication of an event; a classified object may include ice, potholes, traffic cones, signs, etc.; any of which may be determined to trigger an event during which at least a subset of data is a recorded by event recorder 356; perception engine 355 may also be configured to predict locomotive behavior of external objects (e.g., predicting a tree is static or stationary, whereas a cyclist is dynamic and moves); perception engine 355 may transmit object data 370, which includes data describing one or more objects, to event recorder 356; ¶ [0056] with 404 in FIG. 4: data representing control signals originate in one or more control devices in an autonomous vehicle may be monitored; sensor data signals originating on one or more sensors internal or external to an autonomous vehicle may be monitored, whereby at least a subset of values representing sensor data signals may be stored in an event storage repository; ¶¶ [0022] and [0026]-[0032] with 156-157 in FIG.1: examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; also, dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; pattern data stored in event storage 157 may be used by event recorder 156 to determine whether a subset of data matches a pattern of event data stored in the event storage 157; the pattern of event data may be associated with an event; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel; detect via a sensor platform 121 that a vehicle 119 in lane 113 obstructs the new path of travel 199; ¶ [0041] with FIG. 1: map manager 152 may also be configured to generate a dynamic representation of map data 151 by fusing or combining static map data (e.g., image data representing visual characteristics of roadway 126 and static objects, such as road markings, tree 144, stop sign 146, etc.) and dynamic map data to form dynamic map data 151; dynamic map data may include data representing objects detected via image capture (and/or other sensor data, including lidar), whereby an object may have attributes indicative of dynamism, such as a pedestrian or a cyclist; ¶¶ [0017] and [0020]-[0032] with FIG. 1: an event may be identified as a subset of observed values of data that deviate from a range of expected values (e.g., associated with expected behavior of a user or autonomy controller 150) for a specific condition (e.g., autonomous vehicle 120 may be descending a curved roadway during a time at which freezing rain may affect a surface of a roadway, whereby a human user and autonomy controller 150 may differ how to navigate the environmental conditions); an analysis of an event, based on recorded or captured data, may assist in improving actions performed by autonomous vehicle 120 or by interactions between a user and autonomous vehicle 120 via, for example, updates to on-board logic and associated processing; an "event" may refer to, at least in some examples, to one or more conditions (e.g., based on event data) that may cause anomalous or potentially anomalous operation of autonomous vehicle 120 (or a portion thereof); anomalous operation may be due to a behavior anomaly, a vehicle anomaly, an environmental anomaly, and the like; examples of a behavior anomaly may include user-specific driving-related behavior, such as user-specific rates of acceleration and turning, ranges of speed, rates of deceleration (e.g., amounts of braking pressure), driving patterns, and the like; behavior anomaly may also include a human user interaction with autonomy controller 150 or the control devices of autonomous vehicle 120, or both; e.g., human engagement or disengagement control devices with which to provide control signals for driving autonomous vehicle 120 may be an event; a deviation of applied vehicular drive parameter values from computed vehicular drive parameter values by a range of expected values may be deemed an event; analyze steering wheel data, acceleration data, braking data, and the like to reconcile or resolve whether a user and/or autonomy controller 150 operated non-normatively; autonomy controller 150 may be configured to detect deviations or violations of one or more rules, such as maintaining three feet of distance from a cyclist, whereby a deviation from a rule may be an event; examples of a vehicle anomaly include malfunctioning or suboptimal operation of one or more electrical, mechanical, electrical-mechanical, optical, etc. components of autonomous vehicle 120, such as a non-normative sensor (e.g., suboptimal lidar sensor), computational deficiencies (e.g., due to hardware, software or firmware), mechanical actuators (e.g., to cause wheels to turn or application of brake pads), and the like; event recorder 156 may be configured to capture data or information associated with a malfunctioning or suboptimal component or subsystem for analysis and resolution; examples of an environmental anomaly may include static objects that may lie upon one or more trajectories or a path of travel, or static objects (e.g., sloped roadway) that may affect one or more performance characteristics of autonomous vehicle 120 (e.g., increased resistance due to traveling uphill, or decreased friction or traction due to the ice or slippery roadway surfaces); static environmental anomalies may also include road topologies that differ from map data 151 (e.g., from HD map data), such as construction zones, new road markings, new signage (e.g., a new stop sign), and the like; examples of environmental anomalies that may include dynamic objects other vehicles that do not operate in accordance with normative traffic regulations, rules, patterns, etiquette, and behaviors; dynamic environmental anomalies may also include any road user, such as a cyclist, that moves in an unpredictable or unexpected manner; event recorder 156 may be configured to capture data or information associated with static and dynamic environmental anomalies for analysis and resolution; autonomy controller 150 may be configured to transmit event data 138 via communications tower 198 and networks 130 to event-adaptive computing platform 109 to analyze event data in view of other data from other autonomous vehicles 119; event-adaptive computing platform 109 may include centralized or distributed hardware and/or software configured to analyze numerous events associated with numerous autonomous vehicles 120 to identify patterns, deficiencies (whether functional or structural), or any other areas of improving navigation and propulsion of autonomous vehicle 120 in a safe, reliable manner; event recorder 156 may be implemented as on-board logic, algorithms, and processes configured to collect streams of data to track and analyze data coinciding at or substantially coextensive with events of interest (i.e., exceptional conditions, circumstances, or environments in which sensors, sensory platforms 121, and logic may detect exceptions that can be recorded for analysis); exceptions may be caused by an action of a user, autonomy controller 150, or an environmental event, or the like; an event of interest may be an instance during which human input (i.e., manual intervention) overrides autonomy controller 150 or autonomous operation to deviate (e.g., by a threshold range of expected values or actions) from one or more trajectories or courses of action computed by autonomy controller 150; e.g., an event may be an instant in which a human driver overrides a "rule," such as running a red light or crossing a double yellow line during periods of high traffic to evade collision with another vehicle 119 or a pedestrian ( e.g., if the operation can be safely performed without impacting vehicle 119); determine whether a subset of data matches a pattern of event data stored in the event storage 157, which may be used to identify non-normative operation ("an event") and evaluate, e.g., a human decision relative to autonomous vehicle logic; data associated with nonnormative operation may be transmitted as data 136 in view of particular transmission criteria to event-adaptive computing platform 109 for further analysis, whereas data associated with normal operation may be purged, at least in some cases, to preserve bandwidth and reduce non-beneficial computations or data transfers; event recorder 156 may "learn" characteristics, such as vehicular drive parameter values, associated with traversing a path of travel during which an anomaly may be detected as an event; event recorder 156 and event-adaptive computing platform 109, in combination, may "learn" which subset of characteristics may be modified to improve, e.g., reliable autonomous vehicle operation; identify a pothole as defect 140; identify the conflicting courses of action as an event, and begin recording one or more subsets of data as event data; detect a sharp fluctuation and is the Z-direction, which may be associated with a suspension system of autonomous vehicle 120 traveling into and out of a pothole 140; event-adaptive computing platform 109 may evaluate a subset of signals representing event data, including comparing the subset of signals against one or more patterns of similar event data (e.g., from other autonomous vehicles 120); autonomous vehicle 120 is driving via lane 111 prior to reaching a construction zone demarcated by objects 142, which are traffic cones; in this case, traffic cones 142 may not be stored as part of map data 151 (i.e., not part of an HD map) and may be detected precedentially via sensor platform 121 by autonomy controller 150; as detection of traffic cones 142 may be precedential, event recorder 156 may identify the detected cones 142 as an event; event recorder 156 may record multiple subsets of data associated with the event for transmission as data 138 to event-adaptive computing platform 109, which may be configured to, e.g., identify objects 142 as an environmental anomaly and to further characterize the driving behaviors of the human driver to classify the driver as a type of driver with which to modify subsequent encounters with objects 142; autonomous vehicle 120 is driving via lane 111 prior to reaching a stop sign 146, the view of which may be obstructed by a tree 144 as an obstacle; consider that stop sign 146 has been recently implemented, and, thus may not be included in map data 151 (e.g., anomalous controller 150 may not identify a geographic location at which stop sign 146 may be implemented); a camera or image capture device of sensor platform 121 may not identify subset 146 until autonomous vehicle 120 travels beyond tree 144, which then may require an immediate stop at which a human driver may intervene and apply significant braking pressure to avoid driving through stop sign 146; sudden deceleration and/or application significant braking pressure (e.g., slamming the brakes) to cause an immediate stop may be identified as an event; event recorder 156 then may transmit at least a subset of the recorded data 138 to event-adaptive computing platform 109, which may be configured to, e.g., generate updated map data 136 for revising map data 151 to include a location of stop sign 146 for subsequent approaches by autonomous vehicles 120 so as to adjust computed vehicular drive parameters to slowly stop at stop sign 146; ¶¶ [0052]-[0054] with FIG. 3: identifying an environmental anomaly in which a cyclist erroneously and arbitrarily changes its path of travel to intersect that of an autonomous vehicle, thereby creating a potentially hazardous situation; decision-making computer 359 may transmit results or decision data as selection data 371 to event recorder 356; generate control data 372 that may be transmitted to event recorder 356 for recordation; when the human user is driving, autonomy controller 350 may prevent application of control data 372 to vehicle components, and such data may be recorded by event recorder 356 for subsequent comparison at data computing center 390 relative to human input (e.g., determining whether a human driver an autonomous controller 350 operated similarly or in direct conflict, such as during an event; ¶¶ [0057]-[0063] with 406-412 in FIG. 4: an event may be detected which comprises values of either a subset of data representing control signals (e.g., applied vehicular drive parameters, which may be user-generated) or a subset of computed vehicular drive parameters, which may be generated by autonomous logic), or both, deviate from a range of expected values; consider that an event has been classified as a behavior anomaly based on, e.g., inconsistencies between at least one value of a control signal originating at a human user input and at least one value of a computed vehicular drive parameter; e.g., an autonomy controller may compute a wheel or tire angle of 10 degrees to negotiate a curve, whereas a user may apply input to a steering wheel to cause a wheel or tire to turn at an angle of 35 degrees, where the discrepancy or difference between angles may be sufficient to trigger an indication that an event has occurred; inconsistencies (e.g., deviations from a range of one or more expected values) between user input values (e.g., applied vehicular drive parameters) and computed vehicular drive parameters of an autonomy controller may be deemed an event; if the user turns "away" from the skid, which is counter to the commands generated by autonomous logic, such an action may be deemed an event as a user chooses to apply suboptimal control signals to address a skid; an event recorder may detect at least one control signal (e.g., a steering control input) that deviates from one or more values for at least one vehicular drive parameter (e.g., computed tire angles) for a particular situation or condition; a pothole or other roadway defect or obstacle may be identified as an environmental anomaly which, in turn, may be deemed to trigger a description of said defect or obstacle as an event; an unpredictable dynamic object, such as an errant cyclist, may be identified as an environmental anomaly which, in turn, may also be classified as an event; an autonomy controller may be configured to predict a subset of trajectories relative to a detected object, such as a dynamic object; an event may be detected upon determining a subset of trajectories of an autonomous vehicle may intersect a path of travel of the object (e.g., cyclist), thereby possibly causing a collision; an event may arise indicating at least one sensor internal or external to an autonomous vehicle is operating sub-optimally, or is an operable; thus, the event may be classified as a vehicle anomaly; detection of an event may trigger event data to be stored; the event data may represent one or more control signals or one or more vehicular drive parameters, or both; transmission control criteria may be determined, whereby event data may be transmitted in accordance with the transmission control criteria to facilitate data transfer to an event-adaptive computing platform at 412; ¶¶ [0064]-[0074] with FIG. 5: event-adaptive computing platform 509 may include logic configured to receive data ( e.g., remotely) from numerous autonomous vehicles 519a, 519b, 519c, and 519d, and to correlate one or more transmitted patterns of data (e.g., event data); such patterns of data may be indicative of "non-normative" events occurring at any of autonomous vehicles 519a, 519b, 519c, and 519d during an event in which, e.g., human intervention is asserted to control an autonomous vehicle 519; during such situations, human-provided inputs may deviate to a specific degree (e.g., a threshold range of one more parameter values) from computed vehicular drive parameters (e.g., trajectories), pre-programmed input ( e.g., drive parameters based on rules), or dynamically-generated logic, such as path planning rules; examples of such path planning rules may include prohibitions against automatically driving through red lights or crossing a double yellow line during periods of high traffic; as an example, a subset of autonomous vehicles may have a human operator who assumes control in order to drive through a malfunctioning traffic light (not shown) at an intersection of San Antonio Road and El Camino Real in which the traffic light is blinking red in all directions (indicating that all vehicles must stop and proceed individually through the intersection when safe to do so); identify one or more anomalies associated with an event; event recorders (not shown) in each autonomous vehicle may detect one or more events; one event may be responsive to a human driver intervening to apply hard brakes to stop the vehicle; another event may include detection of skidding tires (e.g., of autonomous vehicles 519b and 519c).; note that either a driver or an autonomy controller of autonomous vehicle 519b is turning the wheels in direction 543, which is in the direction of the skid; yet another event may include detection of a driver turning away (e.g., wheels turned into direction 545) from the direction of the skid 541, which may be inapposite and inconsistent with application of rules by an autonomous controller that generate drive parameters to steer autonomous vehicle into the direction of the skid; event predictor 553 is configured to identify an event and its characteristics; event predictor 553 may be configured to analyze data 552a to 552d to determine an event and associated one or more behavioral, vehicular, and environmental anomalies; data value detector 554 may be configured to detect one or more data values surpass a threshold or are in a range of values indicative of non-normative operation; pattern matcher 555 may be configured to match event data 536a to 536b against patterns of other stored event data to determine a match specifying, for example, "skidding" as an event; correlator 536 may be configured to correlate one or more subsets of event data 536a to 536d; correlator 536 may determine that autonomous vehicles 519a, 519b, and 519c stopped abnormally fast, whereas autonomous vehicle 519d stopped at a normative rate of deceleration; event adaption processor 550 may analyze correlated data to determine that, based on historic GPS data, autonomous vehicle 519d has traveled over roadway 511 many times before, but autonomous vehicles 519a, 519b, and 519c each encounter to stop sign 546 for the first time during the event; event adaptation processor 550 may be configured to generate normative models of pedestrians, vehicles, and other objects based on vehicle data aggregated at event-adaptive computing platform 509; an autonomy controller may compare observed data against modeled predictive data to detect an event; e.g., if an error between two measurements exceeds a threshold, an event recorder (not shown) may be configured to store the data related to the event, such as 30 seconds before and 30 seconds after a triggering signal is generated; ¶¶ [0077]-[0080] with FIG. 7: driver behavior may be classified and compared to other driver behaviors relative to the stop sign to determine whether the environmental anomaly (i.e., occluded stop sign) may actually be a behavior anomaly ( e.g., the driver associated with event is aggressive and tends to "slam the brakes."); identify a type of event).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Miguelañez et al. ("Semantic Knowledge-Based Framework to Improve the Situation Awareness of Autonomous Underwater Vehicles", IEEE Transactions on Knowledge and Data Engineering, VOL. 23, NO. 5, MAY 2011, pp. 759-773) discloses in ABSTRACT of Page 759 that (1) proposes a semantic world model framework for hierarchical distributed representation of knowledge in autonomous underwater systems; (2) this framework aims to provide a more capable and holistic system, involving semantic interoperability among all involved information sources; (3) this will enhance interoperability, independence of operation, and situation awareness of the embedded service-oriented agents for autonomous platforms; (4) the results obtained specifically affect the mission flexibility, robustness, and autonomy; (5) the presented framework makes use of the idea that heterogeneous real-world data of very different type must be processed by (and run through) several different layers, to be finally available in a suited format and at the right place to be accessible by high-level decision-making agents; (6) in this sense, the presented approach shows how to abstract away from the raw real-world data step by step by means of semantic technologies; (7) the paper concludes by demonstrating the benefits of the framework in a real scenario; (8) a hardware fault is simulated in a REMUS 100 AUV while performing a mission; (9) this triggers a knowledge exchange between the status monitoring agent and the adaptive mission planner embedded agent; and (10) by using the proposed framework, both services can interchange information while remaining domain independent during their interaction with the platform.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HWEI-MIN LU whose telephone number is (313)446-4913. The examiner can normally be reached Mon - Fri: 9:00 AM - 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D. Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HWEI-MIN LU/Primary Examiner, Art Unit 2142
Prosecution Insights