Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-21 are pending
Claims 1, 7, 14, 16, 18, 20 are amended
Claim 21 is new
Response to Arguments
Rejections under 35 U.S.C. 101
Under new guidance, the generation of training data is now considered an active step and therefore the rejection under 35 U.S.C. 101 has been withdrawn.
Rejections under 35 U.S.C. 102
Applicant’s arguments with respect to claims 1 – 20 have been considered but are moot in view of the new ground(s) of rejection as necessitated by applicant's amendments. More specifically, the modification that the label additionally characterize the suboptimal invokes a change of scope wherein further search and consideration is necessary.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-3, 9-15, and 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG (US 20220297726 A1) in view of BELTA (US 20230391367 A1).
Regarding claim 1:
ZHNANG discloses:
A computer-implemented method for generating training data for training one or more machine-learned models of an autonomous vehicle control system, comprising: (see at least ZHANG, ¶ 0005, “In some embodiments, a first portion of the one or more series of successive sensor data frames in the training dataset is labeled as having an unsafe driving condition and a second portion of the one or more series of successive sensor data frames in the training dataset is labeled as having a safe driving condition.”; ¶ 0029, “In various embodiments, the autonomous vehicle 100 may navigate through roads, streets, and/or terrain with limited or no human input. The word “vehicle” or “vehicles” as used in this paper includes vehicles that travel on ground such as cars, trucks, and busses, but may also include vehicles that travel in air such as drones, airplanes, and helicopters, vehicles that travel on water such as boats, and submarines. Further, “vehicle” or “vehicles” discussed in this paper may or may not accommodate one or more passengers therein. In general, the autonomous vehicle 100 can effectuate any control to itself that a human driver can on a conventional vehicle. For example, the autonomous vehicle 100 can accelerate, brake, turn left or right, or drive in a reverse direction just as a human driver can on the conventional vehicle. The autonomous vehicle 100 can also sense environmental conditions, gauge spatial relationships between objects and the autonomous vehicle 100, detect and analyze road signs just as the human driver. Moreover, the autonomous vehicle 100 can perform more complex operations, such as parallel parking, parking in a crowded parking lot, collision avoidance, without any human input.”)
(a) obtaining log data describing a trajectory of a vehicle traveling through an environment; (see at least ZHANG, ¶ 0024, “Embodiments described in this application provide a system and method of improving testing of embedded algorithms and/or logic of autonomous vehicles through simulation scenarios resembling real-world driving conditions. The embedded algorithms and/or logic may determine or receive data regarding unsafe or dangerous situations and react accordingly to such situations. In various embodiments, sensor data frames during actual navigation of a vehicle may be used to train a machine learning model to recognize, detect, and/or determine potentially unsafe or dangerous, and/or unsafe or dangerous situations or scenarios. The trained machine learning model may be part of the embedded algorithms and/or logic, or may provide input to the embedded algorithms and/or logic. In some embodiments, the sensor data frames may be used to modify or update parameters, logic and/or algorithms specifically used by the machine learning model to detect such situations or scenarios. These situations or scenarios may further include situations which a passenger may perceive as unsafe or uncomfortable. In such a manner, the trained machine learning model may improve and/or expand its detection of unsafe and/or dangerous situations, so that the embedded logic and/or algorithms may be able to better react to such situations. Using inputs from the trained machine learning model regarding which situations are dangerous or unsafe, or potentially so, the embedded logic and/or algorithms may undergo simulations in order to ensure safety and efficacy of the embedded logic and/or algorithms before being deployed on a road. For example, the simulations may refine decision making of the embedded logic and/or algorithms. At least some scenarios used to create the simulations may also be generated from the sensor data frames, so that at least some of the scenarios may be similar or same as portions of the sensor data frames and reflect realistic scenarios. The simulations may be used to assess and ensure the vehicle's ability to complete a mission under a minimal completion time, interpret static and dynamic obstacles within an environment, adjust velocity and/or acceleration based on a context of road conditions, control locomotion and plan navigation, and navigate when sensor data lacks localization, for example, at a position where LiDAR range is insufficient.”)
(b) determining a suboptimal condition associated with the trajectory; (see at least ZHANG, ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”)
the suboptimal condition along one or more constraint dimensions of a motion planner of the autonomous vehicle control system; and (see at least ZHANG, ¶ 0009, “In some embodiments, the disengagement data can comprise at least one of steering wheel rotation data, acceleration pedal depression data, or brake pedal depression data.”; ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”)
(d) generating a training example for training the one or more machine-learned models of the autonomous vehicle control system to decrease a probability of the autonomous vehicle control system inducing the suboptimal condition, (see at least ZHANG, ¶ 0024; ¶ 0035, “The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101. The machine learning model may include, in some embodiments, a random forest model or a neural network. In some examples, the training engine 136 feeds, into the machine learning model, pairs of training datasets juxtaposed such that a first training dataset represent, indicates, and/or is labeled as a safe situation while a second training dataset represent, indicates, and/or is labeled as an unsafe situation. In such a manner, the machine learning model may infer and/or determine differences between the training datasets to determine factors and/or parameters that cause a situation to be unsafe. In some embodiments, a dataset labeled as unsafe corresponds to a vehicle being disengaged from an autonomous mode, while a dataset labeled as safe corresponds to a vehicle remaining in the autonomous mode. One example of the juxtaposition of a first training dataset that includes data corresponding to a safe situation and a second training dataset that includes data corresponding to an unsafe situation occurs when one training dataset includes a frame or a series of frames from a particular location determined or labeled as safe and a second training dataset includes a frame or a series of frames from that same particular location determined or labeled as unsafe. As another example, one training dataset may include a frame or a series of frames from a particular range of timestamps or locations determined or labeled as safe and a second training dataset may include a frame or a series of frames from that same particular range of timestamps or locations determined or labeled as unsafe. In such a manner, the machine learning model may be able to determine or infer which parameters or factors cause a situation to be safe or unsafe, such as a density of traffic, speed or acceleration of vehicles, lighting conditions, concentration of pedestrians, particular locations such as intersections or corners, or certain sounds. By juxtaposing the training datasets, in which at least one factor, such as a time range or a location, is kept constant or similar among the pair of training datasets, such as a time range or a location, the machine learning model may determine or infer whether, and to what extent, any of the differences between the juxtaposed data frames cause a situation to be unsafe. For example, if a difference between a situation determined to be safe and another situation determined to be unsafe at a same location, and/or having similar traffic density, is that the situation determined to be safe was obtained during daytime and the other situation determined to be unsafe was at nighttime, the machine learning model may infer that a lack of lighting causes a situation to be unsafe. By using multiple juxtaposed training datasets, the machine learning model may further isolate, refine and/or confirm parameters and/or factors used to determine a level of safety of a situation. Examples of such juxtaposed training datasets are shown in FIGS. 4-10.”)
wherein the training example comprises at least a suboptimal portion of the trajectory (see at least ZHANG, ¶ 0035; ¶ 0037)
ZHANG does not disclose, but BELTA teaches:
(c) generating a label that characterizes (see at least BELTA, ¶ 0025, “In some cases, the inference system may determine the continuation condition is satisfied when traces of a group include traces labeled a first label (e.g., a positive example of an autonomous vehicle action) and traces labeled a second label (e.g., a negative example of the autonomous vehicle action). In some cases, the inference system may determine the continuation condition is not satisfied when traces of a group include only traces (or a threshold percentage of traces) labeled the first label.”; ¶ 0095, “In some cases, the perception system 402 may associate the trajectory data to particular autonomous vehicle actions. For instance, the perception system 402 may associate trajectory data for a trip to actions taken during the trip. In some cases, the perception system 402 may segment the trajectory data for a trip (or some sub-part thereof) and label the segments in accordance with autonomous vehicle actions taken during the trip (or the sub-part of the trip). As an example, the planning system 404 may determine a particular autonomous vehicle action (e.g., maintain course, change lanes, turn right/left onto different street, stop at stop sign, stop at stop light, yield for pedestrian, and the like) and the perception system 402 may label corresponding trajectory data with the autonomous vehicle action taken.”)
and the label. (see at least BELTA, ¶ 0025; ¶ 0095)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, to modify the detection of the driver intervention and training engine for the autonomous vehicle with binary labeling for determining if a driving activity is safe or unsafe (¶ 0005) within ZHANG to implement a more detailed label including the vehicle actions taken in performing an action within BELTA to yield a safer autonomous driving data with more granular descriptions of the vehicular actions performed that led to the respective binary labels.
Regarding claim 2:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
determining a corrective action initiated by a human operator of the vehicle, wherein the vehicle is an autonomous vehicle. (see at least ZHANG, ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”)
Regarding claim 3:
ZHANG in view of BELTA discloses the limitations within claim 2 and ZHANG further discloses:
the human operator is onboard the autonomous vehicle. (see at least ZHANG, ¶ 0036, “The training dataset can include one or more sensor data frames or series of successive sensor data frames such as point cloud data frames timestamp-synchronized with disengagement data obtained from a data synchronization engine (e.g., the data synchronization engine 134 of FIG. 1B). The training engine 136 can label each data frame or each series of successive sensor data frames as being a safe or an unsafe driving condition based on whether disengagement data indicates that the vehicle 100 was disengaged at the time of capture of the data frame or series of data frames. For example, a series of successive sensor data frames captured by the vehicle 100 may depict a pedestrian standing at a sidewalk of an intersection at some distance ahead. Upon detecting the pedestrian, the vehicle 100 may continue to travel at current speed. A safety driver onboard the vehicle 100 may depress a brake pedal to slow down the vehicle 100 in anticipation of the pedestrian unexpectedly stepping into the intersection. In this example, disengagement data that includes data of depressing the brake pedal, indicated by pressure applied to the brake pedal, can be timestamp-synchronized with the series of successive sensor data frames. In this example, the series of successive sensor data frames can be labeled as having an unsafe driving condition (i.e., the anticipation of the pedestrian unexpectedly stepping into the intersection). In cases where a series of successive sensor data frames depicting driving conditions does not have associated timestamp-synchronized disengagement data, the series of successive sensor data frames can be labeled as having a safe driving condition because no human driver intervention was recorded during the time the series of successive sensor data frames was captured. Once the one or more series of successive sensor data frames of the training dataset are labeled, the training dataset can be used to train the one or more machine learning models. In some embodiments, labels provided by the training engine 136 can be further reviewed and modified by a human reviewer. For example, the training engine 136 may determine a series of successive sensor data frames as having a safe driving condition because there was no associated disengagement data. In this example, upon further review, a human reviewer may determine that the series of successive sensor data frames have an unsafe driving condition. In this example, the human reviewer may change a label for the series of successive sensor data frames as having an unsafe driving condition. In general, the training engine 136 can train the one or more machine learning models using any suitable training method. For example, in one implementation, training of the one or more machine learning models may be supervised. In other implementations, training of the one or more machine learning models may be unsupervised. Many variations are possible. The training engine 136 will be discussed in further detail with reference to FIGS. 3-10.”)
Regarding claim 9:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
(b) comprises receiving an annotation from a user device that indicates a preferred trajectory, different from the trajectory from the log data, that a user inputs in association with the log data. (see at least ZHANG, ¶ 0036, “The training dataset can include one or more sensor data frames or series of successive sensor data frames such as point cloud data frames timestamp-synchronized with disengagement data obtained from a data synchronization engine (e.g., the data synchronization engine 134 of FIG. 1B). The training engine 136 can label each data frame or each series of successive sensor data frames as being a safe or an unsafe driving condition based on whether disengagement data indicates that the vehicle 100 was disengaged at the time of capture of the data frame or series of data frames. For example, a series of successive sensor data frames captured by the vehicle 100 may depict a pedestrian standing at a sidewalk of an intersection at some distance ahead. Upon detecting the pedestrian, the vehicle 100 may continue to travel at current speed. A safety driver onboard the vehicle 100 may depress a brake pedal to slow down the vehicle 100 in anticipation of the pedestrian unexpectedly stepping into the intersection. In this example, disengagement data that includes data of depressing the brake pedal, indicated by pressure applied to the brake pedal, can be timestamp-synchronized with the series of successive sensor data frames. In this example, the series of successive sensor data frames can be labeled as having an unsafe driving condition (i.e., the anticipation of the pedestrian unexpectedly stepping into the intersection). In cases where a series of successive sensor data frames depicting driving conditions does not have associated timestamp-synchronized disengagement data, the series of successive sensor data frames can be labeled as having a safe driving condition because no human driver intervention was recorded during the time the series of successive sensor data frames was captured. Once the one or more series of successive sensor data frames of the training dataset are labeled, the training dataset can be used to train the one or more machine learning models. In some embodiments, labels provided by the training engine 136 can be further reviewed and modified by a human reviewer. For example, the training engine 136 may determine a series of successive sensor data frames as having a safe driving condition because there was no associated disengagement data. In this example, upon further review, a human reviewer may determine that the series of successive sensor data frames have an unsafe driving condition. In this example, the human reviewer may change a label for the series of successive sensor data frames as having an unsafe driving condition. In general, the training engine 136 can train the one or more machine learning models using any suitable training method. For example, in one implementation, training of the one or more machine learning models may be supervised. In other implementations, training of the one or more machine learning models may be unsupervised. Many variations are possible. The training engine 136 will be discussed in further detail with reference to FIGS. 3-10.”)
Regarding claim 10:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
determining the suboptimal portion based on an interval that is associated with the suboptimal condition. (see at least ZHANG, ¶ 0026, “In some embodiments, the training datasets used to train the machine learning model may be juxtaposed such that one training dataset includes one or more frames depicting a situation determined or labeled as safe and a second training dataset includes one or more frames depicting a situation determined or labeled as unsafe.”; ¶ 0034, “The data synchronization engine 134 can be configured to associate the disengagement data 120 and sensor data such as point cloud data frames generated by the data fusion engine 132 to indicate whether the vehicle 100 is disengaged from the autonomous mode at a time of capture of each sensor data frame. Thus, a pair of data frames, or a pair of series of data frames, can be readily juxtaposed in a manner such that one data frame or series in a first training dataset represents the vehicle 100 being disengaged from the autonomous mode and a second data frame or series in a second training dataset represents the vehicle remaining in the autonomous mode. In some embodiments, the data synchronization engine 134 can associate the disengagement data 120 and the sensor data frames by timestamp-synchronization. For example, a set of point cloud data frames can be associated with timestamps ranging from 1:00:00 to 1:10:00. In this example, data relating to a brake pedal depression action (i.e., a disengagement action) can be associated with or correspond to timestamps ranging from 1:05:00 to 1:05:30. In this example, the data relating to the brake pedal depression action can be synchronized to a subset of point cloud data frames in the set that corresponds to 1:05:00 to 1:05:30. In this way, data relating to disengagement actions a safety driver took to intervene in autonomous operations of the vehicle 100 can be synchronized to corresponding point cloud data frames to associate the disengagement actions to the corresponding point cloud data frames.”)
Regarding claim 11:
ZHANG in view of BELTA discloses the limitations within claim 10 and ZHANG further discloses:
a boundary of the interval is based on a corrective action initiated by a human operator; or (see at least ZHANG, ¶ 0034, “The data synchronization engine 134 can be configured to associate the disengagement data 120 and sensor data such as point cloud data frames generated by the data fusion engine 132 to indicate whether the vehicle 100 is disengaged from the autonomous mode at a time of capture of each sensor data frame. Thus, a pair of data frames, or a pair of series of data frames, can be readily juxtaposed in a manner such that one data frame or series in a first training dataset represents the vehicle 100 being disengaged from the autonomous mode and a second data frame or series in a second training dataset represents the vehicle remaining in the autonomous mode. In some embodiments, the data synchronization engine 134 can associate the disengagement data 120 and the sensor data frames by timestamp-synchronization. For example, a set of point cloud data frames can be associated with timestamps ranging from 1:00:00 to 1:10:00. In this example, data relating to a brake pedal depression action (i.e., a disengagement action) can be associated with or correspond to timestamps ranging from 1:05:00 to 1:05:30. In this example, the data relating to the brake pedal depression action can be synchronized to a subset of point cloud data frames in the set that corresponds to 1:05:00 to 1:05:30. In this way, data relating to disengagement actions a safety driver took to intervene in autonomous operations of the vehicle 100 can be synchronized to corresponding point cloud data frames to associate the disengagement actions to the corresponding point cloud data frames.”)
a boundary of the interval is based on a divergence of a preferred trajectory from the trajectory from the log data, the preferred trajectory obtained from a user input associated with the log data. (see at least ZHANG, ¶ 0035, “The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101. The machine learning model may include, in some embodiments, a random forest model or a neural network. In some examples, the training engine 136 feeds, into the machine learning model, pairs of training datasets juxtaposed such that a first training dataset represent, indicates, and/or is labeled as a safe situation while a second training dataset represent, indicates, and/or is labeled as an unsafe situation. In such a manner, the machine learning model may infer and/or determine differences between the training datasets to determine factors and/or parameters that cause a situation to be unsafe. In some embodiments, a dataset labeled as unsafe corresponds to a vehicle being disengaged from an autonomous mode, while a dataset labeled as safe corresponds to a vehicle remaining in the autonomous mode. One example of the juxtaposition of a first training dataset that includes data corresponding to a safe situation and a second training dataset that includes data corresponding to an unsafe situation occurs when one training dataset includes a frame or a series of frames from a particular location determined or labeled as safe and a second training dataset includes a frame or a series of frames from that same particular location determined or labeled as unsafe. As another example, one training dataset may include a frame or a series of frames from a particular range of timestamps or locations determined or labeled as safe and a second training dataset may include a frame or a series of frames from that same particular range of timestamps or locations determined or labeled as unsafe. In such a manner, the machine learning model may be able to determine or infer which parameters or factors cause a situation to be safe or unsafe, such as a density of traffic, speed or acceleration of vehicles, lighting conditions, concentration of pedestrians, particular locations such as intersections or corners, or certain sounds. By juxtaposing the training datasets, in which at least one factor, such as a time range or a location, is kept constant or similar among the pair of training datasets, such as a time range or a location, the machine learning model may determine or infer whether, and to what extent, any of the differences between the juxtaposed data frames cause a situation to be unsafe. For example, if a difference between a situation determined to be safe and another situation determined to be unsafe at a same location, and/or having similar traffic density, is that the situation determined to be safe was obtained during daytime and the other situation determined to be unsafe was at nighttime, the machine learning model may infer that a lack of lighting causes a situation to be unsafe. By using multiple juxtaposed training datasets, the machine learning model may further isolate, refine and/or confirm parameters and/or factors used to determine a level of safety of a situation. Examples of such juxtaposed training datasets are shown in FIGS. 4-10.”)
Regarding claim 12:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
selecting the trajectory from the log data based on a score computed for the trajectory. (see at least ZHANG, ¶ 0027, “The trained machine learning model may output a safety score in different simulations. A safety score can indicate a level of safety of a driving condition. In one implementation, the safety score can be binary, with zero indicating a simulation having an unsafe driving condition and one indicating a simulation having a safe driving condition. In another implementation, the safety score can range on a scale from one to ten, with one indicating a simulation having a least safe condition and ten indicating a simulation having a safest condition. Many variations are possible. Simulations that have satisfied (or exceeded) a threshold score can be classified as safe driving conditions while simulation scenarios not satisfying the threshold score can be classified as unsafe driving conditions. These simulations can be used to test embedded algorithms and/or logic in a simulation to assess and/or confirm how an autonomous vehicle would react to conditions depicted in the simulations. As previously alluded to, at least some of these simulations may be generated by portions of the captured sensor data frames so that the simulations reflect actual conditions that would be encountered by a vehicle.”; ¶ 0048, “FIG. 11 illustrates an example functional block diagram 1100 for determining safety scores for simulations, according to an embodiment of the present disclosure. In various embodiments, the functional block diagram 1100 can include a plurality of simulations 1142, 1144, 1146. Each of the plurality of simulations 1142, 1144, 1146 can comprise a series of sensor data frames, at least a portion of which depict a driving condition in an environment obtained from a data fusion engine (e.g., the data fusion engine 132 of FIG. 1B). Each of the plurality of simulations 1142, 1144, 1146 can be inputted into embedded logic 1150, which may include a machine learning model such as the machine learning model as trained by the training engine 136. The embedded logic 1150 can determine a safety score for each of the plurality of simulations 1142, 1144, 1146. In general, the more unsafe a driving condition depicted in a simulation is, the lower the safety score. As shown in FIG. 11, in the block diagram 1100, the embedded logic 1150 determines the simulation 1142 to have a safety score of 2, the simulation 1144 to have a safety score of 7, and the simulation 1146 to have a safety score of 3. Therefore, in the block diagram 1100, simulation 1142 has been determined by the embedded logic 1150 as having the most unsafe driving condition and the simulation 1144 has been determined by the embedded logic 1150 as having the safest driving condition. In some embodiments, simulations that have a safety score less than a threshold value can be used to test and/or revise embedded algorithms and/or logic of autonomous vehicles. For example, in some embodiments, a threshold value for selecting simulation scenarios is 5. In this example, any simulation scenario having a safety score of 5 or less can be selected for testing embedded algorithms and/or logic of autonomous vehicles. For instance, in FIG. 11, the simulation 1142 and the simulation 1146 can be selected for testing embedded algorithms and/or logic of autonomous vehicles.”)
Regarding claim 13:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
generating one or more additional training examples from the training example by: (see at least ZHANG, ¶ 0035, “The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101. The machine learning model may include, in some embodiments, a random forest model or a neural network. In some examples, the training engine 136 feeds, into the machine learning model, pairs of training datasets juxtaposed such that a first training dataset represent, indicates, and/or is labeled as a safe situation while a second training dataset represent, indicates, and/or is labeled as an unsafe situation. In such a manner, the machine learning model may infer and/or determine differences between the training datasets to determine factors and/or parameters that cause a situation to be unsafe. In some embodiments, a dataset labeled as unsafe corresponds to a vehicle being disengaged from an autonomous mode, while a dataset labeled as safe corresponds to a vehicle remaining in the autonomous mode. One example of the juxtaposition of a first training dataset that includes data corresponding to a safe situation and a second training dataset that includes data corresponding to an unsafe situation occurs when one training dataset includes a frame or a series of frames from a particular location determined or labeled as safe and a second training dataset includes a frame or a series of frames from that same particular location determined or labeled as unsafe. As another example, one training dataset may include a frame or a series of frames from a particular range of timestamps or locations determined or labeled as safe and a second training dataset may include a frame or a series of frames from that same particular range of timestamps or locations determined or labeled as unsafe. In such a manner, the machine learning model may be able to determine or infer which parameters or factors cause a situation to be safe or unsafe, such as a density of traffic, speed or acceleration of vehicles, lighting conditions, concentration of pedestrians, particular locations such as intersections or corners, or certain sounds. By juxtaposing the training datasets, in which at least one factor, such as a time range or a location, is kept constant or similar among the pair of training datasets, such as a time range or a location, the machine learning model may determine or infer whether, and to what extent, any of the differences between the juxtaposed data frames cause a situation to be unsafe. For example, if a difference between a situation determined to be safe and another situation determined to be unsafe at a same location, and/or having similar traffic density, is that the situation determined to be safe was obtained during daytime and the other situation determined to be unsafe was at nighttime, the machine learning model may infer that a lack of lighting causes a situation to be unsafe. By using multiple juxtaposed training datasets, the machine learning model may further isolate, refine and/or confirm parameters and/or factors used to determine a level of safety of a situation. Examples of such juxtaposed training datasets are shown in FIGS. 4-10.”)
perturbing a state in a direction that increases the suboptimality of the trajectory, the state comprising at least one of: (see at least ZHANG, ¶ 0039, “FIG. 5 illustrates an exemplary implementation of the computing system 130, in particular, of the training engine 136. In some embodiments, the training engine 136 may feed, into a machine learning model, a first training dataset 521 that includes one or more frames, or series of frames, having conditions identified as safe and a second training dataset 522 that includes one or more frames, or series of frames, conditions identified as unsafe. For example, the machine learning model may identify differences between the juxtaposed training datasets, specifically, between first training dataset 521 and the second training dataset 522. For example, the first training dataset 521 may be identified to include a driving situation in which two vehicles are driving straight on separate lanes while the second training dataset 522 may be identified to include a driving situation in which a vehicle is attempting to pass another, and both the first training dataset 521 and the second training dataset 522 may have been taken at a same or similar position and/or pose, and/or have similar or same distances between vehicles, in order to isolate relevant factors or parameters. Therefore, the machine learning model may infer that while passing, two vehicles need to maintain a larger distance to be considered a safe condition, compared to two vehicles driving straight on separate lanes. Thus, whether two vehicles are a safe distance apart depends on whether or not a vehicle is passing another vehicle. By using the first training dataset 521 and the second training dataset 522, the machine learning model may better identify, or refine its previous identification of, parameters or factors that cause a situation to be identified as dangerous. Thus, a vehicle may be able to stop or slow down once a situation is identified as dangerous. Other pairs of juxtaposed training datasets may depict vehicles travelling at different speeds. For example, a safe distance that two vehicles need to be separated by may be greater if one or both vehicles is/are travelling faster.”)
(i) a state of the vehicle or (see at least ZHANG, ¶ 0036, “The training dataset can include one or more sensor data frames or series of successive sensor data frames such as point cloud data frames timestamp-synchronized with disengagement data obtained from a data synchronization engine (e.g., the data synchronization engine 134 of FIG. 1B). The training engine 136 can label each data frame or each series of successive sensor data frames as being a safe or an unsafe driving condition based on whether disengagement data indicates that the vehicle 100 was disengaged at the time of capture of the data frame or series of data frames. For example, a series of successive sensor data frames captured by the vehicle 100 may depict a pedestrian standing at a sidewalk of an intersection at some distance ahead. Upon detecting the pedestrian, the vehicle 100 may continue to travel at current speed. A safety driver onboard the vehicle 100 may depress a brake pedal to slow down the vehicle 100 in anticipation of the pedestrian unexpectedly stepping into the intersection. In this example, disengagement data that includes data of depressing the brake pedal, indicated by pressure applied to the brake pedal, can be timestamp-synchronized with the series of successive sensor data frames. In this example, the series of successive sensor data frames can be labeled as having an unsafe driving condition (i.e., the anticipation of the pedestrian unexpectedly stepping into the intersection). In cases where a series of successive sensor data frames depicting driving conditions does not have associated timestamp-synchronized disengagement data, the series of successive sensor data frames can be labeled as having a safe driving condition because no human driver intervention was recorded during the time the series of successive sensor data frames was captured. Once the one or more series of successive sensor data frames of the training dataset are labeled, the training dataset can be used to train the one or more machine learning models. In some embodiments, labels provided by the training engine 136 can be further reviewed and modified by a human reviewer. For example, the training engine 136 may determine a series of successive sensor data frames as having a safe driving condition because there was no associated disengagement data. In this example, upon further review, a human reviewer may determine that the series of successive sensor data frames have an unsafe driving condition. In this example, the human reviewer may change a label for the series of successive sensor data frames as having an unsafe driving condition. In general, the training engine 136 can train the one or more machine learning models using any suitable training method. For example, in one implementation, training of the one or more machine learning models may be supervised. In other implementations, training of the one or more machine learning models may be unsupervised. Many variations are possible. The training engine 136 will be discussed in further detail with reference to FIGS. 3-10.”)
(ii) a state of an object in the environment. (see at least ZHANG, ¶ 0036)
Regarding claim 14:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
a time interval, (see at least ZHANG, ¶ 0034, “The data synchronization engine 134 can be configured to associate the disengagement data 120 and sensor data such as point cloud data frames generated by the data fusion engine 132 to indicate whether the vehicle 100 is disengaged from the autonomous mode at a time of capture of each sensor data frame. Thus, a pair of data frames, or a pair of series of data frames, can be readily juxtaposed in a manner such that one data frame or series in a first training dataset represents the vehicle 100 being disengaged from the autonomous mode and a second data frame or series in a second training dataset represents the vehicle remaining in the autonomous mode. In some embodiments, the data synchronization engine 134 can associate the disengagement data 120 and the sensor data frames by timestamp-synchronization. For example, a set of point cloud data frames can be associated with timestamps ranging from 1:00:00 to 1:10:00. In this example, data relating to a brake pedal depression action (i.e., a disengagement action) can be associated with or correspond to timestamps ranging from 1:05:00 to 1:05:30. In this example, the data relating to the brake pedal depression action can be synchronized to a subset of point cloud data frames in the set that corresponds to 1:05:00 to 1:05:30. In this way, data relating to disengagement actions a safety driver took to intervene in autonomous operations of the vehicle 100 can be synchronized to corresponding point cloud data frames to associate the disengagement actions to the corresponding point cloud data frames.”; ¶ 0037)
a suboptimal state value, and (see at least ZHANG, ¶ 0027, “The trained machine learning model may output a safety score in different simulations. A safety score can indicate a level of safety of a driving condition. In one implementation, the safety score can be binary, with zero indicating a simulation having an unsafe driving condition and one indicating a simulation having a safe driving condition. In another implementation, the safety score can range on a scale from one to ten, with one indicating a simulation having a least safe condition and ten indicating a simulation having a safest condition. Many variations are possible. Simulations that have satisfied (or exceeded) a threshold score can be classified as safe driving conditions while simulation scenarios not satisfying the threshold score can be classified as unsafe driving conditions. These simulations can be used to test embedded algorithms and/or logic in a simulation to assess and/or confirm how an autonomous vehicle would react to conditions depicted in the simulations. As previously alluded to, at least some of these simulations may be generated by portions of the captured sensor data frames so that the simulations reflect actual conditions that would be encountered by a vehicle.”)
a suboptimality type. (see at least ZHANG, ¶ 0035, “The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101. The machine learning model may include, in some embodiments, a random forest model or a neural network. In some examples, the training engine 136 feeds, into the machine learning model, pairs of training datasets juxtaposed such that a first training dataset represent, indicates, and/or is labeled as a safe situation while a second training dataset represent, indicates, and/or is labeled as an unsafe situation. In such a manner, the machine learning model may infer and/or determine differences between the training datasets to determine factors and/or parameters that cause a situation to be unsafe. In some embodiments, a dataset labeled as unsafe corresponds to a vehicle being disengaged from an autonomous mode, while a dataset labeled as safe corresponds to a vehicle remaining in the autonomous mode. One example of the juxtaposition of a first training dataset that includes data corresponding to a safe situation and a second training dataset that includes data corresponding to an unsafe situation occurs when one training dataset includes a frame or a series of frames from a particular location determined or labeled as safe and a second training dataset includes a frame or a series of frames from that same particular location determined or labeled as unsafe. As another example, one training dataset may include a frame or a series of frames from a particular range of timestamps or locations determined or labeled as safe and a second training dataset may include a frame or a series of frames from that same particular range of timestamps or locations determined or labeled as unsafe. In such a manner, the machine learning model may be able to determine or infer which parameters or factors cause a situation to be safe or unsafe, such as a density of traffic, speed or acceleration of vehicles, lighting conditions, concentration of pedestrians, particular locations such as intersections or corners, or certain sounds. By juxtaposing the training datasets, in which at least one factor, such as a time range or a location, is kept constant or similar among the pair of training datasets, such as a time range or a location, the machine learning model may determine or infer whether, and to what extent, any of the differences between the juxtaposed data frames cause a situation to be unsafe. For example, if a difference between a situation determined to be safe and another situation determined to be unsafe at a same location, and/or having similar traffic density, is that the situation determined to be safe was obtained during daytime and the other situation determined to be unsafe was at nighttime, the machine learning model may infer that a lack of lighting causes a situation to be unsafe. By using multiple juxtaposed training datasets, the machine learning model may further isolate, refine and/or confirm parameters and/or factors used to determine a level of safety of a situation. Examples of such juxtaposed training datasets are shown in FIGS. 4-10.”)
ZHANG does not disclose, but BELTA teaches:
the label comprises: (see at least BELTA, ¶ 0025, “In some cases, the inference system may determine the continuation condition is satisfied when traces of a group include traces labeled a first label (e.g., a positive example of an autonomous vehicle action) and traces labeled a second label (e.g., a negative example of the autonomous vehicle action). In some cases, the inference system may determine the continuation condition is not satisfied when traces of a group include only traces (or a threshold percentage of traces) labeled the first label.”; ¶ 0095, “In some cases, the perception system 402 may associate the trajectory data to particular autonomous vehicle actions. For instance, the perception system 402 may associate trajectory data for a trip to actions taken during the trip. In some cases, the perception system 402 may segment the trajectory data for a trip (or some sub-part thereof) and label the segments in accordance with autonomous vehicle actions taken during the trip (or the sub-part of the trip). As an example, the planning system 404 may determine a particular autonomous vehicle action (e.g., maintain course, change lanes, turn right/left onto different street, stop at stop sign, stop at stop light, yield for pedestrian, and the like) and the perception system 402 may label corresponding trajectory data with the autonomous vehicle action taken.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, to modify the detection of the driver intervention and training engine for the autonomous vehicle with binary labeling for determining if a driving activity is safe or unsafe (¶ 0005) within ZHANG to implement a more detailed label including the vehicle actions taken in performing an action within BELTA to yield a safer autonomous driving data with more granular descriptions of the vehicular actions performed that led to the respective binary labels.
Regarding claim 15:
ZHANG in view of BELTA discloses the limitations within claim 2 and ZHANG further discloses:
generating, from the log data, a positive training example for training the machine-learned model to imitate at least a recovery portion of the trajectory, (see at least ZHANG, ¶ 0026, “In some embodiments, the training datasets used to train the machine learning model may be juxtaposed such that one training dataset includes one or more frames depicting a situation determined or labeled as safe and a second training dataset includes one or more frames depicting a situation determined or labeled as unsafe.”; ¶ 0035, “The training engine 136 can be configured to generate and/or organize training datasets, and feed such training datasets, to train one or more machine learning models to detect unsafe or dangerous situations. In some embodiments, the one or more machine learning models may further be trained to generate scenarios, such as from dangerous situations, used to test algorithms and/or logic of the vehicle 101. The machine learning model may include, in some embodiments, a random forest model or a neural network. In some examples, the training engine 136 feeds, into the machine learning model, pairs of training datasets juxtaposed such that a first training dataset represent, indicates, and/or is labeled as a safe situation while a second training dataset represent, indicates, and/or is labeled as an unsafe situation. In such a manner, the machine learning model may infer and/or determine differences between the training datasets to determine factors and/or parameters that cause a situation to be unsafe. In some embodiments, a dataset labeled as unsafe corresponds to a vehicle being disengaged from an autonomous mode, while a dataset labeled as safe corresponds to a vehicle remaining in the autonomous mode. One example of the juxtaposition of a first training dataset that includes data corresponding to a safe situation and a second training dataset that includes data corresponding to an unsafe situation occurs when one training dataset includes a frame or a series of frames from a particular location determined or labeled as safe and a second training dataset includes a frame or a series of frames from that same particular location determined or labeled as unsafe. As another example, one training dataset may include a frame or a series of frames from a particular range of timestamps or locations determined or labeled as safe and a second training dataset may include a frame or a series of frames from that same particular range of timestamps or locations determined or labeled as unsafe. In such a manner, the machine learning model may be able to determine or infer which parameters or factors cause a situation to be safe or unsafe, such as a density of traffic, speed or acceleration of vehicles, lighting conditions, concentration of pedestrians, particular locations such as intersections or corners, or certain sounds. By juxtaposing the training datasets, in which at least one factor, such as a time range or a location, is kept constant or similar among the pair of training datasets, such as a time range or a location, the machine learning model may determine or infer whether, and to what extent, any of the differences between the juxtaposed data frames cause a situation to be unsafe. For example, if a difference between a situation determined to be safe and another situation determined to be unsafe at a same location, and/or having similar traffic density, is that the situation determined to be safe was obtained during daytime and the other situation determined to be unsafe was at nighttime, the machine learning model may infer that a lack of lighting causes a situation to be unsafe. By using multiple juxtaposed training datasets, the machine learning model may further isolate, refine and/or confirm parameters and/or factors used to determine a level of safety of a situation. Examples of such juxtaposed training datasets are shown in FIGS. 4-10.”)
wherein the recovery portion describes the corrective action. (see at least ZHANG, ¶ 0035; ¶ 0039 , “FIG. 5 illustrates an exemplary implementation of the computing system 130, in particular, of the training engine 136. In some embodiments, the training engine 136 may feed, into a machine learning model, a first training dataset 521 that includes one or more frames, or series of frames, having conditions identified as safe and a second training dataset 522 that includes one or more frames, or series of frames, conditions identified as unsafe. For example, the machine learning model may identify differences between the juxtaposed training datasets, specifically, between first training dataset 521 and the second training dataset 522. For example, the first training dataset 521 may be identified to include a driving situation in which two vehicles are driving straight on separate lanes while the second training dataset 522 may be identified to include a driving situation in which a vehicle is attempting to pass another, and both the first training dataset 521 and the second training dataset 522 may have been taken at a same or similar position and/or pose, and/or have similar or same distances between vehicles, in order to isolate relevant factors or parameters. Therefore, the machine learning model may infer that while passing, two vehicles need to maintain a larger distance to be considered a safe condition, compared to two vehicles driving straight on separate lanes. Thus, whether two vehicles are a safe distance apart depends on whether or not a vehicle is passing another vehicle. By using the first training dataset 521 and the second training dataset 522, the machine learning model may better identify, or refine its previous identification of, parameters or factors that cause a situation to be identified as dangerous. Thus, a vehicle may be able to stop or slow down once a situation is identified as dangerous. Other pairs of juxtaposed training datasets may depict vehicles travelling at different speeds. For example, a safe distance that two vehicles need to be separated by may be greater if one or both vehicles is/are travelling faster.”)
Regarding claim 18:
With regards to claim 18, this claim is the computing system claim to claim 1 and is substantially similar to claim 1 and is therefore rejected using the same references and rationale. The additional elements recited (“processor”, “non-transitory, computer-readable medium”) can be found in ZHANG ¶ 0053.
Regarding claim 19:
With regards to claim 18, this claim is substantially similar to claim 2 and is therefore rejected using the same references and rationale.
Regarding claim 20:
With regards to claim 20, this claim is the computer-readable media claim to claim 1 and is substantially similar to claim 1 and is therefore rejected using the same references and rationale. The additional elements recited “non-transitory, computer-readable medium”) can be found in ZHANG ¶ 0053.
Regarding claim 21:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
the one or more constraint dimensions. (see at least ZHANG, ¶ 0009, “In some embodiments, the disengagement data can comprise at least one of steering wheel rotation data, acceleration pedal depression data, or brake pedal depression data.”; ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”)
ZHANG does not disclose, but BELTA teaches:
wherein the label identifies (see at least BELTA, ¶ 0025, “In some cases, the inference system may determine the continuation condition is satisfied when traces of a group include traces labeled a first label (e.g., a positive example of an autonomous vehicle action) and traces labeled a second label (e.g., a negative example of the autonomous vehicle action). In some cases, the inference system may determine the continuation condition is not satisfied when traces of a group include only traces (or a threshold percentage of traces) labeled the first label.”; ¶ 0095, “In some cases, the perception system 402 may associate the trajectory data to particular autonomous vehicle actions. For instance, the perception system 402 may associate trajectory data for a trip to actions taken during the trip. In some cases, the perception system 402 may segment the trajectory data for a trip (or some sub-part thereof) and label the segments in accordance with autonomous vehicle actions taken during the trip (or the sub-part of the trip). As an example, the planning system 404 may determine a particular autonomous vehicle action (e.g., maintain course, change lanes, turn right/left onto different street, stop at stop sign, stop at stop light, yield for pedestrian, and the like) and the perception system 402 may label corresponding trajectory data with the autonomous vehicle action taken.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, to modify the detection of the driver intervention and training engine for the autonomous vehicle with binary labeling for determining if a driving activity is safe or unsafe (¶ 0005) within ZHANG to implement a more detailed label including the vehicle actions taken in performing an action within BELTA to yield a safer autonomous driving data with more granular descriptions of the vehicular actions performed that led to the respective binary labels.
Claims 4, 6-8, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG (US 20220297726 A1) in view of BELTA (US 20230391367 A1) in further view of DAVINS (FR3135685A1).
Regarding claim 4:
ZHANG in view of BELTA discloses the limitations within claim 2 and ZHANG further discloses:
based on one or more features of the corrective action. (see at least ZHANG, ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”)
ZHANG does not disclose, but DAVINS teaches:
determining the one or more constraint dimensions (see at least DAVINS, ¶ 0022, "Preferably, said automotive device is a vehicle which comprises at least one wheel adapted to be turned in a variable direction, at least one power steering actuator, at least one braking actuator and at least one vehicle propulsion actuator. From then on, the lateral steering instruction is transmitted to said at least one power steering actuator to steer said at least one wheel, and the longitudinal steering instruction is transmitted to said at least one braking actuator or to said at least one propulsion actuator to brake or accelerate the vehicle."; ¶ 0080, "The steering angle that the front steered wheels make with the longitudinal axis X of the motor vehicle 10 will be noted “δ<sub>f</sub>” and will be expressed in rad.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the accelerator and steering tracking for the driver intervention (disengagement) within ZANHG in view of BELTA to use the acceleration, braking, and steering data for tracking the longitudinal and lateral axis of the vehicle within DAVINS to effectively provide better detection of trajectory deviations by associating accelerator, braking, and steering with lateral and longitudinal coordinates.
Regarding claim 6:
ZHANG in view of BELTA discloses the limitations within claim 4 and ZHANG further discloses:
the corrective action comprises a braking action or an acceleration action, and the one or more constraint dimensions (see at least ZHANG, ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”; ¶ 0037, “FIG. 3 illustrates an example functional block diagram 300 for training a machine learning model, according to an embodiment of the present disclosure. Functionalities described in the functional block diagram 300 can be supported by the training engine 136 of FIG. 1B. In various embodiments, the functional block diagram 300 can include a training dataset 302. The training data set 302 can comprise sensor data frames 304 depicting a driving condition in an environment (e.g., a series of successive sensor data frames) and disengagement data 306. The sensor data frames 304 can be generated by merging (e.g., fusing) image data and LiDAR sensor data depicting the driving condition. For example, the sensor data frames 304 can comprise image data captured by camera sensors (e.g., the camera sensors 106 of FIG. 1) depicting an object or a driving condition in an environment merged with sensor data captured by a LiDAR sensor (e.g., the LiDAR sensors 102 of FIG. 1) depicting the object or the driving condition in the environment. The disengagement data 306 can include data relating to human driver interventions with autonomous operations of an autonomous vehicle (e.g., the autonomous vehicle 100 of FIG. 1). For example, the disengagement data 306 can include steering wheel rotation data, brake paddle depression data, acceleration paddle depression data, or any other data relating to human driver intervention. The sensor data frames 304 and the disengagement data 306 can be timestamp-synchronized by a data synchronization engine (e.g., the data synchronization engine 134 of FIG. 1B) at block 308 (e.g., “Data Synchronization”). The sensor data frames 304 can be labeled as having a safe or an unsafe driving condition at block 310 (e.g., “Data Labeling”). As discussed above, if the sensor data frames 304 are associated with the disengagement data 306 based on their respective timestamps, the sensor data frame 304 can be labeled as having an unsafe driving condition at block 312 (e.g., “Unsafe Condition”). If the sensor data frames 304 are not associated with the disengagement data 306 based on their respective timestamps, the sensor data frame 304 can be labeled as having a safe driving condition at block 314 (e.g., “Safe Condition”). In some cases, the sensor data frames 304 as labeled may be reviewed and modified by a human reviewer at block 316 (e.g., “Label Modification”). The sensor data frames 304 and other sensor data frames can be used to train a machine learning model 318 to detect a dangerous or unsafe, or potentially dangerous or unsafe condition, and/or revise preexisting logic and/or parameters used by the machine learning model 318.”)
the corrective action comprises a steering action, and the one or more constraint dimensions (see at least ZHANG, ¶ 0030; ¶ 0037)
ZHANG does not disclose, but DAVINS teaches:
correspond to a longitudinal motion parameter; or (see at least DAVINS, ¶ 0022, "Preferably, said automotive device is a vehicle which comprises at least one wheel adapted to be turned in a variable direction, at least one power steering actuator, at least one braking actuator and at least one vehicle propulsion actuator. From then on, the lateral steering instruction is transmitted to said at least one power steering actuator to steer said at least one wheel, and the longitudinal steering instruction is transmitted to said at least one braking actuator or to said at least one propulsion actuator to brake or accelerate the vehicle.")
correspond to a lateral motion parameter. (see at least DAVINS, ¶ 0022; ¶ 0080, "The steering angle that the front steered wheels make with the longitudinal axis X of the motor vehicle 10 will be noted “δ<sub>f</sub>” and will be expressed in rad.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the accelerator and steering tracking for the driver intervention (disengagement) within ZANHG in view of BELTA to use the acceleration, braking, and steering data for tracking the longitudinal and lateral axis of the vehicle within DAVINS to effectively provide better detection of trajectory deviations by associating accelerator, braking, and steering with lateral and longitudinal coordinates.
Regarding claim 7:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
of the suboptimality along the one or more constraint dimensions. (see at least ZHANG, ¶ 0009, “In some embodiments, the disengagement data can comprise at least one of steering wheel rotation data, acceleration pedal depression data, or brake pedal depression data.”; ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”)
ZHANG does not disclose, but BELTA teaches:
the label comprises (see at least BELTA, ¶ 0025, “In some cases, the inference system may determine the continuation condition is satisfied when traces of a group include traces labeled a first label (e.g., a positive example of an autonomous vehicle action) and traces labeled a second label (e.g., a negative example of the autonomous vehicle action). In some cases, the inference system may determine the continuation condition is not satisfied when traces of a group include only traces (or a threshold percentage of traces) labeled the first label.”; ¶ 0095, “In some cases, the perception system 402 may associate the trajectory data to particular autonomous vehicle actions. For instance, the perception system 402 may associate trajectory data for a trip to actions taken during the trip. In some cases, the perception system 402 may segment the trajectory data for a trip (or some sub-part thereof) and label the segments in accordance with autonomous vehicle actions taken during the trip (or the sub-part of the trip). As an example, the planning system 404 may determine a particular autonomous vehicle action (e.g., maintain course, change lanes, turn right/left onto different street, stop at stop sign, stop at stop light, yield for pedestrian, and the like) and the perception system 402 may label corresponding trajectory data with the autonomous vehicle action taken.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, to modify the detection of the driver intervention and training engine for the autonomous vehicle with binary labeling for determining if a driving activity is safe or unsafe (¶ 0005) within ZHANG to implement a more detailed label including the vehicle actions taken in performing an action within BELTA to yield a safer autonomous driving data with more granular descriptions of the vehicular actions performed that led to the respective binary labels.
ZHANG does not disclose, but DAVINS teaches:
a direction characteristic that describes a direction (see at least DAVINS, ¶ 0080, "The steering angle that the front steered wheels make with the longitudinal axis X of the motor vehicle 10 will be noted “δ<sub>f</sub>” and will be expressed in rad."; ¶ 0022, "Preferably, said automotive device is a vehicle which comprises at least one wheel adapted to be turned in a variable direction, at least one power steering actuator, at least one braking actuator and at least one vehicle propulsion actuator. From then on, the lateral steering instruction is transmitted to said at least one power steering actuator to steer said at least one wheel, and the longitudinal steering instruction is transmitted to said at least one braking actuator or to said at least one propulsion actuator to brake or accelerate the vehicle."; ¶ 0114, “The dynamic parameters of the vehicle taken into account, i.e. the parameters characterizing its movements, are as follows: - the longitudinal translational movement of the vehicle body along the X axis, - the lateral translational movement of the vehicle body along the Y axis, - the yaw movement of the vehicle body around the Z axis, - the rolling movement of the front wheel around its axis of rotation Y<sub>w</sub><sub>f</sub>, and - the rolling movement of the rear wheel around its axis of rotation Y<sub>w</sub><sub>r</sub>.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the accelerator and steering tracking for precisely labeling unsafe conditions such as driver intervention (disengagement) within ZANHG in view of BELTA to use the acceleration, braking, and steering data for tracking the longitudinal and lateral axis of the vehicle within DAVINS to effectively provide better labeling of driver intervention data by correlating acceleration and steering with lateral and longitudinal directions.
Regarding claim 8:
ZHANG in view of BELTA in further view of DAVINS discloses the limitations within claim 7 and ZHANG further discloses:
of a corrective action initiated by a human operator. (see at least ZHANG, ¶ 0009, “In some embodiments, the disengagement data can comprise at least one of steering wheel rotation data, acceleration pedal depression data, or brake pedal depression data.”; ¶ 0030, “FIG. 1B illustrates a hybrid data flow and block diagram that depicts a computing system 130 configured to obtain and/or process sensor data from the autonomous vehicle 100, according to an embodiment of the present disclosure. The computing system 130 may be integrated as part of the vehicle 100, or may be remote from the vehicle 100. As shown in FIG. 1B, in some embodiments, the computing system 130 can be communicatively coupled to the autonomous vehicle 100 while the autonomous vehicle 100 is in a non-active resting state. The computer system 130 can be configured to obtain or download sensor data from the autonomous vehicle 100 through one or more data links or buses 140. In some embodiments, the computing system 130 can include a data fusion engine 132, a data synchronization engine 134, and a simulation scenario generation engine 136. In general, functions described with respect to the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be performed by a single processor or any number of processors. For example, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may be implemented using multiple processors. In some cases, functions performed by the data fusion engine 132, the data synchronization engine 134, and the simulation scenario generation engine 136 may not be spatially separated across multiple processors; instead, the functions must be performed by a common processor. As shown in FIG. 1B, in some embodiments, the sensor data of the autonomous vehicle 100 can include disengagement data 120, two-dimensional (2D) data 122, and three-dimensional (3D) data 124. The disengagement data 120 can comprise data indicating whether the vehicle 100 has been disengaged from an autonomous mode, and may include data relating to human driver interventions to autonomous operations of the autonomous vehicle 100. For example, the vehicle 100 may decide to change lanes based on a road condition. A safety driver inside the vehicle 100 may determine that a safer course of action is for the vehicle 100 to stay within the current lane. Therefore, in this example, the safety driver can intervene to take control of the vehicle 100 and prevent the vehicle 100 from changing lanes. In this example, the disengagement data 120 can include, for example, depressing a brake pedal and steering a steering wheel. In various embodiments, the disengagement data 120 can include steering wheel rotation data, acceleration pedal depression data, brake pedal depression data, or any other data relating to human driver intervention. The 2D data 122 can be captured from the camera sensors 106 or the LiDAR sensor 102 and can comprise 2D data frames (e.g., image data or point cloud data). The 3D data 124 can be captured from the LiDAR sensor 102 and can comprise 3D data frames (e.g., LiDAR point cloud data).”))
ZHANG does not disclose, but DAVINS teaches:
the direction characteristic is determined based on a direction (see at least DAVINS, ¶ 0080, "The steering angle that the front steered wheels make with the longitudinal axis X of the motor vehicle 10 will be noted “δ<sub>f</sub>” and will be expressed in rad."; ¶ 0022, "Preferably, said automotive device is a vehicle which comprises at least one wheel adapted to be turned in a variable direction, at least one power steering actuator, at least one braking actuator and at least one vehicle propulsion actuator. From then on, the lateral steering instruction is transmitted to said at least one power steering actuator to steer said at least one wheel, and the longitudinal steering instruction is transmitted to said at least one braking actuator or to said at least one propulsion actuator to brake or accelerate the vehicle."; ¶ 0114, “The dynamic parameters of the vehicle taken into account, i.e. the parameters characterizing its movements, are as follows: - the longitudinal translational movement of the vehicle body along the X axis, - the lateral translational movement of the vehicle body along the Y axis, - the yaw movement of the vehicle body around the Z axis, - the rolling movement of the front wheel around its axis of rotation Y<sub>w</sub><sub>f</sub>, and - the rolling movement of the rear wheel around its axis of rotation Y<sub>w</sub><sub>r</sub>.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the accelerator and steering tracking for labeling unsafe conditions such as driver intervention (disengagement) within ZANHG to use the acceleration, braking, and steering data for tracking the longitudinal and lateral axis of the vehicle within DAVINS to effectively provide better tracking of driver intervention data by correlating acceleration and steering with lateral and longitudinal directions.
Regarding claim 16:
ZHANG in view of BELTA discloses the limitations within claim 1 and ZHANG further discloses:
the suboptimal condition (see at least ZHANG, ¶ 0037, “FIG. 3 illustrates an example functional block diagram 300 for training a machine learning model, according to an embodiment of the present disclosure. Functionalities described in the functional block diagram 300 can be supported by the training engine 136 of FIG. 1B. In various embodiments, the functional block diagram 300 can include a training dataset 302. The training data set 302 can comprise sensor data frames 304 depicting a driving condition in an environment (e.g., a series of successive sensor data frames) and disengagement data 306. The sensor data frames 304 can be generated by merging (e.g., fusing) image data and LiDAR sensor data depicting the driving condition. For example, the sensor data frames 304 can comprise image data captured by camera sensors (e.g., the camera sensors 106 of FIG. 1) depicting an object or a driving condition in an environment merged with sensor data captured by a LiDAR sensor (e.g., the LiDAR sensors 102 of FIG. 1) depicting the object or the driving condition in the environment. The disengagement data 306 can include data relating to human driver interventions with autonomous operations of an autonomous vehicle (e.g., the autonomous vehicle 100 of FIG. 1). For example, the disengagement data 306 can include steering wheel rotation data, brake paddle depression data, acceleration paddle depression data, or any other data relating to human driver intervention. The sensor data frames 304 and the disengagement data 306 can be timestamp-synchronized by a data synchronization engine (e.g., the data synchronization engine 134 of FIG. 1B) at block 308 (e.g., “Data Synchronization”). The sensor data frames 304 can be labeled as having a safe or an unsafe driving condition at block 310 (e.g., “Data Labeling”). As discussed above, if the sensor data frames 304 are associated with the disengagement data 306 based on their respective timestamps, the sensor data frame 304 can be labeled as having an unsafe driving condition at block 312 (e.g., “Unsafe Condition”). If the sensor data frames 304 are not associated with the disengagement data 306 based on their respective timestamps, the sensor data frame 304 can be labeled as having a safe driving condition at block 314 (e.g., “Safe Condition”). In some cases, the sensor data frames 304 as labeled may be reviewed and modified by a human reviewer at block 316 (e.g., “Label Modification”). The sensor data frames 304 and other sensor data frames can be used to train a machine learning model 318 to detect a dangerous or unsafe, or potentially dangerous or unsafe condition, and/or revise preexisting logic and/or parameters used by the machine learning model 318.”)
joined by one or more Boolean operators. (see at least ZHANG, ¶ 0041, “FIG. 7 illustrates an exemplary implementation of the computing system 130, in particular, of the training engine 136. In some embodiments, the training engine 136 may feed, into a machine learning model, a first training dataset 721 that includes one or more frames, or series of frames, having conditions identified as safe and a second training dataset 722 that includes one or more frames, or series of frames, conditions identified as unsafe. For example, the machine learning model may identify differences between the juxtaposed training datasets, specifically, between first training dataset 721 and the second training dataset 722. For example, the first training dataset 721 may include contextual factors that, individually, would not render a situation dangerous, but the second training dataset 722 may include a combination of contextual factors that would render a situation dangerous. For example, the first training dataset 721 may be identified to include an object on a road such as a ball, or a person standing on a side of the road while the second training dataset 722 may be identified to have both an object on a road and a person standing on a side of the road, and both the first training dataset 721 and the second training dataset 722 may have been taken at a same or similar position and/or pose, in order to isolate relevant factors or parameters. Therefore, the machine learning model may infer that an object on a road and a person standing on a side of the road within a threshold distance, taken in context, may render a situation dangerous, even though each factor alone (just an object, or just a person standing on a side of the road) would not render the situation dangerous. By using the first training dataset 721 and the second training dataset 722, the machine learning model may better identify, or refine its previous identification of, parameters or factors that cause a situation to be identified as dangerous. Thus, a vehicle may be able to stop or slow down once a situation is identified as dangerous. Other pairs of juxtaposed training datasets may depict different combinations of contextual factors, such as, for example, a vehicle driving parallel to two trucks, or a truck and a bicycle, even if driving parallel to a single truck or a single bicycle does not render a situation dangerous. Thus, the machine learning model may identify or determine whether certain combinations of factors affect an identification or determination of whether a situation is dangerous.”)
ZHANG does not disclose, but BELTA teaches:
the label characterizes (see at least BELTA, ¶ 0025, “In some cases, the inference system may determine the continuation condition is satisfied when traces of a group include traces labeled a first label (e.g., a positive example of an autonomous vehicle action) and traces labeled a second label (e.g., a negative example of the autonomous vehicle action). In some cases, the inference system may determine the continuation condition is not satisfied when traces of a group include only traces (or a threshold percentage of traces) labeled the first label.”; ¶ 0095, “In some cases, the perception system 402 may associate the trajectory data to particular autonomous vehicle actions. For instance, the perception system 402 may associate trajectory data for a trip to actions taken during the trip. In some cases, the perception system 402 may segment the trajectory data for a trip (or some sub-part thereof) and label the segments in accordance with autonomous vehicle actions taken during the trip (or the sub-part of the trip). As an example, the planning system 404 may determine a particular autonomous vehicle action (e.g., maintain course, change lanes, turn right/left onto different street, stop at stop sign, stop at stop light, yield for pedestrian, and the like) and the perception system 402 may label corresponding trajectory data with the autonomous vehicle action taken.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, to modify the detection of the driver intervention and training engine for the autonomous vehicle with binary labeling for determining if a driving activity is safe or unsafe (¶ 0005) within ZHANG to implement a more detailed label including the vehicle actions taken in performing an action within BELTA to yield a safer autonomous driving data with more granular descriptions of the vehicular actions performed that led to the respective binary labels.
ZHANG does not disclose, but DAVINS teaches:
along a plurality of constraint dimensions (see at least DAVINS, ¶ 0022, "Preferably, said automotive device is a vehicle which comprises at least one wheel adapted to be turned in a variable direction, at least one power steering actuator, at least one braking actuator and at least one vehicle propulsion actuator. From then on, the lateral steering instruction is transmitted to said at least one power steering actuator to steer said at least one wheel, and the longitudinal steering instruction is transmitted to said at least one braking actuator or to said at least one propulsion actuator to brake or accelerate the vehicle."; ¶ 0080, "The steering angle that the front steered wheels make with the longitudinal axis X of the motor vehicle 10 will be noted “δ<sub>f</sub>” and will be expressed in rad.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the collection of steering and acceleration data to determine unsafe conditions for precise labeling within ZHANG in view of BELTA to include the calculation of lateral and longitudinal coordinates from the steering and acceleration of the vehicle within DAVINS to yield a more efficient training data labeling system wherein vehicle processed displacement coordinates are collected for labeling rather than bare sensor data.
Regarding claim 17:
ZHANG in view of BELTA in further view of DAVINS discloses the limitations within claim 16 and ZHANG further discloses:
the suboptimal condition corresponds to (see at least ZHANG, ¶ 0037, “FIG. 3 illustrates an example functional block diagram 300 for training a machine learning model, according to an embodiment of the present disclosure. Functionalities described in the functional block diagram 300 can be supported by the training engine 136 of FIG. 1B. In various embodiments, the functional block diagram 300 can include a training dataset 302. The training data set 302 can comprise sensor data frames 304 depicting a driving condition in an environment (e.g., a series of successive sensor data frames) and disengagement data 306. The sensor data frames 304 can be generated by merging (e.g., fusing) image data and LiDAR sensor data depicting the driving condition. For example, the sensor data frames 304 can comprise image data captured by camera sensors (e.g., the camera sensors 106 of FIG. 1) depicting an object or a driving condition in an environment merged with sensor data captured by a LiDAR sensor (e.g., the LiDAR sensors 102 of FIG. 1) depicting the object or the driving condition in the environment. The disengagement data 306 can include data relating to human driver interventions with autonomous operations of an autonomous vehicle (e.g., the autonomous vehicle 100 of FIG. 1). For example, the disengagement data 306 can include steering wheel rotation data, brake paddle depression data, acceleration paddle depression data, or any other data relating to human driver intervention. The sensor data frames 304 and the disengagement data 306 can be timestamp-synchronized by a data synchronization engine (e.g., the data synchronization engine 134 of FIG. 1B) at block 308 (e.g., “Data Synchronization”). The sensor data frames 304 can be labeled as having a safe or an unsafe driving condition at block 310 (e.g., “Data Labeling”). As discussed above, if the sensor data frames 304 are associated with the disengagement data 306 based on their respective timestamps, the sensor data frame 304 can be labeled as having an unsafe driving condition at block 312 (e.g., “Unsafe Condition”). If the sensor data frames 304 are not associated with the disengagement data 306 based on their respective timestamps, the sensor data frame 304 can be labeled as having a safe driving condition at block 314 (e.g., “Safe Condition”). In some cases, the sensor data frames 304 as labeled may be reviewed and modified by a human reviewer at block 316 (e.g., “Label Modification”). The sensor data frames 304 and other sensor data frames can be used to train a machine learning model 318 to detect a dangerous or unsafe, or potentially dangerous or unsafe condition, and/or revise preexisting logic and/or parameters used by the machine learning model 318.”)
ZHANG does not disclose, but DAVINS teaches:
a magnitude of first parameter value along a first constraint dimension in combination with a magnitude of a second parameter value along a second constraint dimension. (see at least DAVINS, ¶ 0099, “The parameters used to describe the trajectory that the vehicle must follow are here: - the longitudinal position of the vehicle (whose nominal value is noted x<sub>0</sub>and the current value x), - the lateral position of the vehicle (whose nominal value is noted y<sub>0</sub>and the current value y), - the heading angle of the vehicle (whose nominal value is noted Ψ<sub>0</sub>and the current value Ψ), - the longitudinal speed v of the vehicle (whose nominal value is noted v<sub>0</sub>and the current value v), - the lateral speed u of the vehicle (whose nominal value is noted u<sub>0</sub>and the current value u), - the yaw rate r of the vehicle (whose nominal value is noted r<sub>0</sub>and the current value r).”; ¶ 0192, “Thus, the Math24 equations make it possible to take into account the dynamic characteristics of the vehicle, while the Math25 equations make it possible to impose good trajectory tracking on the vehicle. This combination of equations then ensures good control of the vehicle along the reference trajectory T0.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the collection of steering and acceleration data to determine unsafe conditions within ZHANG to include the calculation of lateral and longitudinal coordinates from the steering and acceleration to generate trajectory of the vehicle within DAVINS to effectively yield a description of the trajectory of the vehicle when the unsafe situation occurs.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over ZHANG (US 20220297726 A1) in view of BELTA (US 20230391367 A1) in further view of CHENG (CN 116872971 A).
Regarding claim 5:
ZHANG in view of BELTA discloses the limitations within claim 2 and ZHANG does not disclose, but CHENG teaches:
the suboptimal condition is characterized based on a magnitude of a change associated with the corrective action. (see at least CHENG, ¶ 0048, "The control decision model is trained based on the mixed data, and corresponding weights are introduced when the supervised driver takes over the control, so as to obtain the control strategy of autonomous driving according to the trained control decision model."; ¶ 0054, "The driver's supervision and correction actions are: when the actions performed by the intelligent vehicle do not meet expectations, the supervising driver can interrupt the intelligent vehicle's exploration online and give corresponding correct driving actions, including steering wheel turning and braking force."; ¶ 0084, "a discriminant module configured to predict driving simulation data under the current supervised corrective action based on the supervised corrective action, and score the current supervised corrective action to determine the weight of the difference between the driving data after taking over control with the current supervised corrective action and the vehicle's own driving data when training the control decision model;")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the detection of the driver intervention and training engine for the autonomous vehicle within ZHANG to give scores and weights to the driver correction for training within CHENG to effectively yield a ML trainer that trains based on weights assigned to specific driver actions.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAFAEL VELASQUEZ VANEGAS whose telephone number is (571)272-6999. The examiner can normally be reached M-F 9 - 4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, RACHID BENDIDI can be reached at (571) 272-4896. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAFAEL VELASQUEZ VANEGAS/Patent Examiner, Art Unit 3664
/JOAN T GOODBODY/Examiner, Art Unit 3664
Prosecution Insights