DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
The claims 1-20 are currently pending and have been examined. Applicant amended claims 1, 4, 5, 7, 8, 10, 12, 15, 16, 18, and 19.
Response to Arguments/Amendments
The amendment filed May 29, 2026 has been entered. Claims 1-20 are currently pending in the Application.
Applicant’s arguments with respect to claims 1-20 under 35 U.S.C. 102 and 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 2, 7-13, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over AKASH ‘490 (US 20220324490 A1) in view of MA (CN 116975671 A).
Regarding Claim 1, AKASH ‘490 teaches A prediction system comprising: a memory storing instructions that, when executed by a processor, cause the processor to: detect characteristics about a driving scenario and an operator from acquired sensor data and an operator factor (See at least paragraph [0006], “According to another aspect, a system for providing an RNN-based human trust model that includes a memory storing instructions when executed by a processor cause the processor to receive a plurality of inputs related to an autonomous operation of a vehicle and a driving scene of the vehicle and analyze the plurality of inputs to determine automation variables and scene variables”, paragraph [0031], “FIG. 2 is an exemplary schematic general overview of a trust model provided by the trust model application 106 according to an exemplary embodiment of the present disclosure. In an exemplary embodiment, the trust model provided by the trust model application 106 may be determined based on the driver's reliance on the semi-autonomous/autonomous operation of the vehicle 102. The trust model may receive inputs from one or more systems, sensors, and/or components of the vehicle 102 and from crowdsourced survey data that may be completed by the individuals and/or past operation of the vehicle 102”, and paragraph [0033], “The automation variables and scene variables of the trust model that may be provided by inputs from one or more systems, sensors, and/or components of the vehicle 102 may include a level of automation transparency 204, a level of automation reliability 206, a level of risk 208, a level of scene difficulty 210, and a previous take-over intent 212 of the driver of the vehicle 102. The level of automation transparency 204 that is provided to the driver of the vehicle 102 may be determined by a number of and a description of one or more augmented reality cues that may be presented to the driver of the vehicle 102 during autonomous or semi-autonomous operation of the vehicle 102.”); predict dissonance for an automated takeover that reduces a manual control by the operator using a learning model with the characteristics, a driving command, and a cue about the operator (See at least paragraph [0006], “According to another aspect, a system for providing an RNN-based human trust model that includes a memory storing instructions when executed by a processor cause the processor to receive a plurality of inputs related to an autonomous operation of a vehicle and a driving scene of the vehicle and analyze the plurality of inputs to determine automation variables and scene variables. Crowd-sourced data associated with surveys that pertain to a driver's self-reported trust and the driver's self-reported reliability with respect to the autonomous operation of the vehicle is collected and analyzed. The instructions also cause the processor to output a short-term trust recurrent neural network state that captures an effect of the driver's experience with respect to an instantaneous vehicle maneuver and a long-term trust recurrent neural network state that captures the effect of the driver's experience with respect to the autonomous operation of the vehicle during a traffic scenario based on the automation variables, the scene variables, and the crowd-sourced data. The instructions further cause the processor to predict a take-over intent of the driver to take over control of the vehicle from an automated operation of the vehicle during the traffic scenario based on the short-term trust recurrent neural network state and the long-term trust recurrent neural network state”, paragraph [0070], “The trust model application 106 may also analyze the body movements of the driver with respect to the movement of the driver's arms, hands, legs, feet, and torso. The trust model application 106 may utilize virtually any method to determine the body movements of the driver. In one embodiment, the trust model application 106 may analyze the driver's body to determine movements based off of a linear model that may consider the evaluation of the specific areas of the body of the driver of the vehicle 102 as the vehicle 102 is being operated. For example, the trust model application 106 may discretize the driver's body movements at any time belonging to one of a plurality of values pertaining to one or more components of the vehicle 102”, and paragraph [0132], “With continued reference to the method 600 of FIG. 6, the method 600 may proceed to block 606, wherein the method 600 may include predicting a take-over intent 202 of the driver of the vehicle 102. In one embodiment, the RNN 108 may analyze the current traffic scenario based on current dynamic data, image data, and/or LiDAR data provided by the dynamic sensors 116, camera system 118, and/or laser projection system 120 of the vehicle 102 in addition to the short-term trust RNN state 218 and the long-term trust RNN state 220 to predict the take-over intent 202 of the driver of the vehicle 102. The take-over intent 202 may be predicted as the intent to manually take-over operation of one or more functions of the vehicle 102 in the current traffic scenario as the vehicle 102 is being autonomously or semi-autonomously operated.”); and adapt a shared-driving model (SDM) associated with a vehicle during a maneuver using the dissonance (See at least paragraph [0006], “According to another aspect, a system for providing an RNN-based human trust model that includes a memory storing instructions when executed by a processor cause the processor to receive a plurality of inputs related to an autonomous operation of a vehicle and a driving scene of the vehicle and analyze the plurality of inputs to determine automation variables and scene variables. Crowd-sourced data associated with surveys that pertain to a driver's self-reported trust and the driver's self-reported reliability with respect to the autonomous operation of the vehicle is collected and analyzed. The instructions also cause the processor to output a short-term trust recurrent neural network state that captures an effect of the driver's experience with respect to an instantaneous vehicle maneuver and a long-term trust recurrent neural network state that captures the effect of the driver's experience with respect to the autonomous operation of the vehicle during a traffic scenario based on the automation variables, the scene variables, and the crowd-sourced data. The instructions further cause the processor to predict a take-over intent of the driver to take over control of the vehicle from an automated operation of the vehicle during the traffic scenario based on the short-term trust recurrent neural network state and the long-term trust recurrent neural network state”, paragraph [0132], “With continued reference to the method 600 of FIG. 6, the method 600 may proceed to block 606, wherein the method 600 may include predicting a take-over intent 202 of the driver of the vehicle 102. In one embodiment, the RNN 108 may analyze the current traffic scenario based on current dynamic data, image data, and/or LiDAR data provided by the dynamic sensors 116, camera system 118, and/or laser projection system 120 of the vehicle 102 in addition to the short-term trust RNN state 218 and the long-term trust RNN state 220 to predict the take-over intent 202 of the driver of the vehicle 102. The take-over intent 202 may be predicted as the intent to manually take-over operation of one or more functions of the vehicle 102 in the current traffic scenario as the vehicle 102 is being autonomously or semi-autonomously operated”, paragraph [0134], “The method 600 may proceed to block 608, wherein the method 600 may include controlling one or more systems of the vehicle 102 to operate the vehicle 102 based on the predicted take-over intent 202 of the driver of the vehicle 102”, and paragraph [0135], “In particular, the vehicle control module 308 may be configured to communicate with the vehicle autonomous controller 110 to autonomously control one or more driving functions of the vehicle 102 based on the predicted take-over intent 202 of the driver of the vehicle 102. In one embodiment, if the take-over intent 202 is predicted to be high with respect to one or more functions of the vehicle 102 (e.g., braking, steering, accelerating), the vehicle control module 308 may be configured to communicate with the vehicle autonomous controller 110 to provide a particular level of automation control of one or more systems of the vehicle 102 that may provide respective functions.” The system controls a level of automation control of vehicle systems based on predicted take-over intent and adjusts the allocation of control between the automated driving system and the human driver, which corresponds to adapting a shared-driving model associated with the vehicle during the maneuver.).
AKASH ‘490 does not explicitly disclose, however, MA, in the same field of endeavor teaches wherein the dissonance is associated with a confidence metric by the operator for the driving scenario (See at least paragraph [n0018], “The application of the classic Kalman filter-based continuous state estimation method to assess trust level is beneficial because the continuous output metric of the estimator helps to build a takeover performance prediction model that takes trust level into account, laying the foundation for the design of application controllers and decision-making algorithms. By weakening the potential influence related to the randomness of driver behavior through the Kalman filter with repeatability performance, the real-time and accurate assessment of driver trust level can be achieved”, paragraph [n0026], “The implementation of a Kalman filter requires defining observation variables that can be measured and processed in real time. These observation variables must be related to the variable to be evaluated (i.e., trust level). Therefore, the frequency hi when the driver's hand position is H1 and the frequency fi when the driver's foot position is F1 are defined as the observation variables in the Kalman filter. Combining subjective trust level and the Boolean value of the warning type, an LTI system state-space model is used to represent the driver's dynamic trust level during the experiment”, and paragraph [n0083], “The K-means clustering method was used to classify drivers' subjective trust in the autonomous driving system into different levels. K-means clustering is a commonly used unsupervised clustering method that divides n confidence assessment results (T1, T2, ..., Tn) into k clusters based on minimizing the intracluster variance. Each confidence assessment result belongs to the nearest cluster center (the average value of each cluster). The steps of the K-means algorithm are as follows: (1) Determine the number of clusters; (2) Select an initial cluster center for each cluster; (3) Assign the data in the dataset to the nearest neighbor cluster according to the minimum distance principle; (4) Calculate the mean confidence of each cluster and update the cluster center; (5) Repeat the assignment of categories and update the cluster center until the category to which each sample belongs no longer changes; (6) Output the final cluster center and k cluster divisions.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine the invention of AKASH ‘490 with the teachings of MA such that the driver trust prediction system of AKASH ‘490 is further configured such that the dissonance is associated with a confidence metric by the operator for the driving scenario, as taught by MA (See paragraph [n0018], [n0026], [n0083].), with a reasonable expectation of success. The motivation for doing so would be to assess driver trust in autonomous vehicles to provide a reference for the objective and real-time assessment of driver trust in intelligent transportation environments, as taught by MA (See paragraph [n0003].).
With respect to claim 10, please see the rejection above with respect to claim 1, which is commensurate in scope to claim 10, with claim 1 being drawn to a prediction system and claim 10 being drawn to a corresponding non-transitory computer-readable medium.
With respect to claim 12, please see the rejection above with respect to claim 1, which is commensurate in scope to claim 12, with claim 1 being drawn to a prediction system and claim 12 being drawn to a corresponding method.
Regarding Claim 2, AKASH ‘490 and MA teach The prediction system of claim 1, as set forth in the obviousness rejection above. AKASH ‘490 teaches wherein the instructions to predict the dissonance further include instructions to: derive the cue by the SDM using the sensor data, wherein the cue includes one of a physical cue and a verbal cue associated with the operator for the driving command (See at least paragraph [0070], “The trust model application 106 may also analyze the body movements of the driver with respect to the movement of the driver's arms, hands, legs, feet, and torso. The trust model application 106 may utilize virtually any method to determine the body movements of the driver. In one embodiment, the trust model application 106 may analyze the driver's body to determine movements based off of a linear model that may consider the evaluation of the specific areas of the body of the driver of the vehicle 102 as the vehicle 102 is being operated. For example, the trust model application 106 may discretize the driver's body movements at any time belonging to one of a plurality of values pertaining to one or more components of the vehicle 102” and paragraph [0132], “With continued reference to the method 600 of FIG. 6, the method 600 may proceed to block 606, wherein the method 600 may include predicting a take-over intent 202 of the driver of the vehicle 102. In one embodiment, the RNN 108 may analyze the current traffic scenario based on current dynamic data, image data, and/or LiDAR data provided by the dynamic sensors 116, camera system 118, and/or laser projection system 120 of the vehicle 102 in addition to the short-term trust RNN state 218 and the long-term trust RNN state 220 to predict the take-over intent 202 of the driver of the vehicle 102. The take-over intent 202 may be predicted as the intent to manually take-over operation of one or more functions of the vehicle 102 in the current traffic scenario as the vehicle 102 is being autonomously or semi-autonomously operated.”); and receive by the learning model the driving command and the cue from the SDM (See at least paragraph [0133], “The take-over intent 202 may be output as intent data that pertains to a level of intent to manually take over one or more functions of the vehicle 102. For example, the take-over intent 202 may include an intent to take over braking or not take over braking of the vehicle 102 that is being autonomously operated at a traffic intersection with crossing pedestrians as the vehicle 102 is approaching the traffic intersection. In one embodiment, the RNN 108 may output the intent data that pertains to the level of intent to manually take over one or more functions of the vehicle 102 to the intent prediction module 306 of the trust model application 106. Accordingly, the trust model application 106 utilizes the RNN 108 to complete processing of the trust model as a logistic regression model that considers scene difficultly, automation transparency, driving scene risk, and automation reliability along with the take-over intent of the driver at a previous point in time in a similar traffic scenario as input to predict take-over intent 202.” The trust model application derives cues about the operator from sensor data, such as driver body movements, and provides these inputs to the RNN learning model used to predict driver takeover intent.).
With respect to claim 11, please see the rejection above with respect to claim 2, which is commensurate in scope to claim 11, with claim 2 being drawn to a prediction system and claim 11 being drawn to a corresponding non-transitory computer-readable medium.
With respect to claim 13, please see the rejection above with respect to claim 2, which is commensurate in scope to claim 13, with claim 2 being drawn to a prediction system and claim 13 being drawn to a corresponding method.
Regarding Claim 7, AKASH ‘490 and MA teach The prediction system of claim 1, as set forth in the obviousness rejection above. AKASH ‘490 teaches wherein the dissonance indicates that the operator will counter the driving command with the manual control, and the driving command is one of a steering command, a braking command, and a throttle command (See at least paragraph [0132], “With continued reference to the method 600 of FIG. 6, the method 600 may proceed to block 606, wherein the method 600 may include predicting a take-over intent 202 of the driver of the vehicle 102. In one embodiment, the RNN 108 may analyze the current traffic scenario based on current dynamic data, image data, and/or LiDAR data provided by the dynamic sensors 116, camera system 118, and/or laser projection system 120 of the vehicle 102 in addition to the short-term trust RNN state 218 and the long-term trust RNN state 220 to predict the take-over intent 202 of the driver of the vehicle 102. The take-over intent 202 may be predicted as the intent to manually take-over operation of one or more functions of the vehicle 102 in the current traffic scenario as the vehicle 102 is being autonomously or semi-autonomously operated”, paragraph [0133], “The take-over intent 202 may be output as intent data that pertains to a level of intent to manually take over one or more functions of the vehicle 102. For example, the take-over intent 202 may include an intent to take over braking or not take over braking of the vehicle 102 that is being autonomously operated at a traffic intersection with crossing pedestrians as the vehicle 102 is approaching the traffic intersection. In one embodiment, the RNN 108 may output the intent data that pertains to the level of intent to manually take over one or more functions of the vehicle 102 to the intent prediction module 306 of the trust model application 106. Accordingly, the trust model application 106 utilizes the RNN 108 to complete processing of the trust model as a logistic regression model that considers scene difficultly, automation transparency, driving scene risk, and automation reliability along with the take-over intent of the driver at a previous point in time in a similar traffic scenario as input to predict take-over intent 202”, and paragraph [0135], “In particular, the vehicle control module 308 may be configured to communicate with the vehicle autonomous controller 110 to autonomously control one or more driving functions of the vehicle 102 based on the predicted take-over intent 202 of the driver of the vehicle 102. In one embodiment, if the take-over intent 202 is predicted to be high with respect to one or more functions of the vehicle 102 (e.g., braking, steering, accelerating), the vehicle control module 308 may be configured to communicate with the vehicle autonomous controller 110 to provide a particular level of automation control of one or more systems of the vehicle 102 that may provide respective functions.” The system predicts driver take-over intent corresponding to determining that the operator may disagree with or counter an automated driving command, such as braking, steering, or accelerating.).
With respect to claim 18, please see the rejection above with respect to claim 7, which is commensurate in scope to claim 18, with claim 7 being drawn to a prediction system and claim 18 being drawn to a corresponding method.
Regarding Claim 8, AKASH ‘490 and MA teach The prediction system of claim 1, as set forth in the obviousness rejection above. AKASH ‘490 teaches wherein the SDM is one of a model prediction control (MPC) system, a data-driven system that is trained, an automated driving system (ADS), and a shared-decision making model, and the learning model is a neural network (NN) (See at least paragraph [0006], “According to another aspect, a system for providing an RNN-based human trust model that includes a memory storing instructions when executed by a processor cause the processor to receive a plurality of inputs related to an autonomous operation of a vehicle and a driving scene of the vehicle and analyze the plurality of inputs to determine automation variables and scene variables. Crowd-sourced data associated with surveys that pertain to a driver's self-reported trust and the driver's self-reported reliability with respect to the autonomous operation of the vehicle is collected and analyzed. The instructions also cause the processor to output a short-term trust recurrent neural network state that captures an effect of the driver's experience with respect to an instantaneous vehicle maneuver and a long-term trust recurrent neural network state that captures the effect of the driver's experience with respect to the autonomous operation of the vehicle during a traffic scenario based on the automation variables, the scene variables, and the crowd-sourced data. The instructions further cause the processor to predict a take-over intent of the driver to take over control of the vehicle from an automated operation of the vehicle during the traffic scenario based on the short-term trust recurrent neural network state and the long-term trust recurrent neural network state” and paragraph [0132], “With continued reference to the method 600 of FIG. 6, the method 600 may proceed to block 606, wherein the method 600 may include predicting a take-over intent 202 of the driver of the vehicle 102. In one embodiment, the RNN 108 may analyze the current traffic scenario based on current dynamic data, image data, and/or LiDAR data provided by the dynamic sensors 116, camera system 118, and/or laser projection system 120 of the vehicle 102 in addition to the short-term trust RNN state 218 and the long-term trust RNN state 220 to predict the take-over intent 202 of the driver of the vehicle 102. The take-over intent 202 may be predicted as the intent to manually take-over operation of one or more functions of the vehicle 102 in the current traffic scenario as the vehicle 102 is being autonomously or semi-autonomously operated.”).
With respect to claim 19, please see the rejection above with respect to claim 8, which is commensurate in scope to claim 19, with claim 8 being drawn to a prediction system and claim 19 being drawn to a corresponding method.
Regarding Claim 9, AKASH ‘490 and MA teach The prediction system of claim 1, as set forth in the obviousness rejection above. AKASH ‘490 teaches wherein: the sensor data includes scene information about an environment associated with the vehicle, and the sensor data includes one of steering, braking, and throttle information by the operator (See at least paragraph [0132], “With continued reference to the method 600 of FIG. 6, the method 600 may proceed to block 606, wherein the method 600 may include predicting a take-over intent 202 of the driver of the vehicle 102. In one embodiment, the RNN 108 may analyze the current traffic scenario based on current dynamic data, image data, and/or LiDAR data provided by the dynamic sensors 116, camera system 118, and/or laser projection system 120 of the vehicle 102 in addition to the short-term trust RNN state 218 and the long-term trust RNN state 220 to predict the take-over intent 202 of the driver of the vehicle 102. The take-over intent 202 may be predicted as the intent to manually take-over operation of one or more functions of the vehicle 102 in the current traffic scenario as the vehicle 102 is being autonomously or semi-autonomously operated”, paragraph [0133], “The take-over intent 202 may be output as intent data that pertains to a level of intent to manually take over one or more functions of the vehicle 102. For example, the take-over intent 202 may include an intent to take over braking or not take over braking of the vehicle 102 that is being autonomously operated at a traffic intersection with crossing pedestrians as the vehicle 102 is approaching the traffic intersection. In one embodiment, the RNN 108 may output the intent data that pertains to the level of intent to manually take over one or more functions of the vehicle 102 to the intent prediction module 306 of the trust model application 106. Accordingly, the trust model application 106 utilizes the RNN 108 to complete processing of the trust model as a logistic regression model that considers scene difficultly, automation transparency, driving scene risk, and automation reliability along with the take-over intent of the driver at a previous point in time in a similar traffic scenario as input to predict take-over intent 202”, and paragraph [0135], “In particular, the vehicle control module 308 may be configured to communicate with the vehicle autonomous controller 110 to autonomously control one or more driving functions of the vehicle 102 based on the predicted take-over intent 202 of the driver of the vehicle 102. In one embodiment, if the take-over intent 202 is predicted to be high with respect to one or more functions of the vehicle 102 (e.g., braking, steering, accelerating), the vehicle control module 308 may be configured to communicate with the vehicle autonomous controller 110 to provide a particular level of automation control of one or more systems of the vehicle 102 that may provide respective functions.”); and the operator factor indicates one of a state level for the driving scenario, driving preferences associated with the driving scenario, and driver intent associated with the maneuver (See at least paragraph [0132], “With continued reference to the method 600 of FIG. 6, the method 600 may proceed to block 606, wherein the method 600 may include predicting a take-over intent 202 of the driver of the vehicle 102. In one embodiment, the RNN 108 may analyze the current traffic scenario based on current dynamic data, image data, and/or LiDAR data provided by the dynamic sensors 116, camera system 118, and/or laser projection system 120 of the vehicle 102 in addition to the short-term trust RNN state 218 and the long-term trust RNN state 220 to predict the take-over intent 202 of the driver of the vehicle 102. The take-over intent 202 may be predicted as the intent to manually take-over operation of one or more functions of the vehicle 102 in the current traffic scenario as the vehicle 102 is being autonomously or semi-autonomously operated.”).
With respect to claim 20, please see the rejection above with respect to claim 9, which is commensurate in scope to claim 20, with claim 9 being drawn to a prediction system and claim 20 being drawn to a corresponding method.
Claim(s) 3, 5, 14, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over AKASH ‘490 (US 20220324490 A1) in view of MA (CN 116975671 A) and Penilla (US 20170140757 A1).
Regarding Claim 3, AKASH ‘490 and MA teach The prediction system of claim 2, as set forth in the obviousness rejection above. AKASH ‘490 teaches wherein the instructions to derive the cue further include instructions to: measure one of a pulse and eye movement from the operator associated with focusing on a road object during the driving scenario (See at least paragraph [0066], “In particular, the one or more cameras that are positioned at one or more internal portions of an interior cabin of the vehicle 102 may be configured to capture images of the driver's eyes to be analyzed to determine the driver's eye movements within the vehicle 102. The one or more cameras that are positioned at one or more internal portions of an interior cabin of the vehicle 102 may be also be configured to capture images of the driver's body to be analyzed to determine the driver's body movements”, paragraph [0086], “The method 400 may proceed to block 406, wherein the method 400 may include determining eye gaze directions and body movements of the driver of the vehicle 102. In an exemplary embodiment, upon receiving the image data associated with images that are captured of the driver's eyes and/or portions of the driver's body as the vehicle 102 is being semi-autonomously or autonomously operated, the data processing module 302 may be configured to analyze the image data associated with one or more images captured for a predetermined period of time to analyze one or more gaze cues and body movements that may indicate when the driver takes over control or intends to take over control of the vehicle 102 during semi-autonomous and/or autonomous operation of the vehicle 102”, and paragraph [0088], “The data processing module 302 may thereby determine eye gaze directions of the driver of the vehicle 102 based on the gaze (viewpoint) of the driver and may output respective data. For example, the data processing module 302 may discretize the driver's gaze direction at any time belonging to one of a plurality of values pertaining to the driver's eye gaze direction that may include, but may not be limited to, the driver's eye gaze direction toward the road on which the vehicle 102 is traveling, the driver's eye gaze direction toward a dynamic object that may be located within the driving scene of the vehicle 102, the driver's eye gaze direction toward a static object that may be located within the driving scene of the vehicle 102, the driver's eye gaze direction towards road markings, road signage, traffic infrastructure, and the like that may be located within the driving scene, and the driver's eye gaze direction towards portions of the interior of the vehicle 102.”).
AKASH ‘490 and MA do not explicitly disclose, however, Penilla, in the same field of endeavor, teaches and identify one of a verbal tone and a keyword from the operator during the driving scenario (See at least paragraph [0017], “a tone identifier is descriptive or representative of an actual tone of voice used by the user when making a voice input to a vehicle's voice control interface, which includes at least one microphone used to capture the user's voice. The microphone may be integrated into the vehicle, e.g., near the steering wheel, dash, visor, a seat, etc., and can be connected to electronics of the vehicle. In one embodiment, the voice input is processed to capture an audio sample of the voice input. The audio sample may be the entire command, part of the command, or multiple commands, statements, one or more spoken words, verbal sounds, verbal gestures, grunts, moans, yells, expletives, and/or courteous statements words or the like. In general, the audio sample will include some audible sound that can be captured. The audio sample, in this example, refers to an amount of audio to cache or save to perform the analysis.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine the invention of AKASH ‘490 with the teachings of MA and Penilla such that the driver trust prediction system of AKASH ‘490 is further configured such that the dissonance is associated with a confidence metric by the operator for the driving scenario, as taught by MA (See paragraph [n0018], [n0026], [n0083].), and to identify one of a verbal tone and a keyword from the operator during the driving scenario, as taught by Penilla (See paragraph [0017].), with a reasonable expectation of success. The motivation for doing so would be to assess driver trust in autonomous vehicles to provide a reference for the objective and real-time assessment of driver trust in intelligent transportation environments, as taught by MA (See paragraph [n0003].), and to customize vehicle response and recommendations to the user of the vehicle, as taught by Penilla (See paragraph [0011].).
With respect to claim 14, please see the rejection above with respect to claim 3, which is commensurate in scope to claim 14, with claim 3 being drawn to a prediction system and claim 14 being drawn to a corresponding method.
Regarding Claim 5, AKASH ‘490 and MA teach The prediction system of claim 1, as set forth in the obviousness rejection above. AKASH ‘490 teaches And feed the operator factor directly to the learning model rather than the SDM (See at least paragraph [0048], “As discussed in more detail, the trust model as determined based on the automation variables and scene variables (from inputs from one or more systems, sensors, and/or components of the vehicle 102) and from crowdsourced survey data (that may be completed by the individuals) may be stored as datapoints upon the machine learning dataset 132. The RNN 108 may be utilized to access the machine learning dataset 132 and analyze the respective datapoints to output RNN states”, paragraph [0057], “the trust model application 106 may utilize data included within the machine learning dataset 132 to communicate with the vehicle autonomous controller 110 to control the level of automation transparency and/or an autonomous operation of one or more driving functions of the vehicle 102”, and paragraph [0058], “The trust model application 106 may be configured to communicate with the vehicle autonomous controller 110 to provide a level of automation control of one or more systems of the vehicle 102 in a particular traffic scenario in which the vehicle 102 is being autonomously or semi-autonomously operated to autonomously control one or more driving functions of the vehicle 102 based on the predicted take-over intent 202 of the driver of the vehicle 102.”).
AKASH ‘490 and MA do not explicitly disclose, however, Penilla, in the same field of endeavor, teaches wherein instructions to detect the characteristics further include instructions to: identify the operator factor from one of a driving questionnaire and a cognitive test that is text-based about the driving scenario, the cognitive test indicating a cognitive load during the driving scenario (See at least paragraph [0152], “In one embodiment agitated mood may also be sensed. For example, sensing can include, without limitation, user is short (e.g., curt), user is using known expletives, user is yelling, user asks vehicle to stop asking questions, driving is erratic compared to normal driving patterns, biometric sensors, blood pressure, position of hands on steering wheel, how hard the wheel is being grasped. In one embodiment, if user's heart rate is high, the vehicle may react in various ways. Heart rate may be sensed from user devices that are worn or from devices of the vehicle, e.g., sensors on the steering wheel, or some vehicle service, or optically (non-touch) sensing. In one embodiment, the vehicle may react by scaling back the number of questions posed to the user while in the vehicle. In another embodiment, the vehicle suggests turning off queries visually instead of verbally. In another embodiment, the vehicle GUI becomes more standard and easy to read. In still another embodiment, the vehicle changes ambient lighting to a calming hue.” The vehicle presenting questions to the user via the GUI corresponds to obtaining operator information from a questionnaire or text-based cognitive test.).
With respect to claim 16, please see the rejection above with respect to claim 5, which is commensurate in scope to claim 16, with claim 5 being drawn to a prediction system and claim 16 being drawn to a corresponding method.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine the invention of AKASH ‘490 with the teachings of MA and Penilla such that the driver trust prediction system of AKASH ‘490 is further configured such that the dissonance is associated with a confidence metric by the operator for the driving scenario, as taught by MA (See paragraph [n0018], [n0026], [n0083].), and to identify the operator factor from one of a driving questionnaire and a cognitive test that is text-based about the driving scenario, the cognitive test indicating a cognitive load during the driving scenario, as taught by Penilla (See paragraph [0152].), with a reasonable expectation of success. The motivation for doing so would be to assess driver trust in autonomous vehicles to provide a reference for the objective and real-time assessment of driver trust in intelligent transportation environments, as taught by MA (See paragraph [n0003].), and to customize vehicle response and recommendations to the user of the vehicle, as taught by Penilla (See paragraph [0011].).
Claim(s) 4, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over AKASH ‘490 (US 20220324490 A1) in view of MA (CN 116975671 A), Penilla (US 20170140757 A1), and AKASH (US 20220396287 A1).
Regarding Claim 4, AKASH ‘490, MA, and Penilla teach The prediction system of claim 3, as set forth in the obviousness rejection above. AKASH ‘490, MA, and Penilla do not explicitly disclose, however, AKASH ‘287, in the same field of endeavor, teaches wherein the instructions to adapt the SDM further include instructions to: receive, a score of the dissonance and a decision result about the dissonance from the learning model, the decision result indicates one of the operator controlling the vehicle without a manual takeover and the automated takeover of the vehicle by the SDM (See at least paragraph [0030], “The trust model calculator 110 may include a trust model database utilized for calculating trust scores. The trust model calculator 110 may be modeled based on a deep neural network, a convolutional neural network (CNN), or a recurrent neural network (RNN)”, paragraph [0031], “The trust model calculator 110 may receive occupant sensor data associated with an occupant of an autonomous vehicle, receive a first scene context sensor data associated with an environment of the autonomous vehicle at a first time, a second scene context sensor data associated with the environment of the autonomous vehicle at a second time, and generate a trust model for the occupant based on the occupant sensor data and the scene context sensor data”, and paragraph [0046], “The behavior controller 120 may select a combination or merely one of a driving automation action or an HMI action based on the trust model and the scene context.”); and prevent disagreement with additional commands by the SDM using the score and the decision result to adjust parameters, wherein the parameters are one of hyperparameters of an automated driving system (ADS) and weights of a model prediction control (MPC) (See at least paragraph [0044], “Additionally, workload dynamics for the trust model may be determined based on the eye gaze directions of the driver over a predetermined period of time. Automation transparency, automation reliability, and scene complexity may also be utilized to determine workload dynamics for the trust model. Accordingly, the adaptive trust application may capture a dynamic interaction between trust and workload behavior as it evolves over time (e.g., in real-time and as a predicted future point in time) and may be configured to process and implement an optimal control policy to appropriately vary automation transparency. According to one aspect, in addition to varying automation transparency, the adaptive trust application may be configured to alter the semi-autonomous or autonomous operation of one or more driving functions to achieve trust calibration” and paragraph [0055], “Neural network may be utilized to model the effects of human trust and workload on observable variables using a Markov decision process model to thereby enable the application to analyze the human trust model and workload dynamics based on the effects modeled using the Markov decision process model.” Neural network models include adjustable model parameters such as weights and configuration parameters (hyperparameters) that influence model behavior.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine the invention of AKASH ‘490 with the teachings of MA, Penilla, and AKASH ‘287 such that the driver trust prediction system of AKASH ‘490 is further configured such that the dissonance is associated with a confidence metric by the operator for the driving scenario, as taught by MA (See paragraph [n0018], [n0026], [n0083].), to identify one of a verbal tone and a keyword from the operator during the driving scenario, as taught by Penilla (See paragraph [0017].), and to receive, a score of the dissonance and a decision result about the dissonance from the learning model, the decision result indicates one of the operator controlling the vehicle without a manual takeover and the automated takeover of the vehicle by the SDM; and prevent disagreement with additional commands outputted by the SDM adjusting parameters using the score and the decision result, wherein the parameters are one of hyperparameters of an automated driving system (ADS) and weights of a model prediction control (MPC), as taught by AKASH ‘287 (See paragraph [0030], [0031], [0044], [0046], [0055].), with a reasonable expectation of success. The motivation for doing so would be to assess driver trust in autonomous vehicles to provide a reference for the objective and real-time assessment of driver trust in intelligent transportation environments, as taught by MA (See paragraph [n0003].), to customize vehicle response and recommendations to the user of the vehicle, as taught by Penilla (See paragraph [0011].), and to provide human-aware automation that adapts its behavior to avoid trust miscalibration, as taught by AKASH ‘287 (See paragraph [0003].).
With respect to claim 15, please see the rejection above with respect to claim 4, which is commensurate in scope to claim 15, with claim 4 being drawn to a prediction system and claim 15 being drawn to a corresponding method.
Claim(s) 6, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over AKASH ‘490 (US 20220324490 A1) in view of MA (CN 116975671 A) and AKASH (US 20220396287 A1).
Regarding Claim 6, AKASH ‘490 and MA teach The prediction system of claim 1 as set forth in the obviousness rejection above. AKASH ‘490 and MA do not explicitly disclose, however, AKASH ‘287, in the same field of endeavor, teaches further including instructions to: compute a supervised loss that trains the learning model using the characteristics, the driving command, the cue derived from the sensor data, the dissonance associated with the maneuver, and driving data (See at least paragraph [0031], “The trust model calculator 110 may receive occupant sensor data associated with an occupant of an autonomous vehicle, receive a first scene context sensor data associated with an environment of the autonomous vehicle at a first time, a second scene context sensor data associated with the environment of the autonomous vehicle at a second time, and generate a trust model for the occupant based on the occupant sensor data and the scene context sensor data” and paragraph [0055], “Neural network may be utilized to model the effects of human trust and workload on observable variables using a Markov decision process model to thereby enable the application to analyze the human trust model and workload dynamics based on the effects modeled using the Markov decision process model.” The neural network is trained using occupant and scene sensor data associated with the vehicle and the driver, which correspond to the data inputs used to train the learning model.); calculate an automation loss during training of the SDM using the characteristics, the cue, and a score of the dissonance outputted from the learning model (See at least paragraph [0031], “The trust model calculator 110 may receive occupant sensor data associated with an occupant of an autonomous vehicle, receive a first scene context sensor data associated with an environment of the autonomous vehicle at a first time, a second scene context sensor data associated with the environment of the autonomous vehicle at a second time, and generate a trust model for the occupant based on the occupant sensor data and the scene context sensor data.”); and train the SDM using the automation loss (See at least paragraph [0044], “Additionally, workload dynamics for the trust model may be determined based on the eye gaze directions of the driver over a predetermined period of time. Automation transparency, automation reliability, and scene complexity may also be utilized to determine workload dynamics for the trust model. Accordingly, the adaptive trust application may capture a dynamic interaction between trust and workload behavior as it evolves over time (e.g., in real-time and as a predicted future point in time) and may be configured to process and implement an optimal control policy to appropriately vary automation transparency. According to one aspect, in addition to varying automation transparency, the adaptive trust application may be configured to alter the semi-autonomous or autonomous operation of one or more driving functions to achieve trust calibration.”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine the invention of AKASH ‘490 with the teachings of MA and AKASH ‘287 such that the driver trust prediction system of AKASH ‘490 is further configured such that the dissonance is associated with a confidence metric by the operator for the driving scenario, as taught by MA (See paragraph [n0018], [n0026], [n0083].), and to compute a supervised loss that trains the learning model using the characteristics, the driving command, the cue derived from the sensor data, the dissonance associated with the maneuver, and driving data; calculate an automation loss during training of the SDM using the characteristics, the cue, and a score of the dissonance outputted from the learning model; and train the SDM using the automation loss, as taught by AKASH ‘287 (See paragraph [0031], [0044], [0055].), with a reasonable expectation of success. The motivation for doing so would be to assess driver trust in autonomous vehicles to provide a reference for the objective and real-time assessment of driver trust in intelligent transportation environments, as taught by MA (See paragraph [n0003].), and to provide human-aware automation that adapts its behavior to avoid trust miscalibration, as taught by AKASH ‘287 (See paragraph [0003].).
With respect to claim 17, please see the rejection above with respect to claim 6, which is commensurate in scope to claim 17, with claim 6 being drawn to a prediction system and claim 17 being drawn to a corresponding method.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEWEL ASHLEY KUNTZ whose telephone number is (571)270-5542. The examiner can normally be reached M-F 8:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached at (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEWEL A KUNTZ/Examiner, Art Unit 3666
/ANNE MARIE ANTONUCCI/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights