DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims
Claims 1-31 are pending in the application.
Response to Arguments
Applicant’s arguments, see Remarks on pages 12-13, filed 9/10/2025, with respect to the rejection(s) of claims 1-31 under 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of US 2022/0126878 Al (Moustafa et al.)
Drawings
The drawing objections (mailed 6/25/2024) have been addressed and are withdrawn.
Specification
The objections to the specification given in the 6/25/2024 Office Action have been addressed and are withdrawn.
The title of the application needs to be fixed. METHODS AND SYSTEMS FOR IMPORIVING USER ALERTNESS IN AN AUTONOMOUS VEHICLE [Wingdings font/0xE0] METHODS AND SYSTEMS FOR IMPROVING USER ALERTNESS IN AN AUTONOMOUS VEHICLE
Paragraph [0011] needs to be fixed:
[0011] Advantageously, the provision of a device as described above permits a driver or user of the vehicle to ensure that the vehicle is operating correctly when in an autonomous driving mode. The device may be configured to have a [lower] low level of tolerance in relation to external vehicle events and threats so that it reacts to the vehicle's operation in a more critical manner.
Claim Objections
The objections to claims 1,15,17,19,20 as mentioned in the 6/25/2024 Office Action have been addressed and are withdrawn.
Claim 26 is objected to because of the following informalities: claim 26 mentions “Claim 254” which should be replaced by “Claim 25”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 5-6, 10-12, 15, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over US 2018/0164825 (Matus et al., hence Matus) in light of US 2022/0126878 (Moustafa et al., hence Moustafa)
As for claim 1, Matus teaches a portable [electronic] monitoring device (Matus: mobile device, mentioned [0013],[0074]) for providing an in-vehicle user warning system (Matus: alert to human mentioned in [0072], "driver guidance" mentioned in [0023]) about how a semi-autonomous vehicle (Matus: mentioned in [0023], also under BRI this can also be interpreted as an autonomous vehicle that can shift over to manual; see [0072]) is being driven autonomously by an autonomy system during a driving period (Matus: again see [0072]; the period during which the vehicle is in autonomous mode qualifies), the [monitoring] device being removably and securely mountable to the vehicle (Matus: "mobile computing devices", which are "removably and securely mountable to the vehicle", are mentioned in [0013]), the [monitoring device] comprising:
a sensor set comprising at least one sensor for sensing an exterior environment outside of the vehicle (Matus: vehicle environment sensors 222, see [0014]) and movement of the vehicle within the exterior environment ([0026],[0046]), an interface for receiving user input commands and delivering a warning output (Matus: a mobile device would do this; also see [0013] for a list of the other possible devices; also see [0061] which mentions "passenger input", thus implying a mechanism by which users input data into the system);
and a processor operatively connected to the sensor set and the interface (Matus: Fig. 2A showing system, mention of processor in [0074]);
wherein the sensor set (Matus: vehicle environment sensors 222, see [0014]) is configured to monitor automatic operation of the semi-autonomous vehicle within the exterior environment during the driving period and to generate sensor data representing driving events concerning automated driving behaviour during the automatic operation of the vehicle with respect to the exterior environment occurring during the driving period; (Matus: see paragraphs [0025]-[0026])
the processor being configured to: process the sensor data during the driving period to compare the automated driving behaviour during the automatic operation of the vehicle in the external environment with a model of expected automated vehicle driving behaviour for a particular driving event (Matus: “processor” mentioned in [0074]; Block S110 of the method preferably includes implementing a comparative autonomous model that compares autonomous vehicle behavior to human driver behavior ( e.g., an "average human" behavior determined upon analyzing responses of a population of humans) across different driving maneuvers.[0025]);
identify a dangerous driving event, if the automated driving behaviour deviates beyond a threshold from the expected automated vehicle driving behaviour; and if a dangerous driving event has been detected, generate a warning alert via the interface to alert a driver to the occurrence of the dangerous driving event. (Matus: "The transition into the overridden state can be triggered based upon an analysis that the autonomous vehicle is in unknown territory, experiencing conditions with an above-threshold tolerance of unknowns, in a compromised state due to a security threat, and/or by any other suitable rationale. The transition into the overridden state can be manually performed (e.g., a human driver receives an alert and then provides an input that indicates that he/she is taking control of driving). Alternatively, the transition into the overridden state can be automatic, such that the autonomous vehicle stops driving itself ( e.g., pulls over when safe and slows to a stop) and the human operator must intervene." (underlining added) [0072]; alert shown in Fig. 2A)
Matus does not specifically teach the supervisory monitoring device [being] independent of the autonomy system but this is known in the art. Mustafa teaches the supervisory monitoring device [being] independent of the autonomy system (Mustafa: "The comprehensive cognitive supervisory system (C2S2) 3005 may sit on top of (e.g., may supervise) the regular automation systems of an autonomous vehicle...In some examples, C2S2 3005 includes logic executable to monitor the level of autonomy in the car and comprises three main modules: functional assurance, quality assurance, and safety assurance"[0186]; "In some examples, C2S2 3005 includes logic executable to monitor the level of autonomy in the car and comprises three main modules: functional assurance, quality assurance, and safety assurance" [0187]; "For example, when a sensor goes out of order or passenger safety gets jeopardized in scenarios like sensor/component failure, the autonomy level may have to change. C2S2 3005 can change the level of autonomy and inform both the driver and the remote surveillance system (3010)." [0189]; "When a problem occurs, the vehicle may send out a system malfunction alert (3520). Accordingly, the human driver will receive the alert (3525). This alert can be visual, audio, tactic, or any other type of alert." [0200]; "If it is determined that the malfunction is not serious enough to need immediate driver interaction, the vehicle can switch to a lower autonomous mode (3530)." [0201]; "If there is another error, the vehicle can once again send out a system malfunction alert (3540). Once again, the driver will receive that alert after it is sent (3545)." [0202])
It would have been obvious to one of ordinary skill in the art at the time of the application to have the supervisory monitoring device be independent, as shown in Moustafa, in the system of Matus. The motivation would be to have a separate supervisory system which could sit on top of but be separate from the autonomous vehicle system (which allows the supervisory system to be a separate modular system for easier error-checking.)
As for claim 3, Matus, as modified by Mustafa, teaches wherein the sensor set includes at least one external weather monitoring sensor. (Matus: "In variations, generating and/or implementing the sensor-surrounding model can include collecting data from one or more of:...pressure sensors, moisture sensors, light sensors, temperature sensors, any other suitable sensors, location identifying systems (e.g., GPS) in combination with GPS-based weather services, vehicle subsystem states (e.g., windshield wiper states, AC/heater states, lighting system states, cruise control system states, gear shifting states, overdrive states, etc.), and any other suitable component that can be used to detect road and/or weather conditions that would affect or require sensor performance."[0052].)
As for claim 5, Matus, as modified by Mustafa, teaches wherein the sensor set includes at least one positional sensor and the at least one positional sensor comprises a gyroscope, a magnetometer, an altimeter, a geolocation sensor or an accelerometer. (Matus: "...by leveraging non-generic location data (e.g., GPS data), motion data (e.g., accelerometer data, gyroscope data), and/or other suitable data from a plurality of mobile devices (e.g., non-generalized mobile devices), sensor systems associated with the vehicle and/or surroundings of the vehicle, security diagnostic systems, and any other suitable systems..."[0017])
As for claim 6, Matus, as modified by Mustafa, teaches wherein the sensor set includes an audio sensor and the sensor data includes audio signals. (Matus: "...The method 100 may additionally or alternatively leverage the availability of additional data captured by the data sources ( e.g., audio data, vehicle sensor data, etc.)…[0011]; also mentioned in "audio sensors ( e.g., to detect sounds indicative of interactions between the autonomous vehicle and other entities/objects, to detect vehicle horn usage, etc.);" [0046]).)
As for claim 10, Matus, as modified by Mustafa, teaches comprising an Artificial Intelligence (AI) engine configured to operate as a neural network for learning and modelling autonomous behaviour of the vehicle, the processor being operatively connected to the Al engine. ([0025] has a list of steps including collecting data, analysing such, and comparing the autonomous vehicle behavior to "average human behavior" to perform risk analysis (Block S110). Paragraph [0027] discusses generating the comparative autonomous module using machine learning and neural networks.)
As for claim 11, Matus, as modified by Mustafa, teaches wherein the Al engine comprises a neural network trained to model expected vehicle driving behaviour. (Matus: Paragraph [0027] discusses generating the comparative autonomous module using machine learning and neural networks.)
As for claim 12, Matus, as modified by Mustafa, teaches wherein the neural network is trained using sensor data collected from manual and/or automated operation of the vehicle. (Mustafa: "FIG. 21 depicts a training phase for a handoff decision model 2110. An ML training algorithm 2102 uses driver historical data 2104, driver states 2106, and handoff decisions ground truth 2108 to train handoff decision model 2110." [0141])
As for claim 15, Matus, as modified by Mustafa, teaches wherein the processor is configured to: determine the threshold for the particular driving event (Matus: “thresholds” are mentioned in [0018] as being something that the calculated risks are being compared to. Being able to pick out the correct threshold implies the system being able to determine a threshold for a particular driving event.)
and if the comparison between the detected automated driving behaviour and the model of expected automated vehicle driving behaviour for the particular driving event indicates that a deviation has occurred: compare the deviation and the threshold to determine if the deviation is beyond the threshold. (Matus: "For example, correction of deviations in proper vehicle operation can be initiated, using interfaces to control systems of the autonomous vehicle, in response to detection of one or more risks surpassing defined thresholds."[0018] (if interpreting "risk" under BRI as level of deviation from expected trajectory) )
As for claim 24, Matus, as modified by Mustafa, teaches wherein the sensor set includes at least one sensor for sensing an interior environment of the vehicle. ("...and any other suitable component that can be used to detect human factors ( e.g., pedestrian factors, vehicle occupant factors) conducive to triggering different driving behaviors."[0030]; "...wearable computing devices; biometric monitoring devices ( e.g., to detect physiological states of individuals involved in a traffic situation, to detect cognitive states of individuals involved in a traffic situation, etc.); and/or any other suitable sensors/sensor systems." [0046] (this would cover interior sensors))
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Matus as applied to claim 1 above, and further in view of “Proximity sensor” (attached as NPL-Wikipedia.pdf to the Office Action of 9/10/2025, henceforth “Wikipedia”).
As for claim 2, Matus, as modified by Mustafa, teaches wherein the at least one sensor comprises a proximity sensor (Matus: "data for evaluation can be acquired from one or more of: proximity sensors ( e.g., to determine proximity of the autonomous vehicle to other objects)" [0046]). Matus does not specifically state which type of proximity sensor is being used, but infrared sensors or camera being used as the proximity sensors is known in the art (See Wikipedia).
It would have been obvious to one of ordinary skill in the art at the time of the application to use proximity sensors as known in the art in the system of Matus. The motivation would have been to use known technology as proximity sensors.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Matus, in light of Mustafa as applied to claim 1 above, and further in view of WO 2012/080741 (Wright et al., hence Wright).
Matus, as modified by Mustafa, does not specifically teach wherein the portable supervisory monitoring device comprises a local wireless communications link to a personal telecommunications device which provides a user interface to the monitoring device. (Matus: Fig. 2A shows something that might be a smartphone, but “personal telecommunications device” is not mentioned in the specification.) However, Wright teaches wherein the portable monitoring device comprises a local wireless communications link to a personal telecommunications device which provides a user interface to the supervisory monitoring device. (Wright: “Preferably, the mobile device is arranged to interface with an external device wirelessly, for example via a Bluetooth® connection. Advantageously, this removes the need for the mobile device to be physically connected to the external device. As will be appreciated, following a driving period, a user may want to leave the vehicle and so take a personal mobile telecommunication device with them, and so a wireless interface with such an external device obviates the inconvenience of physically detaching and then subsequently re-attaching the mobile device.” (pg.10, lines 11-17). Note that the mobile device is acting here as a supervisory monitoring device.)
It would have been obvious to one of ordinary skill in the art at the time of the application to use a personal telecommunications device as a possible user interface, as outlined by Wright, in the system of Matus, with a reasonable expectation of success, since the personal telecommunications device would just need to have the relevant application software uploaded on it.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Mustafa as applied to claim 1 above, and further in view of GB 2588983 (Ayeman).
As for claim 7, Matus, as modified by Mustafa, does not specifically teach wherein the interface comprises a touchscreen and a loudspeaker. However, Ayeman teaches wherein the interface comprises a touchscreen and a loudspeaker. ("The display device 312 can include an LCD, LED, or AMOLED display, a touchscreen, a haptic display such as a vibrating pad (e.g., for a blind rider), or a loudspeaker to provide audible output to a passenger" [000118])
It would have been obvious to one of ordinary skill in the art at the time of the application to use a touchscreen and a loudspeaker as a possible user interface, as outlined by Ayeman, in the system of Matus. The motivation would be to add further methods of interaction with the system for the user.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Mustafa as applied to claim 1 above, and further in view of “Head-up display” (NPL-Wiki-HUD, henceforth “Wiki-HUD”.)(copy provided in 6/25/2024 Office Action.)
As for claim 8, Matus, as modified by Mustafa, does not specifically teach wherein the interface comprises a projector configured to project images onto a surface of the vehicle to create a head-up display. However, this is known in the art. See Wiki-HUD. It would have been obvious to one of ordinary skill in the art at the time of the application to add a HUD projection system as mentioned in Wiki-HUD to the electronic monitoring device system described in Matus as modified by Mustafa. The motivation would be to include the benefits of a head’s up display system.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Mustafa as applied to claim 1 above, and further in view of (US 10,861,325 B2) Lorenz.
As for claim 9, Matus, as modified by Mustafa, does not specifically teach wherein the monitoring device is a telecommunications device comprising a wireless communications engine for communicating with a remote server, wherein the wireless communications engine is configured to receive information regarding the external environment through which the vehicle is travelling. However, Lorenz teaches wherein the monitoring device is a telecommunications device comprising a wireless communications engine for communicating with a remote server, wherein the wireless communications engine is configured to receive information regarding the external environment through which the vehicle is travelling. ("For example, the data may be obtained from a weather server. The server may be a different server to a server that, in preferred embodiments, performs the method of the present invention. The weather data may be from a repository storing data indicative of one or more regions of the navigable network currently considered to be affected by one or more adverse weather conditions, optionally wherein the data comprises data indicative of the type or types of adverse weather condition affecting the or each region. The repository may be stored by a remote server."[Col. 12, lines 57-67])
It would have been obvious to one of ordinary skill in the art at the time of the application to use a wireless telecommunication device to obtain weather information, as outlined by Lorenz, in the system of Matus, as modified by Mustafa. The motivation would be to add a method to obtain weather data for the system.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Matus, in light of Mustafa as applied to claim 12 above, and further in view of “Anomaly detection using training data that doesn’t have anomalies”, attached as NPL-Reddit.pdf, henceforth “Reddit”.
As for claim 13, Matus, as modified by Mustafa, does not specifically teach wherein the sensor data collected prior to the current driving period is data that has been validated as being sensed in one or more driving periods during which no dangerous driving events were identified. However, this is known in the art of training neural networks, where “the correct image/data” is used to train the neural network (a first training set). See Reddit, which discusses training a neural network using a training set without anomalies.
It would have been obvious to one of ordinary skill in the art at the time of the application to have used the training techniques as outlined in Reddit to train the neural network in the system of Matus, as modified by Mustafa. The motivation would be to use a known method for the neural network training.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Mustafa as applied to claim 11 above, and further in view of ”Vehicle Trajectory Prediction using Non-linear Input-Output Time Series Neural Network” (attached as NPL-Sushmitha.pdf to the Office Action mailed 9/10/2025, henceforth “Sushmitha”)
As for claim 14, Matus does not specifically teach wherein, based on the neural network and sensor data, the Al engine is configured to generate the model of expected automated vehicle driving behaviour for the particular driving event. However, Sushmitha teaches wherein, based on the neural network and sensor data, the Al engine is configured to generate the model of expected automated vehicle driving behaviour for the particular driving event.
It would have been obvious to one of ordinary skill in the art at the time of the application to use a neural network to generate a model of automated vehicle driving behavior, as outlined by Sushmitha, in the system of Matus, with a reasonable expectation of success, since Sushmitha teaches a method which can be applied using a computer algorithm. The motivation would be to generate possible models for use.
Claims 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Mustafa as applied to claim 15 above, and further in view of US 2010/0102972 (Middelkauff)
As for claim 16, Matus teaches wherein the threshold is determined based on the driving event ([0018]), but does not specifically teach the threshold [is also] based on at least one other parameter selected from the group consisting of: a reaction time of the driver; a level of autonomy of the vehicle; condition of the vehicle; a road type; a weather condition; and one or more user settings. However, Middelkauff teaches the threshold [is also] based on at least one other parameter selected from the group consisting of: a reaction time of the driver; a level of autonomy of the vehicle; condition of the vehicle; a road type; a weather condition; and one or more user settings. ("An exemplary driver inattention detection system according to principles of the invention may be calibrated. Calibration may ensure that steering signals are accurately processed. Calibration may also set a minimum threshold for steering signal activity, below which driver inattention is assumed."[0020])
It would have been obvious to one of ordinary skill in the art at the time of the application to use a driver reaction detection system as outlined in Middelkauff in the system of Matus. The motivation would be to add further factors in the threshold decision-making.
As for claim 17, Matus, as modified by Mustafa and by Middelkauff, also teaches wherein the at least one other parameter comprises reaction time of the driver, and wherein the sensor set includes at least one sensor for sensing an interior environment of the vehicle, the processor being configured to determine the reaction time of the driver based on current and/or historical sensor data sensed from the at least one sensor for sensing the interior environment of the vehicle. (Middelkauff: "A driver inattention condition may be determined to exist if an active steering count is below the determined minimum threshold steering count. In such case, a first alarm perceptible to a driver may be activated upon determining that a driver inattention condition exists."[0025] (here the steering wheel with rotary encoder attached are being used as a real-time sensor to determine level of attention of driver, which can be considered as another way of measuring the reaction time. And since this is the steering wheel plus attached mechanism mounted in the occupant cabin of the vehicle, it can be considered as "sensing an interior environment of the vehicle")
Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Mustafa and in light of Middelkauff as applied to claim 16 above, and further in view of US 2015/0344030 Al (Damerow et al., hence Damerow).
As for claim 18, Matus does not specifically teach wherein the driving event comprises a vehicle manoeuvre and wherein the threshold is based on one or more of: vehicle speed during the manoeuvre; vehicle braking during the manoeuvre; and vehicle steering angle during the manoeuvre. However, Damerow teaches wherein the driving event comprises a vehicle manoeuvre (such as shown in Fig. 7) and wherein the threshold is based on one or more of: vehicle speed during the manoeuvre; (see [0053] for an example involving vehicle speed) vehicle braking during the manoeuvre (See [0048], which mentions “alternative trajectories” which could involve braking); and vehicle steering angle during the manoeuvre. (“heading angles, masses, etc.”[0050] (which would include a vehicle steering angle).)
It would have been obvious to one of ordinary skill in the art at the time of the application to use a risk analysis system as outlined in Damerow in the system of Matus, as modified by Moustafa, with a reasonable expectation of success since Damerow’s system can be implemented via information from sensors and software to implement the algorithm. The motivation would be to add another set of possible factors into the threshold used.
As for claim 19, Matus, as modified by Moustafa, does not specifically teach wherein the driving event comprises an interaction with another vehicle and wherein the threshold is based on one or more of: wherein the driving event comprises an interaction with another vehicle and wherein the threshold is based on one or more of: the direction of travel of the other vehicle; the location of the other vehicle; whether the other vehicle is recognised as operating or capable of operating autonomously; and/or the behaviour of the other vehicle. However, Damerow teaches wherein the driving event comprises an interaction with another vehicle (See Fig. 7, where the situation is one car crossing an intersection and avoiding a collision with vehicles.) and wherein the threshold is based on one or more of:
the speed of one or each vehicle during the interaction; vehicle braking during the interaction; the proximity of the other vehicle; ("For the calculation of the values in the risk map, we use a continuous risk function based on risk indicators, which calculate the risk from the states of the ego-vehicle and the other traffic participant for one moment in time. The risk e.g. depends on the distance and the velocities of the involved traffic participants at one point in time, but can be extended to include e.g. heading angles, masses, etc. The risk map will then exhibit pronounced maxima at certain points of ego-car driven trajectory length and behavior parameters. A favorable behavior would then be one that avoids these maxima.”[0050])
the direction of travel of the other vehicle; (see previous, where "heading angles" are mentioned)
the location of the other vehicle; (see previous, where "distance…of the involved traffic participants…can be extended to include…etc." is mentioned)
whether the other vehicle is recognised as operating or capable of operating autonomously; (could be included under the "...can be extended to include…etc." clause)
and/or the behaviour of the other vehicle. (See above; “velocities and headings of "the involved traffic participants" would cover this.)
It would have been obvious to one of ordinary skill in the art at the time of the application to use a risk analysis system as outlined in Damerow in the system of Matus, as modified by Moustafa, with a reasonable expectation of success since Damerow’s system can be implemented via information from sensors and software to implement the algorithm. Again, the motivation would be to add another set of possible factors into the threshold used.
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Moustafa as applied to claim 24 above, and further in view of US 2019/0318180 (Mudalige et al., hence Mudalige).
As for claim 25, Matus, as modified by Moustafa, teaches wherein the sensor set is further configured to monitor the interior environment of the vehicle during the driving period (Mustafa: "Example 108 includes the method of example 105, wherein the plurality of sensors coupled to the autonomous vehicle include interior sensors inside the autonomous vehicle, and determining whether the requested takeover is safe is based sensor data received from the interior sensors." [0397]) Neither Matus nor Moustafa specifically teaches [to] generate sensor data representing a current attention state of the driver during the driving period. However, Mudalige teaches wherein the sensor set is further configured to monitor the interior environment of the vehicle during the driving period (Mudalige: "The driver monitoring system 14 determines information about the direction in which the driver is gazing. For example, the driver monitoring system 14 includes one more sensor devices 25 that sense activity of the driver and generate sensor signals based thereon; and a data processing module 26 that receives and processes the sensor signals in order to determine gaze data indicating the gaze direction. In various embodiments, the sensor devices 25 include one or more cameras disposed within the vehicle 10 and directed towards the head, face, and/or upper body of the driver. The cameras capture images of the driver and generate image data based thereon. The data processing module 26 receives the image data, processes the image data using one or more image processing techniques, and determines a gaze and/or head direction of the driver." [0025]) and to generate sensor data representing a current attention state of the driver during the driving period. (Mudalige: "In various embodiments, the fourth non-transitory module computes the attention score based on a matching level between the object location and the gaze location and a matching level between the upcoming behavior location and the gaze location. In various embodiments, the fourth non-transitory module computes the attention score based on a weighting factor applied to the matching level between the object location and the gaze location and a weighting factor applied to the matching level between the upcoming behavior location and the gaze location" [0014]).
It would have been obvious to one of ordinary skill in the art at the time of the application to use the driver monitoring unit of Mudalige in the system of Matus, as modified by Mustafa. The motivation would be to incorporate monitoring of the condition of the driver in the monitoring system.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Matus in light of Moustafa, and in light of Mudalige as applied to claim 25 above, and further in view of JP 2012-85747 (Morikawa et al., hence Morikawa).
As for claim 26, Matus, as modified by Moustafa and by Mudalige teaches wherein the processor is configured to: determine a required attention state of the driver with respect to the current operation of the semi- autonomous vehicle within the exterior environment (Mudalige: "The behavior location determination module 54 receives as input upcoming behavior data 68 from the behavior planning system 20. The behavior location determination module 54 determines an approximate location 70 where the driver should be looking to prepare for the provided upcoming behavior. For example, in various embodiments, the behavior location determination module 54 determines the location 70 based on characteristics of the upcoming behavior."[0038]); compare the current attention state of the driver and the required attention state of the driver (Mudalige: "The attention scoring module 56 receives as input the gaze location 62, the general gaze location 63, the object location(s) 66, and the upcoming behavior location(s) 70. The attention scoring module 56 computes an attention score of the driver based on the received data. For example, the attention scoring module computes the attention score by matching the gaze location with the object location(s), matching the gaze location with the upcoming behavior location(s), and matching the gaze location 60 with the general gaze location 63." [0042]); None of Matus, Mustafa, or Mudalige teach [to] generate a warning alert signal if the current attention state deviates beyond a threshold value from the required attention state. However, Morikawa teaches wherein the processor is configured to: ("All or some of the components constituting the attention state determination apparatus described above are realized as a general-purpose processor (semiconductor circuit) that executes a computer program. Alternatively, it is realized as a dedicated processor in which such a computer program and a processor are integrated. For example, the attention state determination apparatus according to the first embodiment includes a general-purpose processor"[pg. 26).) determine a required attention state of the driver with respect to the current operation of the semi- autonomous vehicle within the exterior environment; (Morikawa: predetermined threshold mentioned: "When the magnitude of the eyeball retention related potential is smaller than a predetermined threshold, it may be determined that the user's attention is distracting."(Pg. 7)); compare the current attention state of the driver and the required attention state of the driver; (see previous) and generate a warning alert signal if the current attention state deviates beyond a threshold value from the required attention state. (Morikawa: warning mentioned in S808, page 16).
It would have been obvious to one of ordinary skill in the art at the time of the application to use a driver reaction detection system as outlined in Morikawa in the system of Matus, with a reasonable expectation of success since Morikawa’s eye reaction detection system can be implemented as an add-on. The motivation would be to monitor the activity of the driver as another parameter which affects the vehicle autonomy level.
As for claim 27, Matus, as modified by Mustafa, by Mudalige, and by Morikawa, teaches wherein the required attention state is determined based on one or more vehicle parameters. (Mudalige: "In one embodiment, as shown in FIG. 4, the upcoming behavior includes a forward path including a curve. The curve is provided in a local coordinate frame with the vehicle 10 located at the origin (0,0). The recommended lookahead distance (e.g., 10 seconds) can be determined based on a current vehicle speed and a required bias angle can be determined by locating the recommended lookahead point on the curve (either by choosing the closest communicated point or through linear interpolation)." [0039])
As for claim 28, Matus, as modified by Mustafa, by Mudalige, and by Morikawa, also teaches wherein the one or more vehicle parameters includes a level of autonomy of the vehicle, a vehicle speed, a vehicle occupancy level, and/or a quality of autonomous vehicle operation. (Mudalige: "In one embodiment, as shown in FIG. 4, the upcoming behavior includes a forward path including a curve. The curve is provided in a local coordinate frame with the vehicle 10 located at the origin (0,0). The recommended lookahead distance (e.g., 10 seconds) can be determined based on a current vehicle speed and a required bias angle can be determined by locating the recommended lookahead point on the curve (either by choosing the closest communicated point or through linear interpolation)." (underlining added) [0039])
As for claim 29, Matus, as modified by Mustafa, by Mudalige, and by Morikawa, also teaches wherein the required attention state is determined based on one or more external environment parameters. (Mudalige: "In one embodiment, as shown in FIG. 4, the upcoming behavior includes a forward path including a curve. The curve is provided in a local coordinate frame with the vehicle 10 located at the origin (0,0). The recommended lookahead distance (e.g., 10 seconds) can be determined based on a current vehicle speed and a required bias angle can be determined by locating the recommended lookahead point on the curve (either by choosing the closest communicated point or through linear interpolation)." (underlining added) [0039]; The curve is an external environment parameter.)
As for claim 30, Matus, as modified by Mustafa, by Mudalige, and by Morikawa, also teaches wherein the one or more external environment parameters includes a road type, a road quality, a traffic density, a weather type, a classification of how urban or rural the environment is, driving behaviour of other vehicles in the vicinity, and/or the presence of one or more dangerous driving events and/or other threats. (Morikawa: “For example, if a partial section includes 10 saccades and is defined by four partial sections, the same number of saccades can always be determined. If this method is taken, it can respond flexibly to the saccade frequency assumed to differ depending on road conditions and tasks. For example, in places with high saccade frequency, such as intersections and crowded roads, the time required to collect analysis data is shortened as a whole because a certain number of saccades are reached." (pg. 18) (underlining added))
Claim 31 is rejected under 35 U.S.C. 103 as being unpatentable over Matus, in light of Mustafa as applied to claim 1 above, and further in view of Mudalige.
As for claim 31, Matus, modified by Mustafa, teaches wherein the processor is configured to, if a dangerous driving event is detected, determine [when] resumption of manual control of the vehicle is necessary, and to generate the warning signal. (Matus: “The transition into the overridden state can be triggered based upon an analysis that the autonomous vehicle is in unknown territory, experiencing conditions with an above-threshold tolerance of unknowns, in a compromised state due to a security threat, and/or by any other suitable rationale. The transition into the overridden state can be manually performed ( e.g., a human driver receives an alert and then provides an input that indicates that he/she is taking control of driving). Alternatively, the transition into the overridden state can be automatic, such that the autonomous vehicle stops driving itself ( e.g., pulls over when safe and slows to a stop) and the human operator must intervene. However, transitioning into an overridden state can alternatively be conducted in any other suitable manner."[0072]
Matus does not specifically mention determin[ing] a time point before which resumption of manual control of the vehicle is necessary, and to generate the warning signal before the time point at the latest. However, Reed teaches wherein the processor is configured to, if a dangerous driving event is detected, determine a time point before which resumption of manual control of the vehicle is necessary, and to generate the warning signal before the time point at the latest. (Reed: "For example, the assisted driver is approaching an intersection where the light is red and there is a car stopped at the light directly ahead. The assisted driver is distracted, looking at the radio or in the back seat at their children. Conventional systems would wait until the last moment to intervene whereas the system 200 would intervene gently much sooner because the driver is distracted. For example, while a conventional system may wait until a detected impending collision to apply a break, the current system can first sound a warning chime to refocus the driver, then gently pump the breaks to get the drivers attention if the chime was unsuccessful, and only apply a hard stop break at the last minute to avoid the collision. The result is a safer situation for all traffic involved." [0030].)
It would have been obvious to one of ordinary skill in the art at the time of the application to implement the timing as explained in Reed in the system of Matus, as modified. The motivation would be, as Reed mentions, to implement a safer situation for all traffic.
Allowable Subject Matter
Claims 20-23 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TANYA CHRISTINE SIENKO whose telephone number is (571)272-5816. The examiner can normally be reached Mon - Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito Robinson can be reached at 571-270-3912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TANYA C SIENKO/ Examiner, Art Unit 3664
/KITO R ROBINSON/ Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights