DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Currently, claims 1-20 are pending in the application. Claims 1, 9 & 16 are amended.
Continued Examination Under 37 CFR 1.114 1.
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/23/2026 has been entered.
Response to Arguments / Amendments
Applicant’s arguments have been fully considered but are rendered moot in view of the new ground of rejection necessitated by amendments initiated by the applicant.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-10, 13-17 & 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Rougeaux et al. (US 20190156100, hereinafter Rougeaux in view of Arora et al. (US 20230088021, hereinafter Arora) and Sicconi et al. ( US 20220402517, hereinafter Sicconi)
Regarding Claim 1, Rougeaux discloses a method for determining driver surprise, comprising:
receiving image data of a driver's pupils and video data of a face the driver over a time interval ([0053], [0093], FIG. 1, an imaging device flight camera 104 captures time separated images of the subject's face 106 including one or both of the subject's eyes 108 and 110 in gaze tracking system of driver monitoring system for a vehicle; [0016]);
determining a diameter of the driver's pupils over the time interval based on the image data ([0094], determine pupil/iris boundary when detecting three dimensional position of corneal reflections of the light source in conjunction with detecting the two dimensional position of one or more reference eye features such as a pupil center, iris center, pupil/iris boundary, iris/sclera boundary, eyelids or eye corners; [0100] FIG. 7) and
generating a pupil confidence value based on the diameter over the time interval ([0098], Circle recognition through a 2D Hough transform and radius histogramming; [0099], FIG. 6, Step 604, the two dimensional positions of the corneal reflections and the pupil position are fitted to a three dimensional cornea model having a known cornea center and pupil position to determine three dimensional positions of the corneal reflections and pupil center)
extracting one or more facial features from the video data ([0109], FIG. 8, Step 802, the image is processed to identify reference facial features such as eye corners, nostrils, mouth corners, ears or any other small recognizable areas on the face, and extract the three dimensional positions of those features) and generating a facial confidence value based on the one or more facial features ([0090] Several examples of shape regression are known, such as Supervised Descent Method (SDM) or Convolutional Neural Network (CNN) Or perform similar shape regression algorithm to identify other facial features to estimate a confidence that the region around the facial landmarks has the appearance of a face; [0110], FIG. 8, Step 803, the three dimensional positions of the facial features are fitted to a three dimensional model which is formed of a mesh structure that is deformable to account for the different profiles of individuals);
Rougeaux does not explicitly disclose applying a first weight to the pupil confidence value and applying a second weight to the facial confidence value to determine the driver's emotional response to a driving event; and determining whether the driver is surprised by the driving event based on the weighted pupil confidence value and the weighted facial confidence value.
Arora teaches applying a first weight to the pupil confidence value and applying a second weight to the facial confidence ([0021], FIGS. 6A-6E, apply a trained machine learning model to the image to determine the facial plane using images of other people having the facial landmarks already identified and the facial plane angle already determined and rank or adjust the parameters of the machine learning model based on the first initial 3D eye gaze vector determined using eye tracking as compared to the second 3D eye gaze vector determined using the facial plane; [0022], vehicle computing system 104 may determine 3D eye gaze vector 112 using an average of the first and second initial 3D eye gaze vectors. Vehicle computing system 104 may, in other examples, apply a weighting to one or more of the first and second 3D eye gaze vectors and use the weighted values to determine 3D eye gaze vector 112. Vehicle computing system 104 may determine weights to apply to the first and second initial 3D eye gaze vectors based on a confidence that the first or second initial 3D eye gaze vector was accurately determined ); and determining whether the driver is surprised by the driving event based on the weighted pupil confidence value and the weighted facial confidence value ([0040], vehicle computing system 104 determines that the driver is likely providing the user inputs. If the driver provides more than a threshold number of user inputs within a predetermined period of time or continues to interact with the infotainment system for greater than a predetermined period of time, vehicle computing system 104 may take various actions to encourage the driver to resume paying attention to the road such as output a message reminding the driver to pay attention to the road; [0060]).
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of applying weight to the pupil and to the facial confidence value as taught by Arora ([0021]) into the imaging system of Rougeaux in order to provide systems for determining that a driver's eyes are open, analyzing facial expressions, or blink speed enables the computing system to more precisely determine a location within the vehicle the user is looking and determines what is physically located at that the location within the vehicle (Arora, [0004]).
Rougeaux and Arora do not explicitly disclose determine the driver's emotional response to a driving event.
Sicconi teaches applying determine the driver's emotional response to a driving event ([0033], FIG. 1, Visible and NIR Camera pointed to driver face/eyes to analyze head pose; eye gaze tracking and record driver's face and back passenger seat in case of accident, Speech and Gesture Interface for driver to provide such as Rear Camera to view, analyze, record (in case of accident) back of car, 3D Accelerometer, Gyroscope, Compass, GPS (time, location, speed), plus VIN, Odometer, RPM, Engine Load via OBD II connection, Feature extraction from visual clues (attention, distraction, drowsiness, drunkenness, face identification, problematic interactions between driver and passenger(s), detection of altered voice, detection of hand gestures, Feature extraction of fatigue, stress, reaction to fear/surprise, from biosensors; Feature extraction of driving smoothness/aggressiveness, Feature extraction of ambient “harshness” and impact on driving stress).
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of determine the driver's emotional response to a driving event as taught by Sicconi ([0033]) into the imaging system of Rougeaux & Arora in order to provide systems for increasing the safety of voice conversations between drivers of a vehicle and remote parties, computing the risk level assessment as a function of the monitoring data and other information exchanged between the in-vehicle subsystem and the remote subsystem, and the computing apparatus is capable of generating automatic safety responses, alerts, and notifications for the driver and remote party (Sicconi, [0004]).
Regarding Claim 2, Rougeaux in view of Arora and Sicconi discloses the method of claim 1,
Rougeaux discloses wherein the video data comprises video collected from a video camera centered on the driver's face ([0093], FIG. 1, an imaging device flight camera 104 captures time separated images of the subject's face 106 including one or both of the subject's eyes 108 and 110 in gaze tracking system of driver monitoring system for a vehicle).
Regarding Claim 5, Rougeaux in view of Arora and Sicconi discloses the method of claim 3,
Arora discloses further comprising receiving video of the front of the vehicle and determining the event based on the video ([0002] Vehicles with semi-autonomous driving features that includes features that help keep the vehicle within lane boundaries, guide the vehicle around corners, or automatically accelerate and brake based on the presence of other vehicles).
Regarding Claim 6, Rougeaux in view of Arora and Sicconi discloses the method of claim 5,
Rougeaux discloses wherein the event comprises a safety hazard ([0002], vehicle driver drowsiness and attention monitoring systems). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 7, Rougeaux in view of Arora and Sicconi discloses the method of claim 1,
Rougeaux discloses further comprising determining a classification performance value indicating an accuracy of determining whether the driver is expressing surprise ([0014] upon detection of no corneal reflections, determining three dimensional positions of one or more reference facial features of the subject and performing a second eye gaze tracking procedure on one or both eyes of the subject based on the estimation of head pose of the subject from the positions of the one or more detected reference facial features).
Regarding Claim 8, Rougeaux in view of Arora and Sicconi discloses the method of claim 1,
Arora discloses wherein determining whether the driver is expressing surprise comprises calculating a weighted average of the weighted pupil confidence value and the weighted facial confidence value ([0022], vehicle computing system 104 may determine 3D eye gaze vector 112 using an average of the first and second initial 3D eye gaze vectors. Vehicle computing system 104 may, in other examples, apply a weighting to one or more of the first and second 3D eye gaze vectors and use the weighted values to determine 3D eye gaze vector 112. Vehicle computing system 104 may determine weights to apply to the first and second initial 3D eye gaze vectors based on a confidence that the first or second initial 3D eye gaze vector was accurately determined). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claims 9-10 & 13-15, system claims 9-10 & 13-15 of using the corresponding method claimed in claims 1-2 & 5-8, and the rejections of which are incorporated herein for the same reasons as used above.
Regarding Claims 16-17 & 19-20, Computer machine-readable medium claims 16-17 & 19-20 of using the corresponding method claimed in claims 1-2 & 5-8, and the rejections of which are incorporated herein for the same reasons as used above.
Claims 3-4, 11-12 & 18 are rejected under 35 U.S.C. 103 as being unpatentable over Rougeaux et al. (US 20190156100, hereinafter Rougeaux in view of Arora et al. (US 20230088021, hereinafter Arora), Sicconi et al. ( US 20220402517, hereinafter Sicconi) and Raz et al. (US 20230294706, hereinafter Raz)
Regarding Claim 3, Rougeaux in view of Arora and Sicconi discloses the method of claim 1, but does not explicitly discloses further comprising determining a latency between the driver expressing surprise and an event occurring in front of a vehicle of the driver, wherein the first and second weights are determined based on the latency.
Raz teaches determining a latency between the driver expressing surprise and an event occurring in front of a vehicle of the driver, wherein the first and second weights are determined based on the latency ([0020], FIGS. 1-4, determine the alertness or attentiveness of a driver in combination with an exterior monitoring device 13 (see FIG. 3) that detects event or object in the local environment of the vehicle. The interior monitoring device 12 may detect the response of the driver 11 to the detected event or object based on the reaction delay, timing, or response of the driver 11, the system 10 may determine an alertness level or state of the driver 11 based on an aggregated reaction time (e.g., an average reaction time, a median reaction time, etc.) for multiple objects or events presented during a driving trip. In addition to the response of the driver 11, the attentiveness may be determined based on a number of factors including the nature or relative significance of the events or objects detected in the local environment to improve the accuracy of the assessment of the driver 11; [0031] FIGS. 2A and 2B, determine that the point of interest likely corresponds to the sole event detected in the exterior image data 29c. Further, the controller 28 may track movement of the eyes 20 over multiple frames to determine that the eyes 20 are focused on or are directed to an object moving relative to the vehicle (e.g., due to the vehicle approaching the event)).
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of latency between the driver expressing surprise and an event occurring in front of a vehicle as taught by Raz ([0020]) into the imaging system of Rougeaux & Arora in order to provide systems for improving reaction time of the driver by, communicating instructions for modifying a target distance from another vehicle for an adaptive cruise control function, increasing sensitivity of a blind zone detection function, and modifying heat distribution or an illumination of a panel of a console (Raz, [0004]).
Regarding Claim 4, Rougeaux in view of Arora, Sicconi a and Raz discloses the method of claim 3,
Raz discloses wherein the latency is based on an action threshold associated with vehicle data of the event and a detection threshold associated with the image data ([0020], detect the response of the driver 11 to the detected event or object based on the reaction delay, timing, or response of the driver 11, the system 10 may determine an alertness level or state of the driver 11 based on an aggregated reaction time (e.g., an average reaction time, a median reaction time, etc.) for multiple objects or events presented during a driving trip). The same reason or rational of obviousness motivation applied as used above in claim 3.
Regarding Claims 11-12, system claims 11-12 of using the corresponding method claimed in claims 3-4, and the rejections of which are incorporated herein for the same reasons as used above.
Regarding Claim 18, Computer machine-readable medium claim 18 of using the corresponding method claimed in claims 3-4, and the rejections of which are incorporated herein for the same reasons as used above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Samuel D Fereja whose telephone number is (469)295-9243. The examiner can normally be reached 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DAVID CZEKAJ can be reached at (571) 272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAMUEL D FEREJA/Primary Examiner, Art Unit 2487
Prosecution Insights