DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 9-11, 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Westbrook (US 2019/0056731) in view of Coles (WO 2023/193038), and Coimbra De Andrade (US 2022/0051038)
As to claim 1 Westbrook discloses an apparatus comprising:
a processor configured to
receive multi-modal sensor data associated with an occupant driving a vehicle form a sensor of the vehicle (Paragraph 30-32 “some aspects, the brainwave activity measured by the one or more EEG sensors 140 may be provided to the automatic driving system 132, such that the automatic driving system 132 may establish a baseline of brainwave activity of the driver and/or one or more passengers…….In yet another example, the risk assessment profile may be generated based on a combination of the mental state of the driver and the eye movement of the driver, as captured by the interior camera 142.” Paragraph 23 “The vehicle 102 may also include one or more electroencephalograph (EEG) sensors 140. In some aspects, the one or more EEG sensors 140 may be placed in the headrest or the roof liner of the vehicle 102, may be worn by the occupants (i.e., a driver and/or one or more passengers) of the vehicle 102, or embedded in the mobile device of the occupants. In some aspects, the one or more EEG sensors 140 may detect the following brainwaves of the occupants: 1) delta brainwaves; 2) theta brainwaves; 3) alpha brainwaves; 4) beta brainwaves; and 5) gamma brainwaves.”),),
generate a prediction, that the occupant of the vehicle has attention deficit hyperactivity disorder (ADHD) based on the multi-modal data using a machine learning model (Paragraph 27 “For example, beta brainwaves may be generated when the driver is actively driving, i.e., alert, attentive, engaged in problem solving, exercising judgment/decision making, and engaged in a focused mental activity. Too much beta brainwaves may be indicative of excessive amounts of adrenaline, increased anxiety, increased arousal, an inability to relax, and/or increased stress levels; whereas, too little beta brainwaves may be indicative of ADHD, daydreaming, depression, and/or poor cognition. An ideal amount of beta brainwaves may promote conscious focus, increased memory, and/or increased problem solving abilities.”, Paragraph 4 “For example, a system may generate a risk assessment for a driver indicating a mental state of the driver, what risks the driver takes based on the mental state, and under what circumstances such risks are taken, for example. In some aspects, the risk assessment profile may be generated using machine learning algorithms, such that the risk assessment profile may dynamically evolve over time.”),
based on the prediction, control the vehicle to influence how the occupant drives the vehicle(Paragraph 33 “In some aspects, when the automatic driving system 132 determines that the driver is at an increased risk of driving unsafely, the automatic driving system 132 may prevent the vehicle 102 from transitioning from the autonomous driving mode to the manual driving mode. However, when the automatic driving system 132 determines that the driver is not at an increased risk of driving unsafely, the automatic driving system 132 may enable the vehicle 102 to transition from the autonomous driving mode to the manual driving mode. That is, the automatic driving system 132 may control the driving mode of the vehicle 102 based on the risk assessment profile.”)
Westbrook does not explicitly disclose limit an operation of the vehicle while the occupant drives the vehicle.
Coles teaches limit an operation of the vehicle while the occupant drives the vehicle (Paragraph 41 “Research is currently being directed to the use of blepharometric analysis and Al to identify biomarkers for other degradations to neurological brain function in conditions such as epilepsy (due to onset of seizures), ADHD, Parkinson’s, Alzheimer’s, traumatic brain injuries, concussion and motor neuron diseases, and aim to flag potential early signs to the driver (and also passengers) and/or with a recommendation to seek medical advice. The technology disclosed herein is readily applied in such scenarios.”, Paragraph 50-53 “n a fifth example, there is an additional control level at which driver assistance systems are not available or only available in a limited capacity, whilst an initial/baseline impairment level is determined. In a further example vehicle operation is restricted/limited until an initial/baseline impairment level is determined. [0051] In the examples above, intervention measures applied to driver assistance systems may include, for instance, one or more of the following:……• Adjusting steering damping.• Adjusting suspension settings. • Adjusting braking sensitivity.• Limiting vehicle maximum speed.”).
It would have been obvious to one of ordinary skill to modify Westbrook to include the teachings of limiting an operation of the vehicle for the purpose of improving safety.
Westbrook does not explicitly disclose executing a plurality of machine learning (ML) models, each exclusively corresponding to a specific type of data
Coimbra De Andrade executing a plurality of machine learning (ML) models, each exclusively corresponding to a specific type of data (Paragraph 21 “In some implementations, the driver behavior system 115 may process the video data using a third machine learning model to detect the behavior data. For example, the driver behavior system 115 may use one or more neural network models, SVM models, random forest classification models, and/or the like to process the video data to detect the behavior data. The behavior data may indicate, for example, that the driver is looking down (e.g., not looking at the road on which the vehicle 110 is traveling), that the driver is using a user device (e.g., to text while driving), that the driver is not wearing a seat belt, that the driver is not wearing an authorized uniform, and/or the like.”);
Wherein the multi-modal data comprises two or more different types of data corresponding to two or more ML models of the plurality of ML models (Paragraph 26 “In some implementations, processing the behavior data (e.g., using the fourth machine learning model) to determine the distraction data includes detecting a plurality of facial points of the driver from the behavior data and determining a yaw, roll, and/or pitch of a head of the driver based on the plurality of facial points. For example, the driver behavior system 115 may use a deep neural network (e.g., trained on images of faces annotated with facial orientation) to predict a yaw, roll, and/or pitch of the driver based on the plurality of facial points. In some implementations, the driver behavior system 115 may identify a forward looking position of the driver's face (e.g., a median yaw, roll, and/or pitch of the head of the driver) based on median values associated with the plurality of facial points and may determine a deviation from the forward looking position based on the predicted yaw, roll, and/or pitch of the driver.”),
Communicate outputs form the two or more ML models to an additional ML model, wherein the additional ML model combines the outputs (Paragraph 49 “s shown by reference number 220, the machine learning system may train a machine learning model using the set of observations and using one or more machine learning algorithms, such as a regression algorithm, a decision tree algorithm, a neural network algorithm, a k-nearest neighbor algorithm, a support vector machine algorithm, and/or the like. After training, the machine learning system may store the machine learning model as a trained machine learning model 225 to be used to analyze new observations.”)
It would have been obvious to one of ordinary skill to modify Westbrook to include the teachings of including multiple ML models for the purpose of improving safety.
As to claim 2 Westbrook discloses an apparatus wherein the processor is configured to receive two or more of images of the occupant from a camera, audio spoken by the occupant from a microphone, and weight distribution values from a seat used by the occupant (Paragraph 20).
As to claim 3 Westbrook discloses an apparatus wherein the processor is configured to extract eye motion data of the occupant from image data included in the multi-model data, and
wherein when the processor generates the prediction, the processor is configured to:
generate the prediction based on the eye motion data of the occupant extracted from the multi-modal data (Paragraph 32).
As to claim 9 the claim is interpreted and rejected as in claim 1.
As to claim 10 the claim is interpreted and rejected as in claim 2.
As to claim 11 the claim is interpreted and rejected as in claim 3.
As to claim 17 the claim is interpreted and rejected as in claim 1.
As to claim 18 the claim is interpreted and rejected as in claim 2.
As to claim 19 the claim is interpreted and rejected as in claim 3.
Claims 4,5,7,8, 12-13, 15-16, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Westbrook (US 2019/0056731) in view of Coles (WO 2023/193038), and Coimbra De Andrade (US 2022/0051038) as applied to claim 1 above, and in further view of Grant (US 10,525,979)
As to claim 4 Grant teaches an apparatus wherein the processor is configured to extract skeletal position data of the occupant from image data included in the multi-model data, and
wherein, when the processor generates the prediction, the processor is further configured to:
generate the prediction based on the skeletal position data of the occupant extracted from the obtained multi-modal data (Column 8 lines 66-Column 9 lines 1-3).
It would have been obvious to one of ordinary skill to modify Westbrook to include the teachings of generating prediction data based on skeletal position data for the purpose of assessing the occupants state.
As to claim 5 Grant teaches an apparatus wherein the processor is configured to extract paralinguistic features from audio data of the occupant from the multi-modal data, and
wherein, when the processor generates the prediction, the processor is further configured to:
generate the prediction based on the paralinguistic features of the occupant extracted from the multi-modal data (Column 11 lines 15-32).
As to claim 7 Grant teaches an apparatus wherein when the processor controls the vehicle to influence how the occupant drives the vehicle, the processor is further configured to: to display an alert on a user interface of the vehicle with a request to modify a behavior of the occupant(Column 4 lines 29-44).
As to claim 8 Grant teaches an apparatus wherein the processor controls the vehicle to influence how the occupant drives the vehicle, the processor is further configured to:
automatically modify one or more of a current speed and a current direction of the vehicle via an advanced driver assistance system (ADAS) (Column 4 lines 45-54).
As to claim 12 the claim is interpreted and rejected as in claim 4.
As to claim 13 the claim is interpreted and rejected as in claim 5.
As to claim 15 the claim is interpreted and rejected as in claim 7.
As to claim 16 the claim is interpreted and rejected as in claim 8.
As to claim 20 the claim is interpreted and rejected as in claim 4.
Claims 6,14 are rejected under 35 U.S.C. 103 as being unpatentable over Westbrook (US 2019/0056731) in view of Coles (WO 2023/193038), and Coimbra De Andrade (US 2022/0051038) as applied to claim 1 above, and in further view of Kaliouby (US 2018/0330178)
As to claim 6 Kaliouby teaches an apparatus wherein the additional ML model comprises a Bayesian model, and the processor is configured to: execute the Bayesian model using at least one of eye motion data, body motion data, audio data, or seat pressure data included in the multi-modal data (Paragraph 6, 119). It would have been obvious to one of ordinary skill to modify Westbrook to include the teachings of using a Bayesian model for the purpose of detecting the drivers cognitive state.
As to claim 14 the claim is interpreted and rejected as in claim 6.
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IMRAN K MUSTAFA whose telephone number is (571)270-1471. The examiner can normally be reached Mon-Fri 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James J Lee can be reached at 571-270-5965. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
IMRAN K. MUSTAFA
Primary Examiner
Art Unit 3668
/IMRAN K MUSTAFA/ Primary Examiner, Art Unit 3668 4/4/2026
Prosecution Insights