DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 9-11, 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Coimbra De Andrade (US 2022/0051038) in view of Westbrook (US 2019/0056731)
As to claim 1 Coimbra De Andrade discloses an apparatus comprising:
a processor configured to execute a plurality of machine learning (ML) models, each exclusively corresponding to a specific type of data (Paragraph 21 “In some implementations, the driver behavior system 115 may process the video data using a third machine learning model to detect the behavior data. For example, the driver behavior system 115 may use one or more neural network models, SVM models, random forest classification models, and/or the like to process the video data to detect the behavior data. The behavior data may indicate, for example, that the driver is looking down (e.g., not looking at the road on which the vehicle 110 is traveling), that the driver is using a user device (e.g., to text while driving), that the driver is not wearing a seat belt, that the driver is not wearing an authorized uniform, and/or the like.”);
receive multi-modal sensor data associated with an occupant driving a vehicle form a sensor of the vehicle (Paragraph 14 “As further shown in FIG. 1A, and by reference number 125, the driver behavior system 115 may receive vehicle data from the vehicle device 105. For example, the vehicle device 105 (e.g., using the VTU) may send the vehicle data to the driver behavior system 115 (e.g. via the cellular network). The vehicle data may include identifying characteristics of the vehicle 110 (e.g., speed, acceleration, location, orientation, braking, and/or the like) during capture of the video data.”), wherein the multi-modal data comprises two or more different types of data corresponding to two or more ML models of the plurality of ML models (Paragraph 21 “As shown in FIG. 1D, and by reference number 140, the driver behavior system 115 may process the video data associated with the identified driver to detect behavior data identifying a behavior of the driver. In some implementations, the driver behavior system 115 may process the video data using a third machine learning model to detect the behavior data. For example, the driver behavior system 115 may use one or more neural network models, SVM models, random forest classification models, and/or the like to process the video data to detect the behavior data.”[In this case the system can useone or more neural networks ,SVM models,… to detect behavior data])
communicate outputs from the two or more ML models to an additional ML model, wherein the additional ML model combines the outputs to generate a prediction (Paragraph 33 “In some implementations, the driver behavior system 115 may aggregate the distraction data and the vehicle data to calculate the distraction score. The driver behavior system 115 may identify one or more distracted behaviors (e.g., driver looking down, user device present, user device being used, and/or the like) based on the distraction data, and determine a distraction sub-score for each distracted behavior (e.g., that indicates severity of each distracted behavior) based on the vehicle data (e.g., that indicates a speed of the vehicle 110, an acceleration of the vehicle 110, and/or the like). For example, the driver behavior system 115 may determine that a first distracted behavior (e.g., looking down) has a first distraction sub-score that is a function of the speed of the vehicle 110, a second distracted behavior (e.g., using a user device) has a second distraction sub-score that is a function of the speed of the vehicle 110 and the length of time the user device is used, and/or the like. Accordingly, the driver behavior system 115 may combine (e.g., sum; average; use a set of custom rules and/or nonlinear functions; use a machine learning model, such as one or more neural network models, SVM models, random forest classification models, and/or the like) the distraction sub-scores together to determine the distraction score.”[Here outputs of the two machine learning models which are the distraction data[Fig. 1E], and the Policy Complaince Data[Fig. 1F], are combined using an two or more ML models to generate a prediction of a level of distraction of the driver as well as a policy compliance score)
Coimbra De Andrade does not explicitly disclose generate a prediction, that the occupant of the vehicle has attention deficit hyperactivity disorder (ADHD) based on the multi-modal data using a machine learning model (Paragraph 27 “For example, beta brainwaves may be generated when the driver is actively driving, i.e., alert, attentive, engaged in problem solving, exercising judgment/decision making, and engaged in a focused mental activity. Too much beta brainwaves may be indicative of excessive amounts of adrenaline, increased anxiety, increased arousal, an inability to relax, and/or increased stress levels; whereas, too little beta brainwaves may be indicative of ADHD, daydreaming, depression, and/or poor cognition. An ideal amount of beta brainwaves may promote conscious focus, increased memory, and/or increased problem solving abilities.”, Paragraph 4 “For example, a system may generate a risk assessment for a driver indicating a mental state of the driver, what risks the driver takes based on the mental state, and under what circumstances such risks are taken, for example. In some aspects, the risk assessment profile may be generated using machine learning algorithms, such that the risk assessment profile may dynamically evolve over time.”),
based on the prediction, control the vehicle to influence how the occupant drives the vehicle(Paragraph 33 “In some aspects, when the automatic driving system 132 determines that the driver is at an increased risk of driving unsafely, the automatic driving system 132 may prevent the vehicle 102 from transitioning from the autonomous driving mode to the manual driving mode. However, when the automatic driving system 132 determines that the driver is not at an increased risk of driving unsafely, the automatic driving system 132 may enable the vehicle 102 to transition from the autonomous driving mode to the manual driving mode. That is, the automatic driving system 132 may control the driving mode of the vehicle 102 based on the risk assessment profile.”)
It would have been obvious to one of ordinary skill to modify Coimbra De Andrade to include the teachings of generating a prediction of the condition of the driver for the purpose of improving safety.
As to claim 2 Westbrook teaches an apparatus wherein the processor is configured to receive two or more of images of the occupant from a camera, audio spoken by the occupant from a microphone, and weight distribution values from a seat used by the occupant (Paragraph 20).
As to claim 3 Westbrook teaches an apparatus wherein the processor is configured to extract eye motion data of the occupant from image data included in the multi-model data, and
wherein when the processor generates the prediction, the processor is configured to:
generate the prediction based on the eye motion data of the occupant extracted from the multi-modal data (Paragraph 32).
As to claim 9 the claim is interpreted and rejected as in claim 1.
As to claim 10 the claim is interpreted and rejected as in claim 2.
As to claim 11 the claim is interpreted and rejected as in claim 3.
As to claim 17 the claim is interpreted and rejected as in claim 1.
As to claim 18 the claim is interpreted and rejected as in claim 2.
As to claim 19 the claim is interpreted and rejected as in claim 3.
Claims 4,5,7,8, 12-13, 15-16, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Coimbra De Andrade (US 2022/0051038) in view of Westbrook (US 2019/0056731) as applied to claim 1 above, and in further view of Grant (US 10,525,979)
As to claim 4 Grant teaches an apparatus wherein the processor is configured to extract skeletal position data of the occupant from image data included in the multi-model data, and
wherein, when the processor generates the prediction, the processor is further configured to:
generate the prediction based on the skeletal position data of the occupant extracted from the obtained multi-modal data (Column 8 lines 66-Column 9 lines 1-3).
It would have been obvious to one of ordinary skill to modify Westbrook to include the teachings of generating prediction data based on skeletal position data for the purpose of assessing the occupants state.
As to claim 5 Grant teaches an apparatus wherein the processor is configured to extract paralinguistic features from audio data of the occupant from the multi-modal data, and
wherein, when the processor generates the prediction, the processor is further configured to:
generate the prediction based on the paralinguistic features of the occupant extracted from the multi-modal data (Column 11 lines 15-32).
As to claim 7 Grant teaches an apparatus wherein when the processor controls the vehicle to influence how the occupant drives the vehicle, the processor is further configured to: to display an alert on a user interface of the vehicle with a request to modify a behavior of the occupant(Column 4 lines 29-44).
As to claim 8 Grant teaches an apparatus wherein the processor controls the vehicle to influence how the occupant drives the vehicle, the processor is further configured to:
automatically modify one or more of a current speed and a current direction of the vehicle via an advanced driver assistance system (ADAS) (Column 4 lines 45-54).
As to claim 12 the claim is interpreted and rejected as in claim 4.
As to claim 13 the claim is interpreted and rejected as in claim 5.
As to claim 15 the claim is interpreted and rejected as in claim 7.
As to claim 16 the claim is interpreted and rejected as in claim 8.
As to claim 20 the claim is interpreted and rejected as in claim 4.
Claims 6,14 are rejected under 35 U.S.C. 103 as being unpatentable over Coimbra De Andrade (US 2022/0051038) in view of Westbrook (US 2019/0056731) as applied to claim 1 above, and in further view of Kaliouby (US 2018/0330178)
As to claim 6 Kaliouby teaches an apparatus wherein the additional ML model comprises a Bayesian model, and the processor is configured to: execute the Bayesian model using at least one of eye motion data, body motion data, audio data, or seat pressure data included in the multi-modal data (Paragraph 6, 119). It would have been obvious to one of ordinary skill to modify Westbrook to include the teachings of using a Bayesian model for the purpose of detecting the drivers cognitive state.
As to claim 14 the claim is interpreted and rejected as in claim 6.
Response to Arguments
Applicant's arguments filed have been fully considered but they are not persuasive.
On page 3 of the applicants arguments applicants argue that the inputs to the combining steps are sub-scores derived from a single ML models’ distraction output, together with vehicle telemetry data such as speed and acceleration.
The examiner respectfully disagrees with the applicants arguments. The applicant is reminded that the examiner interprets the claim with the broadest reasonable interpretation. De Andrade disclose communicate outputs from the two or more ML models to an additional ML model, wherein the additional ML model combines the outputs to generate a prediction (Paragraph 33 “In some implementations, the driver behavior system 115 may aggregate the distraction data and the vehicle data to calculate the distraction score. The driver behavior system 115 may identify one or more distracted behaviors (e.g., driver looking down, user device present, user device being used, and/or the like) based on the distraction data, and determine a distraction sub-score for each distracted behavior (e.g., that indicates severity of each distracted behavior) based on the vehicle data (e.g., that indicates a speed of the vehicle 110, an acceleration of the vehicle 110, and/or the like). For example, the driver behavior system 115 may determine that a first distracted behavior (e.g., looking down) has a first distraction sub-score that is a function of the speed of the vehicle 110, a second distracted behavior (e.g., using a user device) has a second distraction sub-score that is a function of the speed of the vehicle 110 and the length of time the user device is used, and/or the like. Accordingly, the driver behavior system 115 may combine (e.g., sum; average; use a set of custom rules and/or nonlinear functions; use a machine learning model, such as one or more neural network models, SVM models, random forest classification models, and/or the like) the distraction sub-scores together to determine the distraction score.”[Here outputs of the two machine learning models which are the distraction data[Fig. 1E], and the Policy Complaince Data[Fig. 1F], are combined using an two or more ML models to generate a prediction of a level of distraction of the driver as well as a policy compliance score). De Andrade discloses of using one or more neural networks [machine learning models] to determine a distraction score as well as a policy compliance score. As shown in Figures 1E and 1F the fourth and fifth machine learning models output the distraction data as well as the policy compliance data. Paragraph 33 states that the calculation of the scores can be fed into one or more neural networks to determine the level of distraction of the driver.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IMRAN K MUSTAFA whose telephone number is (571)270-1471. The examiner can normally be reached Mon-Fri 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James J Lee can be reached at 571-270-5965. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
IMRAN K. MUSTAFA
Primary Examiner
Art Unit 3668
/IMRAN K MUSTAFA/ Primary Examiner, Art Unit 3668
10/9/2026
