DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on April 9, 2026 is considered by
the examiner.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 29, 2026 has been entered.
Response to Amendment
Applicant submitted amendments and remarks on April 29, 2026. Therein, Applicant submitted substantive arguments. Claims 1 and 12 have been amended. No claims were added. Claims 3 and 17 were cancelled.
The submitted claims are considered below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 5, 12, 14, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto, et al. (U.S. Patent Application Publication No. 20230037467) in view of Williams, et al. (U.S. Patent No. 6906619).
Regarding claim 1, Yamamoto, et al. teaches: A method, performed by a device of vehicle, for providing a human-machine interface (HMI) mode, the method comprising: determining an occupant's state; (Steps (1713) - (1714), Fig. 7A, Paragraph [0262]: "After the attention information is presented in step (1713) or step (1714) in a mode corresponding to the driver behavior, that is, the driver's visual line direction, the processing proceeds to step (1715) [determining occupant's state].")
determining, based on the occupant's state, an HMI mode which is associated with a type of medium to be used for interaction with an occupant, a number of devices for interaction with the occupant, or a medium-specific output level corresponding to each media (Step (1714), Fig. 7A, Paragraph [0262]: "In step (1714), the in-vehicle system (10) presents attention information regarding the entry of the subject vehicle into the congestion section by sound through a speaker not illustrated provided in the HMI device (20). Note that in this case, when the driver is operating the terminal device (24) as a second task, presentation of the attention information by screen and/or sound output equipment provided in the terminal device (24) can also be executed [determining HMI mode associated with specific type of medium as a function of occupant's state].")
providing, based on the determined HMI mode, to the occupant information on a driving situation of the vehicle or a behavior that the vehicle will autonomously perform (Paragraph [0133]: "Due to this, the HMI device (20) notifies, by image display and/or sound, the subject vehicle passenger including the driver of the driving automation level that is being executed or is planned to be executed [provide occupant information about autonomous driving situation].")
wherein the occupant's state includes the occupant's attention level to an external situation of the vehicle (Paragraph [0134]: "The driving environment acquisition unit (252) acquires information related to the driving environment of the subject vehicle [occupant's attention level to external vehicle situation].").
Yamamoto, et al. does not teach wherein as the occupant's attention level to the external situation of the vehicle is varied based on a frequency or a duration of the occupant's gaze toward the outside of the vehicle, and in response to an increase in the occupant's attention level, the HMI mode is determined to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level, and wherein the determined HMI mode comprises: a first HMI mode configured to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level compared to other HMI modes, in response to a state in which the occupant continuously gazes outside of the vehicle, a second HMI mode configured to interact with the occupant using an intermediate variety of media types, an intermediate number of devices, or an intermediate medium-specific output level compared to other HMI modes, in response to a state in which the occupant intermittently gazes outside of the vehicle, or, a third HMI mode configured to interact with the occupant using a smaller variety of media types, a smaller number of devices, or a lower medium-specific output level compared to other HMI modes, in response to a state in which the occupant does not gaze outside of the vehicle.
In a similar field of endeavor (vehicle control based on visual attention by vehicle occupants), Williams, et al. teaches: wherein as the occupant's attention level to the external situation of the vehicle is varied based on a frequency or a duration of the occupant's gaze toward the outside of the vehicle, and in response to an increase in the occupant's attention level, the HMI mode is determined to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level, (Col. 5, lines 39-46: "…human-machine interfaces (HMI) (36) that serve to present the selected condition indicia. Such an HMI (36) can include one or more displays, graphic elements, lights, or the like, as well as audible information (including alarm tones as well as pre-recorded verbal instructions) and haptic sensations (such as a vibrating steering wheel, a vibrating accelerator pedal, and the like) [HMI mode - interactions with occupant]." ; Col. 5, lines 54-64: "So configured, such an apparatus can control the HMI (36) pursuant to a first mode of operation when the direction-of-visual-attention sensor (32) indicates that the predetermined person is likely to view a given predetermined vehicle condition (which predetermined vehicle condition correlates to a particular condition indicia) and pursuant to a second mode of operation when the direction-of-view-attention sensor (32) indicates that the predetermined person is not likely to view the predetermined vehicle condition. Additional modes of operation that correspond to yet other alternatives regarding the presentation of a given condition indicia can of course be added as desired and appropriate to a given application [HMI modes determined to interact with occupant using higher medium-specific output level]." ; Col. 6, lines 3-6: "When a potential hazard is detected (41), however, the system determines (43) whether the driver appears to be looking in the direction of the detected hazard [occupant's gaze toward outside of vehicle].")
and wherein the determined HMI mode comprises: a first HMI mode configured to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level compared to other HMI modes, in response to a state in which the occupant continuously gazes outside of the vehicle, a second HMI mode configured to interact with the occupant using an intermediate variety of media types, an intermediate number of devices, or an intermediate medium-specific output level compared to other HMI modes, in response to a state in which the occupant intermittently gazes outside of the vehicle, or, a third HMI mode configured to interact with the occupant using a smaller variety of media types, a smaller number of devices, or a lower medium-specific output level compared to other HMI modes, in response to a state in which the occupant does not gaze outside of the vehicle (Steps (43-47), Fig. 4, Col. 6, lines 28-39: "When this determination (43) reveals, however, that the driver is likely viewing (or, in some embodiments, will likely view in the immediate or near future) the potential hazard, the system can determine whether the driver has taken an action (45) in apparent response to the potential hazard [response to occupant intermittently gazing outside of vehicle; condition of hazard]. [...] When such an action has been detected, the system can determine to take no particular action (47) with respect to the provision of any cautionary warnings to the driver. When, however, no such action has yet been taken, the system can provide (46) a relatively low-level alert. For example, a low-level audible alarm can be provided or a relatively less-compelling graphic indicator can be provided on a heads-up, high-heads down, or other display for the driver. This lower level of cautionary warning accords with the previously determined information that the driver is already likely aware of the potential hazard [HMI mode in which intermediate medium-specific output level as a function of gazing].").
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify Yamamoto, et al. to include the teaching of Williams, et al. based on a reasonable expectation of success and motivation to improve the attention level of vehicle occupants with respect to hazardous driving conditions outside the vehicle (Williams, et al. Col. 1, lines 8-10).
Regarding claim 5, Yamamoto, et al. and Williams, et al. remain as applied to claim 1, and in a further embodiment, teach: the method of claim 1, wherein the HMI mode is determined to interact with the occupant using: at least one medium among a graphical user interface (GUI), an auditory user interface (AUI), and a physical user interface (PUI), in accordance with an output scheme determined for each medium (Paragraph [0123]: The display control unit controls the image and sound output operation of the HMI device and various information such as level related information. See also Yamamoto, et al. Paragraph [0124] stating the display control unit controls the information presentation depending on the automation level acquired by the automation level acquisition unit).
Regarding claim 12, Yamamoto, et al. teaches: A device for providing a human-machine interface (HMI) mode, the device comprising: a controller configured to determine an occupant's state in a vehicle, (Paragraph [0134]: "That is, in the HMI device (20), […] The second behavior acquisition unit (253) acquires driver behavior [controller to determine occupant's state in vehicle].")
determine, based on the occupant's state, an HMI mode which is associated with a type of media to be used for interaction with an occupant, a number of devices for interaction with the occupant, or a medium specific output level corresponding to each media (Paragraph [0133]: "The display command transmission unit (186) cause the HMI device (20) to execute presentation operation of the level-related information by transmitting display command information to the HMI control device (25) that controls the HMI device (20) [determine HMI mode]. Due to this, the HMI device (20) notifies, by image display and/or sound, the subject vehicle passenger including the driver of the driving automation level that is being executed or is planned to be executed [HMI mode is associated with type of media use for interaction with occupant].")
and provide, based on the determined HMI mode, to the occupant information on a driving situation of the vehicle or a behavior that the vehicle will autonomously perform (Paragraph [0133]: "Due to this, the HMI device (20) notifies, by image display and/or sound, the subject vehicle passenger including the driver of the driving automation level that is being executed or is planned to be executed [provide occupant information about autonomous driving situation].")
wherein the occupant's state includes the occupant's attention level to an external situation of the vehicle, (Paragraph [0134]: "The driving environment acquisition unit (252) acquires information related to the driving environment of the subject vehicle [occupant's attention level to external vehicle situation].").
Yamamoto, et al. does not teach wherein as the occupant's attention level to the external situation of the vehicle is varied based on a frequency or a duration of the occupant's gaze toward the outside of the vehicle, and in response to an increase in the occupant's attention level, the HMI mode is determined to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level, and wherein the determined HMI mode comprises: a first HMI mode configured to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level compared to other HMI modes, in response to a state in which the occupant continuously gazes outside of the vehicle, a second HMI mode configured to interact with the occupant using an intermediate variety of media types, an intermediate number of devices, or an intermediate medium-specific output level compared to other HMI modes, in response to a state in which the occupant intermittently gazes outside of the vehicle, or, a third HMI mode configured to interact with the occupant using a smaller variety of media types, a smaller number of devices, or a lower medium-specific output level compared to other HMI modes, in response to a state in which the occupant does not gaze outside of the vehicle.
In a similar field of endeavor (vehicle control based on visual attention by vehicle occupants), Williams, et al. teaches: wherein as the occupant's attention level to the external situation of the vehicle is varied based on a frequency or a duration of the occupant's gaze toward the outside of the vehicle, and in response to an increase in the occupant's attention level, the HMI mode is determined to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level, (Col. 5, lines 39-46: "…human-machine interfaces (HMI) (36) that serve to present the selected condition indicia. Such an HMI (36) can include one or more displays, graphic elements, lights, or the like, as well as audible information (including alarm tones as well as pre-recorded verbal instructions) and haptic sensations (such as a vibrating steering wheel, a vibrating accelerator pedal, and the like) [HMI mode - interactions with occupant]." ; Col. 5, lines 54-64: "…control the HMI (36) pursuant to a first mode of operation when the direction-of-visual-attention sensor (32) indicates that the predetermined person is likely to view a given predetermined vehicle condition (which predetermined vehicle condition correlates to a particular condition indicia) and pursuant to a second mode of operation when the direction-of-view-attention sensor (32) indicates that the predetermined person is not likely to view the predetermined vehicle condition. Additional modes of operation that correspond to yet other alternatives regarding the presentation of a given condition indicia can of course be added as desired and appropriate to a given application [HMI modes determined to interact with occupant using higher medium-specific output level]." ; Col. 6, lines 3-6: "When a potential hazard is detected (41), however, the system determines (43) whether the driver appears to be looking in the direction of the detected hazard [occupant's gaze toward outside of vehicle].")
wherein the determined HMI mode comprises: a first HMI mode configured to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level compared to other HMI modes, in response to a state in which the occupant continuously gazes outside of the vehicle, a second HMI mode configured to interact with the occupant using an intermediate variety of media types, an intermediate number of devices, or an intermediate medium-specific output level compared to other HMI modes, in response to a state in which the occupant intermittently gazes outside of the vehicle, or a third HMI mode configured to interact with the occupant using a smaller variety of media types, a smaller number of devices, or a lower medium-specific output level compared to other HMI modes, in response to a state in which the occupant does not gaze outside of the vehicle (Col. 6, lines 28-39: "When this determination (43) reveals, however, that the driver is likely viewing (or, in some embodiments, will likely view in the immediate or near future) the potential hazard, the system can determine whether the driver has taken an action (45) in apparent response to the potential hazard [response to occupant intermittently gazing outside of vehicle; condition of hazard]. [...] When such an action has been detected, the system can determine to take no particular action (47) with respect to the provision of any cautionary warnings to the driver. When, however, no such action has yet been taken, the system can provide (46) a relatively low-level alert. For example, a low-level audible alarm can be provided or a relatively less-compelling graphic indicator can be provided on a heads-up, high-heads down, or other display for the driver. This lower level of cautionary warning accords with the previously determined information that the driver is already likely aware of the potential hazard [HMI mode in which intermediate medium-specific output level as a function of gazing].").
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify Yamamoto, et al. to include the teaching of Williams, et al. based on a reasonable expectation of success and motivation to improve the attention level of vehicle occupants with respect to hazardous driving conditions outside the vehicle (Williams, et al. Col. 1, lines 8-10).
Regarding claim 14, Yamamoto, et al. and Williams, et al. remain as applied to claim 12, and in a further embodiment, teach: The device of claim 12, determine wherein the HMI mode is determined to interact with the occupant using at least one medium to be used to provide guide information to the occupant among a graphical user interface (GUI), an auditory user interface (AUI), and a physical user interface (PUI), in accordance with an output scheme determined for each medium (Paragraph ([0123]: The display control unit controls the image and sound output operation of the HMI device and various information such as level-related information. See also Yamamoto, et al. Paragraph ([0124]) stating the display control unit controls the information presentation depending on the automation level acquired by the automation level acquisition unit.).
Regarding claim 19, Yamamoto, et al. and Williams, et al. remain as applied to claim 12, and in a further embodiment, teach: A vehicle comprising the device of claim 12 (The in-vehicle system (10) includes a vehicle state sensor (11), an external state sensor (12), a surrounding monitoring sensor (13), a locator (14), a DCM (15), a navigation device (16), a driver state detection unit (17), a driving control device (18), and an HMI device (20) (Yamamoto, et al. Paragraph [0070]). See also Yamamoto, et al. Paragraph [0123]: The in vehicle system which includes the HMI device which provides guidance information through an image or audio.").
Claims 6, 8, 11, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto, et al. (U.S. Patent Application Publication No. 20230037467) and Williams, et al. (U.S. Patent No. 6906619) in view of Martin, et al. (U.S. Patent Application Publication No. 20190308555).
Regarding claim 6, the combination of Yamamoto, et al. and Williams, et al. does not teach the method of claim 5 wherein the output scheme for each medium is determined based on one or more tables in which a medium-specific output scheme or a medium specific output level is mapped to each of the plurality of predefined RMI modes.
In a similar field of endeavor (vehicle wheel torque adjustment based on occupant detection), Martin, et al. teaches: the method of claim 5, wherein the output scheme for each medium is determined based on one or more tables in which a medium-specific output scheme or a medium specific output level is mapped to each of the plurality of predefined RMI modes (Table 4 which displays various output levels and the corresponding mediums that will activate when that level is selected. See Paragraph [0050] stating that the output level corresponds to an intensity of an output signal and is based on the occupant alertness level.).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al. and Williams, et al. to include the teaching of Martin, et al. based on a reasonable expectation of success and motivation to improve the process of providing a haptic output to vehicle occupants (Martin, et al. Paragraph [0027]).
Regarding claim 8, the combination of Yamamoto, et al. and Williams, et al. does not teach the method of claim 5, wherein the output scheme for each medium includes one or more of whether to stop a task that has already been performed by a target device corresponding to the medium, a condition for the target device corresponding to the medium to output guidance information, an amount of information contained in the guidance information, and the number of the target device corresponding to the medium that is to provide the guidance information.
In a similar field of endeavor (vehicle wheel torque adjustment based on occupant detection), Martin, et al. teaches: The method of claim 5, wherein the output scheme for each medium includes one or more of whether to stop a task that has already been performed by a target device corresponding to the medium, a condition for the target device corresponding to the medium to output guidance information, an amount of information contained in the guidance information, and the number of the target device corresponding to the medium that is to provide the guidance information (Table 4 which displays at what output levels an audio, visual, or haptic signal is sent and how much information is sent. See also Paragraphs [0093]-[0094] stating that the outputs in Table 4 are examples and different haptic signals and messages may be used.).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al. and Williams, et al. to include the teaching of Martin, et al. based on a reasonable expectation of success and motivation to improve the process of providing a haptic output to vehicle occupants (Martin, at al. Paragraph [0027]).
Regarding claim 11, the combination of Yamamoto, et al. and Williams, et al. does not teach the method of claim 1, further comprising: providing guidance information to the occupant in a medium-specific output scheme corresponding to a preset HMI mode in response to determining that a plurality of occupants are present in the vehicle or that a predefined second reference position is not occupied.
In a similar field of endeavor (vehicle wheel torque adjustment based on occupant detection), Martin, et al. teaches: The method of claim 1, further comprising: providing guidance information to the occupant in a medium-specific output scheme corresponding to a preset HMI mode in response to determining that a plurality of occupants are present in the vehicle or that a predefined second reference position is not occupied (Paragraph [0015]: Stating that a second occupant alertness level may be calculated upon detecting a second occupant. See also Paragraph [0061] stating that the output level corresponds to the occupant alertness level.).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al. and Williams, et al. to include the teaching of Martin, et al. based on a reasonable expectation of success and motivation to improve the process of providing a haptic output to vehicle occupants (Martin, et al. Paragraph [0027]).
Regarding claim 15, the combination of Yamamoto, et al. and Williams, et al. does not teach the device of claim 12, further comprising: a storage configured to store one or more tables in which a medium-specific output scheme or a medium-specific output level is mapped to each of a plurality of predefined HMI modes.
In a similar field of endeavor (vehicle wheel torque adjustment based on occupant detection), Martin, et al. teaches: The device of claim 12, further comprising: a storage configured to store one or more tables in which a medium-specific output scheme or a medium-specific output level is mapped to each of a plurality of predefined HMI modes (Table 4 which displays various output levels and the corresponding mediums that will activate when that level is selected. See also Paragraph [0050] stating that the output level corresponds to an intensity of an output signal and is based on the occupant alertness level and Paragraph [0097] stating that a storage medium is used to store instructions.).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al. and Williams, et al. to include the teaching of Martin, et al. based on a reasonable expectation of success and motivation to improve the process of providing a haptic output to vehicle occupants (Martin, et al. Paragraph [0027]).
Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto, et al. (U.S. Patent Application Publication No. 20230037467), Williams, et al. (U.S. Patent No. 6906619), and Martin, et al. (U.S. Patent Application Publication No. 20190308555) in view of Tian, et al. (U.S. Patent Application Publication No. 20200231179).
Regarding claim 7, the combination of Yamamoto, et al., Williams, et al., and Martin, et al. does not teach the method of claim 6, wherein the output scheme for each medium is determined corresponding to an event that has occurred in the vehicle.
In a similar field of endeavor (vehicle guidance based on occupant parameters), Tian et al. teaches: The method of claim 6, wherein the output scheme for each medium is determined corresponding to an event that has occurred in the vehicle (Paragraph ([0143]: Guidance information will be provided if there is a possibility of an event occurring in the vehicle. See also Tian, et al. Paragraph [0074] which the HMI outputs the information as an image or sound.).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al., Williams, et al., and Martin, et al. to include the teaching of Tian, et al. based on a reasonable expectation of success and motivation to improve the process of prompting the occupant to pay attention and provide information for prompting to change the route according to the guidance information (Tian, et al. Paragraphs [0010] – [0011]).
The combination of Yamamoto, et al., Williams, et al., Martin, et al., and Tian, et al. does not teach mapping the output scheme to a specific table. However, since Tian, et al. has the ability to create prediction output model data in the form of a similar table like format (ref. Fig. 6, Paragraph [0088]), and it is obvious to one of the ordinary skill in the art to create the capability of an output scheme in the form of a table to properly inform and prepare the occupant in the vehicle of an unexpected event, this teaching would have made it obvious to modify the combination of Yamamoto, et al., Williams, et al., Martin, et al., and Tian, et al. to include an output scheme in the form of a specific table based on the motivation to improve the process of prompting the occupant to pay attention and provide information for prompting to change the route according to the guidance information.
Regarding claim 16, the combination of Yamamoto, et al., Williams, et al., and Martin, et al. does not teach the device of claim 15, wherein the controller is configured to: determine a medium-specific output scheme corresponding to the HMI mode using a table corresponding to an event occurring in the vehicle.
In a similar field of endeavor (vehicle guidance based on occupant parameters), Tian et al. teaches: The device of claim 15, wherein the controller is configured to: determine a medium-specific output scheme corresponding to the HMI mode corresponding to an event occurring in the vehicle (Paragraph [0143] Guidance information will be provided by the HMI if there is a possibility of an event occurring in the vehicle. See also Paragraph [0074] which the HMI outputs the guidance information as an image or sound).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al., Williams, et al., and Martin, et al. to include the teaching of Tian, et al. based on a reasonable expectation of success and motivation to improve the process of prompting the occupant to pay attention and provide information for prompting to change the route according to the guidance information (Tian, et al. Paragraph [0010] – [0011]).
The combination of Yamamoto, et al, Williams, et al., Martin, et al., and Tian, et al. does not teach mapping the output scheme to a specific table. However, since Tian, et al. has the ability to create prediction output model data in the form of a similar table like format (ref. Fig. 6, Paragraph [0088]), and it is obvious to one of the ordinary skill in the art to create the capability of an output scheme in the form of a table to properly inform and prepare the occupant in the vehicle of an unexpected event, this teaching would have made it obvious to modify the combination of Yamamoto, et al, Williams, et al., Martin, et al., and Tian, et al. to include an output scheme in the form of a specific table based on the motivation to improve the process of prompting the occupant to pay attention and provide information for prompting to change the route according to the guidance information.
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Yamamoto, et al. (U.S. Patent Application Publication No. 20230037467) and Williams, et al. (U.S. Patent No. 6906619) in view of Yamada (U.S. Patent No. 9783202).
Regarding claim 9, the combination of Yamamoto, et al. and Williams, et al. does not teach the method of claim 1, further comprising: determining a first reference position, which is a position occupied by a reference occupant, among a plurality of positions in the vehicle, and wherein the determining the occupant's state comprises: determining the occupant's state on the basis of occupant information acquired at the first reference position.
In a similar field of endeavor (vehicle occupant information acquisition and control), Yamada, et al. teaches: The method of claim 1, further comprising: determining a first reference position, which is a position occupied by a reference occupant, among a plurality of positions in the vehicle, and wherein the determining the occupant's state comprises: determining the occupant's state on the basis of occupant information acquired at the first reference position (Col. 4 lines 58-64: stating a method of capturing images of an occupant and determining the state of the occupant. The captured images are also used for facial recognition to identify occupants).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al. and Williams, et al. to include the teaching of Yamada based on a reasonable expectation of success and motivation to improve the process of controlling a vehicle based on vehicle occupant information (Yamada Col. 1, line 66 to Col. 2, lines 1-19).
Regarding claim 10, the combination of Yamamoto, et al. and Williams, et al. does not teach the method of claim 1, further comprising: guiding the occupant to a predefined second reference position; and determining whether the second reference position is occupied, and wherein the determining the occupant's state comprises: determining the occupant's state on the basis of occupant information acquired at the second reference position.
In a similar field of endeavor (vehicle occupant information acquisition and control), Yamada, et al. teaches: The method of claim 1, further comprising: guiding the occupant to a predefined second reference position; and determining whether the second reference position is occupied, and wherein the determining the occupant's state comprises: determining the occupant's state on the basis of occupant information acquired at the second reference position (Col. 12 lines 47-53 gives an example of identifying multiple occupants and retrieving respective conditions of the occupants. See also Col. 12 lines 8-16 stating riding occupants are displayed at the display panel.).
Therefore, it would have been obvious to one of the ordinary skill of the art before the effective filing date of the claimed invention to modify the combination of Yamamoto, et al. and Williams, et al. to include the teaching of Yamada based on a reasonable expectation of success and motivation to improve the process of controlling a vehicle based on vehicle occupant information (Yamada Col. 1, line 66 to Col. 2, lines 1-19).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 12 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant asserted that amended claims 1 and 12 were patentable over Goldman-Shenhar, et al. (U.S. Patent No. 9815481) in view of Toyoda (U.S. Patent No. 11262755) because the references did not meet the claim limitation “wherein the occupant's state includes the occupant's attention level to an external situation of the vehicle”. Please note that Yamamoto, et al. (U.S. Patent Application Publication No. 20230037467) was cited in order to teach these features. In Yamamoto, et al., the occupant’s state from outside the vehicle is determined through the use of “…the driving environment acquisition unit (252)”, which “…acquires information related to the driving environment of the subject vehicle” (Paragraph [0134]). Subsequently, it would have been obvious to combine Yamamoto, et al. with Williams, et al. (U.S. Patent No. 6906619) because Williams, et al. teaches the relationship between the occupant’s attention level to an external situation of the vehicle as a function of an occupant’s gaze (Col. 5, lines 39-46, Col. 6, lines 3-6) and the resulting changes to the vehicle’s HMI mode as a result of the determination (Col. 5, lines 54-64).
Applicant also asserted that amended claim 1 was patentable over Goldman-Shenhar, et al. (U.S. Patent No. 9815481) in view of Toyoda (U.S. Patent No. 11262755) because the references did not meet the claim limitation “and wherein as the occupant's attention level to the external situation of the vehicle increases, the HMI mode is determined to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level”. Please note that Williams, et al. (U.S. Patent No. 6906619) was cited in order to teach these features. In Williams, et al., the HMI setup in a vehicle includes various forms of media, including “…one or more displays, graphic elements, lights, or the like, as well as audible information (including alarm tones as well as pre-recorded verbal instructions) and haptic sensations (such as a vibrating steering wheel, a vibrating accelerator pedal, and the like)” (Col. 5, lines 39-46). As a result, the relationship between the HMI mode can change as a function of the gaze of the occupant, such as the condition in which “…a potential hazard is detected (41), however, the system determines (43) whether the driver appears to be looking in the direction of the detected hazard” (Col. 6, lines 3-6), which results in the control of “…the HMI (36) pursuant to a first mode of operation when the direction-of-visual-attention sensor (32) indicates that the predetermined person is likely to view a given predetermined vehicle condition (which predetermined vehicle condition correlates to a particular condition indicia)”, or a “…second mode of operation when the direction-of-view-attention sensor (32) indicates that the predetermined person is not likely to view the predetermined vehicle condition”, which indicates that a higher medium-specific output level with respect to the occupant can be determined when “…Additional modes of operation that correspond to yet other alternatives regarding the presentation of a given condition indicia can of course be added as desired and appropriate to a given application” (Col. 5, lines 54-64). Subsequently, it would have been obvious to combine Williams, et al. with Yamamoto, et al. because Yamamoto, et al. teaches an HMI device which determines an occupant’s state in a vehicle (Paragraph [0134]) and which HMI mode (and associated media type) can be associated with the behavior of an autonomous vehicle (Paragraphs [0133] – [0134]).
Therefore, it can be concluded that since the combination of Yamamoto, et al., and Williams, et al. reads on the claim limitations “wherein the occupant's state includes the occupant's attention level to an external situation of the vehicle” and “and wherein as the occupant's attention level to the external situation of the vehicle increases, the HMI mode is determined to interact with the occupant using a greater variety of media types, a larger number of devices, or a higher medium-specific output level”, as stated in amended claims 1 and 12, the arguments presented by the Applicant are not persuasive, and the rejection is maintained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Goldman-Shenhar, et al. (U.S. Patent No. 9815481) teaches a mode-determining module which contains a processor which can be used to determine an interaction mode corresponding to a driver of the autonomous-driving-capable vehicle.
Katz, et al. (U.S. Patent Application Publication No. 20200207358) teaches the process of determining the state of attentiveness of a driver with respect to an object and initiating actions based on the determined state of attentiveness of a driver.
Applicant is considered to have implicit knowledge of the entire disclosure once a reference has been cited. Therefore, any previously cited figures, columns and lines should not be considered to limit the references in any way. The entire reference must be taken as a whole; accordingly, the Examiner contends that the art supports the rejection of the claims and the rejection is maintained.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TORRENCE S MARUNDA II whose telephone number is (571)272-5172. The examiner can normally be reached Monday-Friday 8:00-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANGELA Y ORTIZ can be reached on 571-272-1206. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TORRENCE S MARUNDA II/Examiner, Art Unit 3663
/ANGELA Y ORTIZ/Supervisory Patent Examiner, Art Unit 3663
Prosecution Insights