DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
This action is in response to the Applicant’s filing on July 30, 2026. Claims 1-8 and 10-23 are pending and examined below.
Response to Arguments
The previous rejections of claims 1-8 and 10-23 under 35 U.S.C. 103 are withdrawn in consideration of amended independent claims 1, 10 and 17. However, new rejections of claims 1-8 and 10-23 under 35 U.S.C. 103 are set forth below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2017/0045885 by Okumura et al. (herein after “Okumura”), U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”), and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 1, Kim discloses an autonomous driving apparatus comprising: an external camera having a field of view around a vehicle and configured to acquire image data; a radar having a field of sensing around the vehicle and configured to acquire radar data (Kim ¶ [0045]-[0046]: First, if sensor information is acquired from one or more sensors of the autonomous vehicle, the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information, ultrasound sensor information, but the scope of the present disclosure is not limited thereto, and may include any sensor information or its processed value acquired from any sensors installed on the autonomous vehicle; S10 in Fig. 2); and
a controller (Kim ¶ [0039]: By referring to FIG. 1, the autonomous driving control device 100 may include a memory 110 for storing instructions to remotely control the autonomous vehicle capable of switching between the autonomous driving mode and the manual driving mode, and a processor 120 for performing processes corresponding to the instructions in the memory 110 to remotely control the autonomous vehicle capable of switching between the autonomous driving mode and the manual driving mode) configured to determine whether a condition for control authority transfer of the vehicle to a pre-registered remote controller (Kim ¶ [0060]: by referring to FIG. 4, the autonomous driving control device 100 may perform a process of transmitting the request information for the remote control to the remote control service providing server, to thereby allow the remote control service providing server to confirm the specific remote vehicle 200, selectable by the remote control service providing server, which is available among pre-registered remote vehicles; 200 in Fig. 4) is satisfied (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2) based on at least one of the image data or the radar data during autonomous driving of the vehicle (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; Kim ¶ [0050]: on condition that the autonomous driving control device 100 allows the autonomous vehicle to travel on the autonomous driving mode or on the manual driving mode according to the driving environment, the autonomous driving control device 100 may perform one of a process of determining whether the autonomous driving control device fails to establish the driving plan by using the detected driving environment while on the autonomous driving mode and a process of determining whether the autonomous driving control device fails to change from the autonomous driving mode to the manual driving mode by using the detected driving environment; Fig. 2),
wherein the controller is configured to: determine whether the vehicle normally drives (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2) based on at least one of the image data or the radar data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2);
determine that the condition is satisfied when the driving of the vehicle is in an abnormal state (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2)
transmit a remote control request to the pre-registered remote controller (Kim ¶ [0060]: by referring to FIG. 4, the autonomous driving control device 100 may perform a process of transmitting the request information for the remote control to the remote control service providing server, to thereby allow the remote control service providing server to confirm the specific remote vehicle 200, selectable by the remote control service providing server, which is available among pre-registered remote vehicles; 200 in Fig. 4) when the condition is satisfied (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 and S40 in Fig. 2);
transfer a control authority to the pre-registered remote controller when a response to the remote control request is received from the pre-registered remote controller (Kim ¶ [0062]: if approval information for the remote control is acquired from the remote control service providing server or the specific remote vehicle 200, the autonomous driving control device 100 may perform a process of transmitting the sensor information to the specific remote vehicle 200 over the wireless communication, to thereby allow the specific remote vehicle 200 to be synchronized with the autonomous vehicle; Kim ¶ [0065]: if at least one remote driving control signal is acquired which is to be used by the remote driver 210 for controlling the specific remote vehicle 200 by referring to the sensor information, the autonomous driving control device 100 may perform a process of allowing the autonomous vehicle to be driven according to the remote driving control signal); and
.
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim fails to particularly disclose a condition is satisfied when a driver of the vehicle does not respond for a predetermined time to a control authority transfer to the driver; stop the vehicle when the response is not received from the pre-registered remote controller, and wherein the controller is further configured to, when some of a plurality of sensors involved in autonomous driving of the vehicle fail, transfer partial control authority to the pre- registered remote controller so that functions related to the failed sensors are performed by the pre-registered remote controller, while functions related to normally operating sensors are performed by the vehicle.
However, Okumura, in the same field endeavor, teaches wherein the controller (Okumura: computing device 100 in Fig. 1) is configured to: determine that the condition is satisfied when the driving of the vehicle is in an abnormal state (Okumura ¶ [0025]: If an unexpected driving environment is identified, operation of the vehicle 200 can switch to remote operation mode) and a driver of the vehicle does not respond for a predetermined time to a control authority transfer to the driver (Okumura ¶ [0033]: the vehicle 200 can be configured so that the driver is given the opportunity to affirmatively elect to retain control of the vehicle 200 and enter it into manual mode, and if the driver does not so elect within a predefined number of seconds, then the remote operator is contacted and control of the vehicle 200 passes automatically to the remote operator).
Further, Iwamoto, in the same field of endeavor, teaches stop the vehicle (Iwamoto ¶ [0059]: When an abnormality occurs in the remote assistance system 4, the remote assistance cannot be provided to the autonomous driving vehicle 1, or the accuracy of the remote assistance is reduced. Then, when an abnormality is detected in the remote assistance system 4, the autonomous driving system 10 executes “retracting processing” for safely retracting the autonomous driving vehicle 1; Iwamoto ¶ [0060]: A “target retracting position PE” is a target stop position when causing the autonomous driving vehicle 1 to stop by the retracting processing. The target retracting position PE may be set at a safe position on a road) when the response is not received from the pre-registered remote controller (Iwamoto ¶ [0056]: abnormalities in the remote assistance system 4 include a “functional failure” in which a function of the remote assistance system 4 is lost. An example of the functional failure of the remote assistance system 4 is a communication disruption. For example, when trouble occurs in the communication network 3, a communication disruption may occur. Another example of the functional failure in the remote assistance system 4 is a defect (down) of the remote assistance device 2. Yet another example of the functional failure in the remote assistance system 4 is a defect of the communication device mounted on the autonomous driving vehicle 1. When the functional failure occurs in the remote assistance system 4, it is impossible to provide the remote assistance to the autonomous driving vehicle 1).
Finally, Lacaze, in the same field of endeavor, teaches wherein the controller (Lacaze: onboard controller 110-1 and computing device 110-2 in Fig. 1) is further configured to, when some of a plurality of sensors involved in autonomous driving of the vehicle fail (Lacaze ¶ [0025]: the remote operator safety station can be configured to allow intervention primarily or only when the autonomous vehicle experiences an exception condition (e.g., deviation from normal operation, sensor failure, unexpected obstacle, etc.) or emergency condition), transfer partial control authority to the pre- registered remote controller so that functions related to the failed sensors are performed by the pre-registered remote controller, while functions related to normally operating sensors are performed by the vehicle (Lacaze ¶ [0026]: the remote operator safety station can provide a human operator with a set of control modalities (e.g., partial and/or upper-hierarchy commands and/or discrete options), which can intertwine full autonomy by the autonomous vehicle with partial or high-level tele-operation and/or other remote-control methods. In some embodiments, the human operator can choose to use one or more of these capabilities to at least partly control or maneuver (or decide not to move) the autonomous vehicle; Lacaze ¶ [0031]: Steer with autonomous speed control: The operator may choose to drive in an area where the autonomous vehicle is not usually allowed to drive; however, the operator may allow the autonomous vehicle to set the speed at which it travels. Alternatively or additionally, in some embodiments, the operator can allow the autonomous vehicle to use its obstacle avoidance system to stop the autonomous vehicle if there is an obstacle (e.g., pedestrian) and/or if the obstacle avoidance system otherwise detects a problem with the driving of the autonomous vehicle by the human operator via the remote safety station. In contrast to tele-operation (e.g., where the human operator completely and remotely controls the operation of the autonomous vehicle), the autonomous vehicle's autonomous systems and the remote operator share control over safe operation of the autonomous vehicle; Lacaze ¶ [0032]: Manual speed control with autonomous steering: In this modality, the operator controls the speed while the autonomous vehicle controls the steering. For example, this modality may be useful if the operations require the autonomous vehicle to cross intersections for which the sensor suite of the autonomous vehicle cannot warrantee safety (e.g., if the sensors cannot see far enough or are blocked). The autonomous vehicle can continue to steer autonomously while having the operator dictate speed).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim to include the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make these modifications in order to allow a driver to be undisturbed if they would rather not manually control a vehicle that can no longer operate autonomously (Okumura ¶ [0003]), to appropriately control an autonomous vehicle when an abnormality occurs in a remote assistance system (Iwamoto ¶ [0005]), and to allow a remote operator to temporarily, fully, partially, and/or periodically control an autonomous vehicle when the autonomous vehicle encounters an exception condition or emergency situation (Lacaze ¶ [0023]-[0024]).
Regarding claim 2, Kim discloses wherein the controller (Kim: 100 in Fig. 1) determines whether the vehicle departs from a lane (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; C in Fig. 3) based on the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2), and when the vehicle departs from the lane, determines that the driving of the vehicle is abnormal (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
Regarding claim 10, Kim discloses an autonomous driving apparatus comprising: at least one memory configured to store a program for autonomous driving of a vehicle; and at least one processor configured to execute the stored program (Kim ¶ [0039]: the autonomous driving control device 100 may include a memory 110 for storing instructions to remotely control the autonomous vehicle capable of switching between the autonomous driving mode and the manual driving mode, and a processor 120 for performing processes corresponding to the instructions in the memory 110 to remotely control the autonomous vehicle capable of switching between the autonomous driving mode and the manual driving mode; 110 and 120 in Fig. 1), wherein the processor is configured to:
determine whether the vehicle normally drives (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2) based on at least one of image data acquired by an external camera provided in the vehicle, radar data acquired by a radar provided in the vehicle, or light detection and ranging (LiDAR) data acquired by a LiDAR provided in the vehicle (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; Kim ¶ [0050]: on condition that the autonomous driving control device 100 allows the autonomous vehicle to travel on the autonomous driving mode or on the manual driving mode according to the driving environment, the autonomous driving control device 100 may perform one of a process of determining whether the autonomous driving control device fails to establish the driving plan by using the detected driving environment while on the autonomous driving mode and a process of determining whether the autonomous driving control device fails to change from the autonomous driving mode to the manual driving mode by using the detected driving environment; Fig. 2);
determine that a condition for control authority transfer of the vehicle to a pre-registered remote controller (Kim ¶ [0060]: by referring to FIG. 4, the autonomous driving control device 100 may perform a process of transmitting the request information for the remote control to the remote control service providing server, to thereby allow the remote control service providing server to confirm the specific remote vehicle 200, selectable by the remote control service providing server, which is available among pre-registered remote vehicles; 200 in Fig. 4) is satisfied when the driving of the vehicle is abnormal (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2) ;
transmit a remote control request to the pre-registered remote controller (Kim ¶ [0060]: by referring to FIG. 4, the autonomous driving control device 100 may perform a process of transmitting the request information for the remote control to the remote control service providing server, to thereby allow the remote control service providing server to confirm the specific remote vehicle 200, selectable by the remote control service providing server, which is available among pre-registered remote vehicles; 200 in Fig. 4) when the condition is satisfied (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 and S40 in Fig. 2);
transfer a control authority to the pre-registered remote controller when a response to the remote control request is received from the pre-registered remote controller (Kim ¶ [0062]: if approval information for the remote control is acquired from the remote control service providing server or the specific remote vehicle 200, the autonomous driving control device 100 may perform a process of transmitting the sensor information to the specific remote vehicle 200 over the wireless communication, to thereby allow the specific remote vehicle 200 to be synchronized with the autonomous vehicle; Kim ¶ [0065]: if at least one remote driving control signal is acquired which is to be used by the remote driver 210 for controlling the specific remote vehicle 200 by referring to the sensor information, the autonomous driving control device 100 may perform a process of allowing the autonomous vehicle to be driven according to the remote driving control signal); and
.
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera, radar, and LiDAR sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim fails to particularly disclose a condition is satisfied when a driver of the vehicle does not respond for a predetermined time to a control authority transfer to the driver; stop the vehicle when the response is not received from the pre-registered remote controller, and wherein the processor is further configured to, when some of a plurality of sensors involved in autonomous driving of the vehicle fail, transfer partial control authority to the pre- registered remote controller so that functions related to the failed sensors are performed by the pre-registered remote controller, while functions related to normally operating sensors are performed by the vehicle.
However, Okumura, in the same field endeavor, teaches determine that a condition for control authority transfer of the vehicle to a remote controller is satisfied when the driving of the vehicle is abnormal (Okumura ¶ [0025]: If an unexpected driving environment is identified, operation of the vehicle 200 can switch to remote operation mode) and a driver of the vehicle does not respond for a predetermined time to a control authority transfer to the driver (Okumura ¶ [0033]: the vehicle 200 can be configured so that the driver is given the opportunity to affirmatively elect to retain control of the vehicle 200 and enter it into manual mode, and if the driver does not so elect within a predefined number of seconds, then the remote operator is contacted and control of the vehicle 200 passes automatically to the remote operator).
Further, Iwamoto, in the same field of endeavor, teaches stop the vehicle (Iwamoto ¶ [0059]: When an abnormality occurs in the remote assistance system 4, the remote assistance cannot be provided to the autonomous driving vehicle 1, or the accuracy of the remote assistance is reduced. Then, when an abnormality is detected in the remote assistance system 4, the autonomous driving system 10 executes “retracting processing” for safely retracting the autonomous driving vehicle 1; Iwamoto ¶ [0060]: A “target retracting position PE” is a target stop position when causing the autonomous driving vehicle 1 to stop by the retracting processing. The target retracting position PE may be set at a safe position on a road) when the response is not received from the pre-registered remote controller (Iwamoto ¶ [0056]: abnormalities in the remote assistance system 4 include a “functional failure” in which a function of the remote assistance system 4 is lost. An example of the functional failure of the remote assistance system 4 is a communication disruption. For example, when trouble occurs in the communication network 3, a communication disruption may occur. Another example of the functional failure in the remote assistance system 4 is a defect (down) of the remote assistance device 2. Yet another example of the functional failure in the remote assistance system 4 is a defect of the communication device mounted on the autonomous driving vehicle 1. When the functional failure occurs in the remote assistance system 4, it is impossible to provide the remote assistance to the autonomous driving vehicle 1).
Finally, Lacaze, in the same field of endeavor, teaches wherein the processor (Lacaze: onboard controller 110-1 and computing device 110-2 in Fig. 1) is further configured to, when some of a plurality of sensors involved in autonomous driving of the vehicle fail (Lacaze ¶ [0025]: the remote operator safety station can be configured to allow intervention primarily or only when the autonomous vehicle experiences an exception condition (e.g., deviation from normal operation, sensor failure, unexpected obstacle, etc.) or emergency condition), transfer partial control authority to the pre- registered remote controller so that functions related to the failed sensors are performed by the pre-registered remote controller, while functions related to normally operating sensors are performed by the vehicle (Lacaze ¶ [0026]: the remote operator safety station can provide a human operator with a set of control modalities (e.g., partial and/or upper-hierarchy commands and/or discrete options), which can intertwine full autonomy by the autonomous vehicle with partial or high-level tele-operation and/or other remote-control methods. In some embodiments, the human operator can choose to use one or more of these capabilities to at least partly control or maneuver (or decide not to move) the autonomous vehicle; Lacaze ¶ [0031]: Steer with autonomous speed control: The operator may choose to drive in an area where the autonomous vehicle is not usually allowed to drive; however, the operator may allow the autonomous vehicle to set the speed at which it travels. Alternatively or additionally, in some embodiments, the operator can allow the autonomous vehicle to use its obstacle avoidance system to stop the autonomous vehicle if there is an obstacle (e.g., pedestrian) and/or if the obstacle avoidance system otherwise detects a problem with the driving of the autonomous vehicle by the human operator via the remote safety station. In contrast to tele-operation (e.g., where the human operator completely and remotely controls the operation of the autonomous vehicle), the autonomous vehicle's autonomous systems and the remote operator share control over safe operation of the autonomous vehicle; Lacaze ¶ [0032]: Manual speed control with autonomous steering: In this modality, the operator controls the speed while the autonomous vehicle controls the steering. For example, this modality may be useful if the operations require the autonomous vehicle to cross intersections for which the sensor suite of the autonomous vehicle cannot warrantee safety (e.g., if the sensors cannot see far enough or are blocked). The autonomous vehicle can continue to steer autonomously while having the operator dictate speed).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim to include the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make these modifications in order to allow a driver to be undisturbed if they would rather not manually control a vehicle that can no longer operate autonomously (Okumura ¶ [0003]), to appropriately control an autonomous vehicle when an abnormality occurs in a remote assistance system (Iwamoto ¶ [0005]), and to allow a remote operator to temporarily, fully, partially, and/or periodically control an autonomous vehicle when the autonomous vehicle encounters an exception condition or emergency situation (Lacaze ¶ [0023]-[0024]).
Regarding claim 11, Kim discloses wherein the processor (Kim: 120 in Fig. 1) determines whether the vehicle departs from a lane (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; C in Fig. 3) based on the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2), and when the vehicle departs from the lane, determines that the driving of the vehicle is abnormal (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera, radar, and LiDAR sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
Claims 3-4 and 12 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2017/0045885 by Okumura et al. (herein after “Okumura”), U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”) and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”), further in view of U.S. Patent Application Publication No. US 2021/0339741 by Rezvan Behbahani et al. (herein after “Rezvan Behbahani”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 3, Kim discloses wherein the controller (Kim: 100 in Fig. 1) determines a degree of risk of collision between the vehicle and a nearby vehicle (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; A in Fig. 3) based on at least one of the radar data or the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2), and (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2), and wherein the nearby vehicle is detected based on at least one of the radar data or the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim differs from the present invention in that it does not particularly disclose when the degree of risk of collision exceeds a reference level.
However, Rezvan Behbahani, in the same field of endeavor, teaches wherein the controller determines a degree of risk of collision between the vehicle and a nearby vehicle (Rezvan Behbahani ¶ [0017]: the system may determine a first risk or probability of collision between the autonomous vehicle and the object using the estimated locations) based on at least one of the radar data or the image data (Rezvan Behbahani ¶ [0016]: the system may fuse image data, lidar data, and radar data and utilize the fused data to detect the objects, classify the objects, and predict outcomes related to the objects), and when the degree of risk of collision exceeds a reference level (Rezvan Behbahani ¶ [0018]: The system may then determine whether the first probability of collision meets or exceeds a threshold), determines that the driving of the vehicle is abnormal (Rezvan Behbahani ¶ [0020]: the system may be unable to determine a set of safety requirements that maintains a safety rating above a desired safety threshold within a threshold period of time. In these cases, the system may contact or otherwise alert a remote operator to manually operate or drive the vehicle around the object or situation that is causing the policy violation or potentially unsafe or policy violating scenario).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the determination of collision risk of Rezvan Behbahani with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification in order to constrain operations of an autonomous vehicle so that the vehicle operates safely to avoid potential collisions (Rezvan Behbahani ¶ [0012]).
Regarding claim 4, Kim discloses further comprising a light detection and ranging (LiDAR) having a field of sensing around the vehicle to acquire LiDAR data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2), wherein the controller determines a degree of risk of collision based on at least one of the LiDAR data or the image data (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; A in Fig. 3), and (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2), and wherein the nearby vehicle is detected based on at least one of the LiDAR data or the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim differs from the present invention in that it does not particularly disclose when the degree of risk of collision exceeds a reference level.
However, Rezvan Behbahani, in the same field of endeavor, teaches further comprising a light detection and ranging (LiDAR) having a field of sensing around the vehicle to acquire LiDAR data (Rezvan Behbahani ¶ [0016]: the system may fuse image data, lidar data, and radar data and utilize the fused data to detect the objects, classify the objects, and predict outcomes related to the objects), wherein the controller determines a degree of risk of collision based on at least one of the LiDAR data or the image data (Rezvan Behbahani ¶ [0017]: the system may determine a first risk or probability of collision between the autonomous vehicle and the object using the estimated locations), and when the degree of risk of collision exceeds a reference level (Rezvan Behbahani ¶ [0018]: The system may then determine whether the first probability of collision meets or exceeds a threshold), determines that the driving of the vehicle is abnormal (Rezvan Behbahani ¶ [0020]: the system may be unable to determine a set of safety requirements that maintains a safety rating above a desired safety threshold within a threshold period of time. In these cases, the system may contact or otherwise alert a remote operator to manually operate or drive the vehicle around the object or situation that is causing the policy violation or potentially unsafe or policy violating scenario).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the determination of collision risk of Rezvan Behbahani with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification in order to constrain operations of an autonomous vehicle so that the vehicle operates safely to avoid potential collisions (Rezvan Behbahani ¶ [0012]).
Regarding claim 12, Kim discloses wherein the processor (Kim: 100 in Fig. 1) determines a degree of risk of collision between the vehicle and a nearby vehicle (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; A in Fig. 3) based on at least one of the radar data, the image data, or the LiDAR data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2), and (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2), and wherein the nearby vehicle is detected based on at least one of the radar data, the image data, or the LiDAR data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim differs from the present invention in that it does not particularly disclose when the degree of risk of collision exceeds a reference level.
However, Rezvan Behbahani, in the same field of endeavor, teaches wherein the processor determines a degree of risk of collision between the vehicle and a nearby vehicle (Rezvan Behbahani ¶ [0017]: the system may determine a first risk or probability of collision between the autonomous vehicle and the object using the estimated locations) based on at least one of the radar data, the image data, or the LiDAR data (Rezvan Behbahani ¶ [0016]: the system may fuse image data, lidar data, and radar data and utilize the fused data to detect the objects, classify the objects, and predict outcomes related to the objects), and when the degree of risk of collision exceeds a reference level (Rezvan Behbahani ¶ [0018]: The system may then determine whether the first probability of collision meets or exceeds a threshold), determines that the driving of the vehicle is abnormal (Rezvan Behbahani ¶ [0020]: the system may be unable to determine a set of safety requirements that maintains a safety rating above a desired safety threshold within a threshold period of time. In these cases, the system may contact or otherwise alert a remote operator to manually operate or drive the vehicle around the object or situation that is causing the policy violation or potentially unsafe or policy violating scenario).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the determination of collision risk of Rezvan Behbahani with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification in order to constrain operations of an autonomous vehicle so that the vehicle operates safely to avoid potential collisions (Rezvan Behbahani ¶ [0012]).
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2017/0045885 by Okumura et al. (herein after “Okumura”), U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”), and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”), further in view of U.S. Patent Application Publication No. US 2022/0169265 by Wu et al. (herein after “Wu”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 5, Kim fails to particularly disclose wherein the controller determines whether the vehicle is suddenly steered based on an output of a steering angle sensor provided in the vehicle, and when the vehicle is suddenly steered, determines that the driving of the vehicle is abnormal.
However, Wu, in the same field of endeavor, teaches wherein the controller (Wu: On-board equipment in Fig. 2) determines whether the vehicle is suddenly steered (Wu ¶ [0118]: Since the first control instruction is a control instruction from the driver who drives the vehicle, and the data of the second sensor group include vehicle dynamics data, which can reflect the driving state of the vehicle, it can be determined whether the driver's driving behavior is abnormal according to the first control instruction and the data of at least one sensor in the second sensor group. For example, if the vehicle has experienced abnormal events such as sharp turning, emergency braking and is fast overtaking, it means that the driver's behavior is abnormal) based on an output of a steering angle sensor provided in the vehicle (Wu ¶ [0038]: the second sensor group includes a wheel speed sensor, a speed sensor, an acceleration sensor and/or a steering angle sensor), and when the vehicle is suddenly steered, determines that the driving of the vehicle is abnormal (Wu ¶ [0118]: Since the first control instruction is a control instruction from the driver who drives the vehicle, and the data of the second sensor group include vehicle dynamics data, which can reflect the driving state of the vehicle, it can be determined whether the driver's driving behavior is abnormal according to the first control instruction and the data of at least one sensor in the second sensor group. For example, if the vehicle has experienced abnormal events such as sharp turning, emergency braking and is fast overtaking, it means that the driver's behavior is abnormal).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the sudden steering determination and steering angle sensor of Wu with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification to allow an autonomous driving system to update a planning and control algorithm to better cater to the driving habits of a driver thereby improving the driving experience for the driver without compromising the reliability of planning and decision-making of autonomous driving (Wu ¶ [0060]).
Regarding claim 13, Kim fails to particularly disclose wherein the processor determines whether the vehicle is suddenly steered based on an output of a steering angle sensor provided in the vehicle, and when the vehicle is suddenly steered, determines that the driving of the vehicle is abnormal.
However, Wu, in the same field of endeavor, teaches wherein the processor (Wu: 201 in Fig. 2) determines whether the vehicle is suddenly steered (Wu ¶ [0118]: Since the first control instruction is a control instruction from the driver who drives the vehicle, and the data of the second sensor group include vehicle dynamics data, which can reflect the driving state of the vehicle, it can be determined whether the driver's driving behavior is abnormal according to the first control instruction and the data of at least one sensor in the second sensor group. For example, if the vehicle has experienced abnormal events such as sharp turning, emergency braking and is fast overtaking, it means that the driver's behavior is abnormal) based on an output of a steering angle sensor provided in the vehicle (Wu ¶ [0038]: the second sensor group includes a wheel speed sensor, a speed sensor, an acceleration sensor and/or a steering angle sensor), and when the vehicle is suddenly steered, determines that the driving of the vehicle is abnormal (Wu ¶ [0118]: Since the first control instruction is a control instruction from the driver who drives the vehicle, and the data of the second sensor group include vehicle dynamics data, which can reflect the driving state of the vehicle, it can be determined whether the driver's driving behavior is abnormal according to the first control instruction and the data of at least one sensor in the second sensor group. For example, if the vehicle has experienced abnormal events such as sharp turning, emergency braking and is fast overtaking, it means that the driver's behavior is abnormal).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the sudden steering determination and steering angle sensor of Wu with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification to allow an autonomous driving system to update a planning and control algorithm to better cater to the driving habits of a driver thereby improving the driving experience for the driver without compromising the reliability of planning and decision-making of autonomous driving (Wu ¶ [0060]).
Claims 6-8 and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2017/0045885 by Okumura et al. (herein after “Okumura”), U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”), and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”), further in view of U.S. Patent Application Publication No. US 2018/0284759 by Michalakis et al. (herein after “Michalakis”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 6, Kim fails to particularly disclose wherein the controller determines a control authority return condition based on at least one of the image data or the radar data, and when the control authority return condition is satisfied, transfers the control authority from the remote controller to the vehicle.
However, Michalakis, in the same field of endeavor, teaches wherein the controller (Michalakis: electronic control unit 102 in Fig. 1) determines a control authority return condition based on at least one of the image data or the radar data (Michalakis ¶ [0048]: the one more driver condition sensors may include biologic sensors such as, for example, heart rate monitors, breathing monitors, blood pressure monitors, and the like; Michalakis ¶ [0049]: the one or more driver condition sensors 130 may include one or more cameras to determine a position of the driver's eyes or head to determine driver awareness/availability; Michalakis ¶ [0035]: the one or more driving condition sensors 120 may include radar sensors and/or lidar sensors (using radio or light detection, respectively) that can help identify objects (such as other vehicles) in the vicinity of the vehicle 100), and when the control authority return condition is satisfied, transfers the control authority from the remote controller to the vehicle (Michalakis ¶ [0042]: the vehicle 100 may from time to time encounter driving environments, expected or unexpected, in which autonomous operation is inappropriate, for example, a construction zone, an obstruction in the middle of the road, heavy traffic, inclement weather, autonomous vehicle restricted driving zones, and the like. In addition, certain road patterns may be deemed too challenging for the autonomous driving mode, such as an intersection where there is no clear line of sight, a mountainous region with winding roads, or a dense urban environment. The electronic control unit 102 can detect that the vehicle 100 is in such a driving environment or will be in such a driving environment at some future time based on driving condition signal received from the one or more driving condition sensors 120; Michalakis ¶ [0047]: When it is determined that the driver is later available to take control of the vehicle, control of the vehicle may automatically be transferred from the remote operator to the driver. In yet further embodiments, if the driver is still unavailable, but the vehicle 100 is no longer driving in a driving environment that is inappropriate for autonomous driving, control of the vehicle can be returned to the autonomous driving mode).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the return condition of Michalakis. A person of ordinary skill in the art would be motivated to make this modification to allow a vehicle to switch vehicle control from an autonomous driving mode to a remote operator in light of a future handover situation that does not interrupt the vehicle’s travel (Michalakis ¶ [0003]).
Regarding claim 7, Kim discloses wherein the condition (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode) further includes (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2).
It is noted Kim differs from the present invention in that it does not particularly disclose a driver's abnormal state of the vehicle.
However, Michalakis, teaches wherein the control authority transfer condition (Michalakis ¶ [0047]: In instances wherein it is determined that the autonomous driving mode of the vehicle 100 will terminate, control of the vehicle 100 may be transferred to one of the driver, if the driver is available, or a remote operator when the driver is unavailable) further includes at least one of a driver's abnormal state of the vehicle (Michalakis ¶ [0047]: the driver may be determined to be unavailable if the driver is asleep or distracted. In such case, the electronic control unit 102 will automatically transfer control of the vehicle 100 to the remote operator over the network interface hardware 110) or a limit situation of the autonomous driving (Michalakis ¶ [0042]: the vehicle 100 may from time to time encounter driving environments, expected or unexpected, in which autonomous operation is inappropriate, for example, a construction zone, an obstruction in the middle of the road, heavy traffic, inclement weather, autonomous vehicle restricted driving zones, and the like. In addition, certain road patterns may be deemed too challenging for the autonomous driving mode, such as an intersection where there is no clear line of sight, a mountainous region with winding roads, or a dense urban environment. The electronic control unit 102 can detect that the vehicle 100 is in such a driving environment or will be in such a driving environment at some future time based on driving condition signal received from the one or more driving condition sensors 120; Michalakis ¶ [0044]: the electronic control unit 102, based on information received from the one or more driving condition sensors 120 may be able to determine that an autonomous driving mode of the vehicle 100 will terminate at some future time based on an upcoming driving environment that is inappropriate for the autonomous driving mode).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the transfer condition including a driver’s abnormal state of Michalakis with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification to allow a vehicle to switch vehicle control from an autonomous driving mode to a remote operator in light of a future handover situation that does not interrupt the vehicle’s travel (Michalakis ¶ [0003]).
Regarding claim 8, Kim fails to particularly disclose wherein the controller determines the driver's abnormal state based on an output of at least one of an internal camera provided in the vehicle to capture the driver or a driver sensor configured to acquire biosignals of the driver.
However, Michalakis, in the same field endeavor, teaches wherein the controller (Michalakis: electronic control unit 102 in Fig. 1) determines the driver's abnormal state (Michalakis ¶ [0047]: the driver may be determined to be unavailable if the driver is asleep or distracted. In such case, the electronic control unit 102 will automatically transfer control of the vehicle 100 to the remote operator over the network interface hardware 110) based on an output of at least one of an internal camera provided in the vehicle to capture the driver (Michalakis ¶ [0049]: the one or more driver condition sensors 130 may include one or more cameras to determine a position of the driver's eyes or head to determine driver awareness/availability) or a driver sensor configured to acquire biosignals of the driver (Michalakis ¶ [0048]: the one more driver condition sensors may include biologic sensors such as, for example, heart rate monitors, breathing monitors, blood pressure monitors, and the like).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, the partial vehicle control based on sensor failure of Lacaze, and the driver’s abnormal state of Michalakis to further include the internal camera and biosignal sensors of Michalakis with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification to allow a vehicle to switch vehicle control from an autonomous driving mode to a remote operator in light of a future handover situation that does not interrupt the vehicle’s travel (Michalakis ¶ [0003]).
Regarding claim 14, Kim fails to particularly disclose wherein the processor determines a control authority return condition based on at least one of the image data, the LiDAR data, or the radar data, and when the control authority return condition is satisfied, transfers the control authority from the remote controller to the vehicle.
However, Michalakis, in the same field of endeavor, teaches wherein the processor (Michalakis: 105 in Fig. 1) determines a control authority return condition based on at least one of the image data, the LiDAR data, or the radar data (Michalakis ¶ [0048]: the one more driver condition sensors may include biologic sensors such as, for example, heart rate monitors, breathing monitors, blood pressure monitors, and the like; Michalakis ¶ [0049]: the one or more driver condition sensors 130 may include one or more cameras to determine a position of the driver's eyes or head to determine driver awareness/availability; Michalakis ¶ [0035]: the one or more driving condition sensors 120 may include radar sensors and/or lidar sensors (using radio or light detection, respectively) that can help identify objects (such as other vehicles) in the vicinity of the vehicle 100), and when the control authority return condition is satisfied, transfers the control authority from the remote controller to the vehicle (Michalakis ¶ [0042]: the vehicle 100 may from time to time encounter driving environments, expected or unexpected, in which autonomous operation is inappropriate, for example, a construction zone, an obstruction in the middle of the road, heavy traffic, inclement weather, autonomous vehicle restricted driving zones, and the like. In addition, certain road patterns may be deemed too challenging for the autonomous driving mode, such as an intersection where there is no clear line of sight, a mountainous region with winding roads, or a dense urban environment. The electronic control unit 102 can detect that the vehicle 100 is in such a driving environment or will be in such a driving environment at some future time based on driving condition signal received from the one or more driving condition sensors 120; Michalakis ¶ [0047]: When it is determined that the driver is later available to take control of the vehicle, control of the vehicle may automatically be transferred from the remote operator to the driver. In yet further embodiments, if the driver is still unavailable, but the vehicle 100 is no longer driving in a driving environment that is inappropriate for autonomous driving, control of the vehicle can be returned to the autonomous driving mode).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the return condition of Michalakis with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification to allow a vehicle to switch vehicle control from an autonomous driving mode to a remote operator in light of a future handover situation that does not interrupt the vehicle’s travel (Michalakis ¶ [0003]).
Regarding claim 15, Kim discloses wherein the condition (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode) further includes (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2).
It is noted Kim differs from the present invention in that it does not particularly disclose a driver's abnormal state of the vehicle.
However, Michalakis, in the same field of endeavor, teaches wherein the control authority transfer condition (Michalakis ¶ [0047]: In instances wherein it is determined that the autonomous driving mode of the vehicle 100 will terminate, control of the vehicle 100 may be transferred to one of the driver, if the driver is available, or a remote operator when the driver is unavailable) further includes at least one of a driver's abnormal state of the vehicle (Michalakis ¶ [0047]: the driver may be determined to be unavailable if the driver is asleep or distracted. In such case, the electronic control unit 102 will automatically transfer control of the vehicle 100 to the remote operator over the network interface hardware 110) or a limit situation of the autonomous driving (Michalakis ¶ [0042]: the vehicle 100 may from time to time encounter driving environments, expected or unexpected, in which autonomous operation is inappropriate, for example, a construction zone, an obstruction in the middle of the road, heavy traffic, inclement weather, autonomous vehicle restricted driving zones, and the like. In addition, certain road patterns may be deemed too challenging for the autonomous driving mode, such as an intersection where there is no clear line of sight, a mountainous region with winding roads, or a dense urban environment. The electronic control unit 102 can detect that the vehicle 100 is in such a driving environment or will be in such a driving environment at some future time based on driving condition signal received from the one or more driving condition sensors 120; Michalakis ¶ [0044]: the electronic control unit 102, based on information received from the one or more driving condition sensors 120 may be able to determine that an autonomous driving mode of the vehicle 100 will terminate at some future time based on an upcoming driving environment that is inappropriate for the autonomous driving mode).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the transfer condition including a driver’s abnormal state of Michalakis with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification to allow a vehicle to switch vehicle control from an autonomous driving mode to a remote operator in light of a future handover situation that does not interrupt the vehicle’s travel (Michalakis ¶ [0003]).
Regarding claim 16, Kim fails to particularly disclose wherein the processor determines the driver's abnormal state based on an output of at least one of an internal camera provided in the vehicle to capture the driver or a driver sensor configured to acquire biosignals of the driver.
However, Michalakis, in the same field endeavor, teaches wherein the processor (Michalakis: electronic control unit 102 in Fig. 1) determines the driver's abnormal state (Michalakis ¶ [0047]: the driver may be determined to be unavailable if the driver is asleep or distracted. In such case, the electronic control unit 102 will automatically transfer control of the vehicle 100 to the remote operator over the network interface hardware 110) based on an output of at least one of an internal camera provided in the vehicle to capture the driver (Michalakis ¶ [0049]: the one or more driver condition sensors 130 may include one or more cameras to determine a position of the driver's eyes or head to determine driver awareness/availability) or a driver sensor configured to acquire biosignals of the driver (Michalakis ¶ [0048]: the one more driver condition sensors may include biologic sensors such as, for example, heart rate monitors, breathing monitors, blood pressure monitors, and the like).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, the partial vehicle control based on sensor failure of Lacaze, and the driver’s abnormal state of Michalakis to further include the internal camera and biosignal sensors of Michalakis with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification to allow a vehicle to switch vehicle control from an autonomous driving mode to a remote operator in light of a future handover situation that does not interrupt the vehicle’s travel (Michalakis ¶ [0003]).
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”) and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 17, Kim discloses an autonomous driving control method comprising: acquiring image data through an external camera provided in a vehicle; acquiring radar data through a radar provided in the vehicle (Kim ¶ [0045]-[0046]: First, if sensor information is acquired from one or more sensors of the autonomous vehicle, the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information, ultrasound sensor information, but the scope of the present disclosure is not limited thereto, and may include any sensor information or its processed value acquired from any sensors installed on the autonomous vehicle; S10 in Fig. 2);
determining whether the vehicle normally drives (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2) based on at least one of the image data or the radar data during autonomous driving of the vehicle (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2);
determining that a condition for control authority transfer to a pre-registered remote controller (Kim ¶ [0060]: by referring to FIG. 4, the autonomous driving control device 100 may perform a process of transmitting the request information for the remote control to the remote control service providing server, to thereby allow the remote control service providing server to confirm the specific remote vehicle 200, selectable by the remote control service providing server, which is available among pre-registered remote vehicles; 200 in Fig. 4) is satisfied when the driving of the vehicle is abnormal (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings, and (iv) situations D and E where at least one movement of at least one nearby vehicle fails to be estimated due to traffic congestion, but the scope of the present disclosure is not limited thereto, and may include various situations where the driving plan is difficult to be acquired according to analysis of the driving environment; S30 in Fig. 2);
transmitting a remote control request to the pre-registered remote controller (Kim ¶ [0060]: by referring to FIG. 4, the autonomous driving control device 100 may perform a process of transmitting the request information for the remote control to the remote control service providing server, to thereby allow the remote control service providing server to confirm the specific remote vehicle 200, selectable by the remote control service providing server, which is available among pre-registered remote vehicles; 200 in Fig. 4) when the condition is satisfied (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 and S40 in Fig. 2);
transferring a control authority to the pre-registered remote controller when a response to the remote control request is received from the pre-registered remote controller (Kim ¶ [0062]: if approval information for the remote control is acquired from the remote control service providing server or the specific remote vehicle 200, the autonomous driving control device 100 may perform a process of transmitting the sensor information to the specific remote vehicle 200 over the wireless communication, to thereby allow the specific remote vehicle 200 to be synchronized with the autonomous vehicle; Kim ¶ [0065]: if at least one remote driving control signal is acquired which is to be used by the remote driver 210 for controlling the specific remote vehicle 200 by referring to the sensor information, the autonomous driving control device 100 may perform a process of allowing the autonomous vehicle to be driven according to the remote driving control signal); and
.
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim fails to particularly disclose stop the vehicle when the response is not received from the pre-registered remote controller, and the autonomous driving control method further comprises:
when some of a plurality of sensors involved in autonomous driving of the vehicle fail, transferring partial control authority to the pre-registered remote controller so that functions related to the failed sensors are performed by the pre-registered remote controller, while functions related to normally operating sensors are performed by the vehicle.
However, Iwamoto, in the same field of endeavor, teaches stop the vehicle (Iwamoto ¶ [0059]: When an abnormality occurs in the remote assistance system 4, the remote assistance cannot be provided to the autonomous driving vehicle 1, or the accuracy of the remote assistance is reduced. Then, when an abnormality is detected in the remote assistance system 4, the autonomous driving system 10 executes “retracting processing” for safely retracting the autonomous driving vehicle 1; Iwamoto ¶ [0060]: A “target retracting position PE” is a target stop position when causing the autonomous driving vehicle 1 to stop by the retracting processing. The target retracting position PE may be set at a safe position on a road) when the response is not received from the pre-registered remote controller (Iwamoto ¶ [0056]: abnormalities in the remote assistance system 4 include a “functional failure” in which a function of the remote assistance system 4 is lost. An example of the functional failure of the remote assistance system 4 is a communication disruption. For example, when trouble occurs in the communication network 3, a communication disruption may occur. Another example of the functional failure in the remote assistance system 4 is a defect (down) of the remote assistance device 2. Yet another example of the functional failure in the remote assistance system 4 is a defect of the communication device mounted on the autonomous driving vehicle 1. When the functional failure occurs in the remote assistance system 4, it is impossible to provide the remote assistance to the autonomous driving vehicle 1).
Further, Lacaze, in the same field of endeavor, teaches the autonomous driving control method further comprises:
when some of a plurality of sensors involved in autonomous driving of the vehicle fail (Lacaze ¶ [0025]: the remote operator safety station can be configured to allow intervention primarily or only when the autonomous vehicle experiences an exception condition (e.g., deviation from normal operation, sensor failure, unexpected obstacle, etc.) or emergency condition), transferring partial control authority to the pre-registered remote controller so that functions related to the failed sensors are performed by the pre-registered remote controller, while functions related to normally operating sensors are performed by the vehicle (Lacaze ¶ [0026]: the remote operator safety station can provide a human operator with a set of control modalities (e.g., partial and/or upper-hierarchy commands and/or discrete options), which can intertwine full autonomy by the autonomous vehicle with partial or high-level tele-operation and/or other remote-control methods. In some embodiments, the human operator can choose to use one or more of these capabilities to at least partly control or maneuver (or decide not to move) the autonomous vehicle; Lacaze ¶ [0031]: Steer with autonomous speed control: The operator may choose to drive in an area where the autonomous vehicle is not usually allowed to drive; however, the operator may allow the autonomous vehicle to set the speed at which it travels. Alternatively or additionally, in some embodiments, the operator can allow the autonomous vehicle to use its obstacle avoidance system to stop the autonomous vehicle if there is an obstacle (e.g., pedestrian) and/or if the obstacle avoidance system otherwise detects a problem with the driving of the autonomous vehicle by the human operator via the remote safety station. In contrast to tele-operation (e.g., where the human operator completely and remotely controls the operation of the autonomous vehicle), the autonomous vehicle's autonomous systems and the remote operator share control over safe operation of the autonomous vehicle; Lacaze ¶ [0032]: Manual speed control with autonomous steering: In this modality, the operator controls the speed while the autonomous vehicle controls the steering. For example, this modality may be useful if the operations require the autonomous vehicle to cross intersections for which the sensor suite of the autonomous vehicle cannot warrantee safety (e.g., if the sensors cannot see far enough or are blocked). The autonomous vehicle can continue to steer autonomously while having the operator dictate speed).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim to include the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto and the partial vehicle control based on sensor failure of Lacaze with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make these modifications in order to appropriately control an autonomous vehicle when an abnormality occurs in a remote assistance system (Iwamoto ¶ [0005]) and to allow a remote operator to temporarily, fully, partially, and/or periodically control an autonomous vehicle when the autonomous vehicle encounters an exception condition or emergency situation (Lacaze ¶ [0023]-[0024]).
Regarding claim 18, Kim discloses wherein the determining of whether the vehicle normally drives includes determining whether the vehicle departs from a lane (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; C in Fig. 3) based on the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2) and determining that the driving of the vehicle is abnormal when the vehicle departs from the lane (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
Claims 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”) and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”), further in view of U.S. Patent Application Publication No. US 2021/0339741 by Rezvan Behbahani et al. (herein after “Rezvan Behbahani”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 19, Kim discloses wherein the determining of whether the vehicle normally drives includes determining a degree of risk of collision between the vehicle and a nearby vehicle (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; A in Fig. 3) based on the radar data or the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2) and determining that the driving of the vehicle is abnormal (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2) and wherein the nearby vehicle is detected based on at least one of the radar data or the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim differs from the present invention in that it fails to particularly disclose when the degree of risk of collision exceeds a reference level.
However, Rezvan Behbahani, in the same field of endeavor, teaches wherein the determining of whether the vehicle normally drives includes determining a degree of risk of collision between the vehicle and a nearby vehicle (Rezvan Behbahani ¶ [0017]: the system may determine a first risk or probability of collision between the autonomous vehicle and the object using the estimated locations) based on the radar data or the image data (Rezvan Behbahani ¶ [0016]: the system may fuse image data, lidar data, and radar data and utilize the fused data to detect the objects, classify the objects, and predict outcomes related to the objects) and determining that the driving of the vehicle is abnormal (Rezvan Behbahani ¶ [0020]: the system may be unable to determine a set of safety requirements that maintains a safety rating above a desired safety threshold within a threshold period of time. In these cases, the system may contact or otherwise alert a remote operator to manually operate or drive the vehicle around the object or situation that is causing the policy violation or potentially unsafe or policy violating scenario) when the degree of risk of collision exceeds a reference level (Rezvan Behbahani ¶ [0018]: The system may then determine whether the first probability of collision meets or exceeds a threshold).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto and the partial vehicle control based on sensor failure of Lacaze to further include the determination of collision risk of Rezvan Behbahani with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification in order to constrain operations of an autonomous vehicle so that the vehicle operates safely to avoid potential collisions (Rezvan Behbahani ¶ [0012]).
Regarding claim 20, Kim discloses further comprising acquiring light detection and ranging (LiDAR) data through an LiDAR provided in the vehicle (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2), wherein the determining of whether the vehicle normally drives includes determining a degree of risk of collision between the vehicle and a nearby vehicle based on the LiDAR data or the image data (Kim ¶ [0051]: by referring to FIG. 3, a situation where the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode may include at least one of (i) a situation A where an expected path of the autonomous vehicle collides with that of a vehicle on an opposite side on a crossroad, (ii) a situation B where the autonomous vehicle is traveling on a roadway without lane markings, (iii) a situation C where the autonomous vehicle must travel in violation of the lane markings; A in Fig. 3) and determining that the driving of the vehicle is abnormal (Kim ¶ [0058]: if the autonomous driving control device fails to establish the driving plan while on the autonomous driving mode or fails to change from the autonomous driving mode to the manual driving mode, the autonomous driving control device 100 may allow the autonomous vehicle to be driven on a remote control mode; S30 in Fig. 2) and wherein the nearby vehicle is detected based on at least one of the LiDAR data or the image data (Kim ¶ [0045]-[0046]: the autonomous driving control device 100 may detect a driving environment of the autonomous vehicle by referring to the sensor information, at a step of S10. Herein, the sensor information may include at least part of camera sensor information, LiDAR sensor information, radar sensor information; S10 in Fig. 2).
It is noted that Kim does not explicitly describe the above elements in a single embodiment. However, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kim to combine the disclosed elements according to known methods to yield predictable results in order to create a comprehensive system that fuses data from camera and radar sensors to detect a driving environment with the goal of utilizing the driving environment data to determine if a condition for requesting remote control of an autonomous vehicle is satisfied. A person of ordinary skill in the art would be motivated to make this modification in order to prevent accidents that may result from a situation where the autonomous driving vehicle cannot easily determine a driving plan (Kim ¶ [0009] and ¶ [0035]).
It is further noted that Kim differs from the present invention in that it does not particularly disclose when the degree of risk of collision exceeds a reference level.
However, Rezvan Behbahani, in the same field of endeavor, teaches further comprising acquiring light detection and ranging (LiDAR) data through an LiDAR provided in the vehicle (Rezvan Behbahani ¶ [0016]: the system may fuse image data, lidar data, and radar data and utilize the fused data to detect the objects, classify the objects, and predict outcomes related to the objects), wherein the determining of whether the vehicle normally drives includes determining a degree of risk of collision between the vehicle and a nearby vehicle based on the LiDAR data or the image data (Rezvan Behbahani ¶ [0017]: the system may determine a first risk or probability of collision between the autonomous vehicle and the object using the estimated locations) and determining that the driving of the vehicle is abnormal (Rezvan Behbahani ¶ [0020]: the system may be unable to determine a set of safety requirements that maintains a safety rating above a desired safety threshold within a threshold period of time. In these cases, the system may contact or otherwise alert a remote operator to manually operate or drive the vehicle around the object or situation that is causing the policy violation or potentially unsafe or policy violating scenario) when the degree of risk of collision exceeds a reference level (Rezvan Behbahani ¶ [0018]: The system may then determine whether the first probability of collision meets or exceeds a threshold).
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto and the partial vehicle control based on sensor failure of Lacaze to further include the determination of collision risk of Rezvan Behbahani with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make this modification in order to constrain operations of an autonomous vehicle so that the vehicle operates safely to avoid potential collisions (Rezvan Behbahani ¶ [0012]).
Claims 21-22 are rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2017/0045885 by Okumura et al. (herein after “Okumura”), U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”), and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”), further in view of U.S. Patent Application Publication No. US 2024/0059323 by Gerrese et al. (herein after “Gerrese”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 21, Kim fails to particularly disclose wherein the controller is further configured to make an emergency call to a pre-designated target when the response is not received from the pre-registered remote controller.
However, Gerrese, in the same field of endeavor, teaches wherein the controller is further configured to make an emergency call to a pre-designated target (Gerrese ¶ [0088]-[0089]: In response to detecting the emergency event 204, the AV 102 can automatically generate a response 206 to the emergency event 204. The response 206 can include performing an action(s), generating an alert(s), sending a signal(s) and/or communication(s), establishing a call, changing a behavior and/or mode of the AV 102, outputting information, requesting help, etc. For example, the response 206 can include, without limitation, … requesting help from one or more third parties (e.g., emergency responders, remote assistance entities, etc.) … sending a notification to one or more devices (e.g., to a device of a passenger, to a device of an emergency contact, to a device of an emergency responder, to a traffic device, etc.)) when the response is not received from the pre-registered remote controller (Gerrese ¶ [0020]: the systems and techniques described herein can enable AVs to detect when they have encountered (e.g., experience, have entered, are involved in, etc.) a degraded state (also referred to herein as an emergency event) such as, for example and without limitation, an accident, a system or component malfunction, a medical/health event or emergency, an error state, an inoperable state, an event having a threshold risk of danger, a collision, a critical and/or urgent condition and/or situation, a crash, etc). Examiner interprets the system of Gerrese to respond to a degraded state (emergency events) by making an emergency call to an emergency responder or an emergency contact which would include a degraded state (emergency event) where a communication connection is inoperable, malfunctioning or in an error state.
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the emergency call to a pre-designated target of Gerrese with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make these modifications in order to allow an AV to determine how to respond to a situation based on one or more factors and respond to the situation accordingly (Gerrese ¶ [0021]).
Regarding claim 22, Kim fails to particularly disclose wherein the processor is further configured to: make an emergency call to a pre-designated target when the response is not received from the pre-registered remote controller.
However, Gerrese, in the same field of endeavor, teaches wherein the processor is further configured to: make an emergency call to a pre-designated target (Gerrese ¶ [0088]-[0089]: In response to detecting the emergency event 204, the AV 102 can automatically generate a response 206 to the emergency event 204. The response 206 can include performing an action(s), generating an alert(s), sending a signal(s) and/or communication(s), establishing a call, changing a behavior and/or mode of the AV 102, outputting information, requesting help, etc. For example, the response 206 can include, without limitation, … requesting help from one or more third parties (e.g., emergency responders, remote assistance entities, etc.) … sending a notification to one or more devices (e.g., to a device of a passenger, to a device of an emergency contact, to a device of an emergency responder, to a traffic device, etc.)) when the response is not received from the pre-registered remote controller (Gerrese ¶ [0020]: the systems and techniques described herein can enable AVs to detect when they have encountered (e.g., experience, have entered, are involved in, etc.) a degraded state (also referred to herein as an emergency event) such as, for example and without limitation, an accident, a system or component malfunction, a medical/health event or emergency, an error state, an inoperable state, an event having a threshold risk of danger, a collision, a critical and/or urgent condition and/or situation, a crash, etc). Examiner interprets the system of Gerrese to respond to a degraded state (emergency events) by making an emergency call to an emergency responder or an emergency contact which would include a degraded state (emergency event) where a communication connection is inoperable, malfunctioning or in an error state.
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the transfer conditions of Okumura, the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto, and the partial vehicle control based on sensor failure of Lacaze to further include the emergency call to a pre-designated target of Gerrese with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make these modifications in order to allow an AV to determine how to respond to a situation based on one or more factors and respond to the situation accordingly (Gerrese ¶ [0021]).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable U.S. Patent Application Publication No. US 2020/0241526 by Kim et al. (herein after “Kim”), in view of U.S. Patent Application Publication No. US 2022/0363291 by Iwamoto et al. (herein after “Iwamoto”) and U.S. Patent Application Publication No. US 2025/0036131 by Lacaze et al. (herein after “Lacaze”), further in view of U.S. Patent Application Publication No. US 2024/0059323 by Gerrese et al. (herein after “Gerrese”).
Note: Text written in bold typeface is claim language from the instant application. Text written in normal typeface are comments made by the Examiner and/or passages from the prior art reference(s).
Regarding claim 23, Kim fails to particularly disclose further comprising: making an emergency call to a pre-designated target when the response is not received from the pre-registered remote controller.
However, Gerrese, in the same field of endeavor, teaches further comprising: making an emergency call to a pre-designated target (Gerrese ¶ [0088]-[0089]: In response to detecting the emergency event 204, the AV 102 can automatically generate a response 206 to the emergency event 204. The response 206 can include performing an action(s), generating an alert(s), sending a signal(s) and/or communication(s), establishing a call, changing a behavior and/or mode of the AV 102, outputting information, requesting help, etc. For example, the response 206 can include, without limitation, … requesting help from one or more third parties (e.g., emergency responders, remote assistance entities, etc.) … sending a notification to one or more devices (e.g., to a device of a passenger, to a device of an emergency contact, to a device of an emergency responder, to a traffic device, etc.)) when the response is not received from the pre-registered remote controller (Gerrese ¶ [0020]: the systems and techniques described herein can enable AVs to detect when they have encountered (e.g., experience, have entered, are involved in, etc.) a degraded state (also referred to herein as an emergency event) such as, for example and without limitation, an accident, a system or component malfunction, a medical/health event or emergency, an error state, an inoperable state, an event having a threshold risk of danger, a collision, a critical and/or urgent condition and/or situation, a crash, etc). Examiner interprets the system of Gerrese to respond to a degraded state (emergency events) by making an emergency call to an emergency responder or an emergency contact which would include a degraded state (emergency event) where a communication connection is inoperable, malfunctioning or in an error state.
Therefore, given the teachings as a whole, it would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the autonomous driving control device of Kim modified by the condition for stopping a vehicle when an abnormality occurs in a remote assistance system of Iwamoto and the partial vehicle control based on sensor failure of Lacaze to further include the emergency call to a pre-designated target of Gerrese with a reasonable expectation of success. A person of ordinary skill in the art would be motivated to make these modifications in order to allow an AV to determine how to respond to a situation based on one or more factors and respond to the situation accordingly (Gerrese ¶ [0021]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to the applicant’s disclosure:
US 20230138112 by Gross et al. discloses an apparatus and methods related to a controller configured to monitor incident risk levels of multiple independently governed autonomous vehicles remote from the controller simultaneously; and, in response to an unsafe incident risk level for one or more vehicles, take control of vehicles having an unsafe incident risk level, to restore a safe incident risk level; and, in response to determining incident risk has been restored to a safe level, returning control to the autonomous vehicles.
JP 2018142921 by Yaginuma discloses a predetermined condition based on a degree of risk of an autonomous driving vehicle exceeding a threshold value. The degree of risk is calculated based on various parameters such as LDW ( Lane Departure Warning ), FCW ( Forward collision warning ), sudden steering, sudden braking, time zone, place, and climate. Alternatively, it may be calculated based on a danger prediction algorithm generated by artificial intelligence based on deep learning.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS P LANGHORNE whose telephone number is (571)272-5670. The examiner can normally be reached M-F 8:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Antonucci can be reached at (313) 446-6519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.P.L./Examiner, Art Unit 3666
/ANNE MARIE ANTONUCCI/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights