Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 7/15/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claim 9 is objected to because of the following informalities:
Claim 9 teaches a “second electronic computing device [which] does not receive the video footage”. Based on the claim language of independent claims 1 and 8, it appears that the “second electronic computer device” of claim 9 should instead read the “first electronic computer device”, which is receiving first signals “from a second electronic computing device mounted to a vehicle” (as taught in independent claim 8).
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-8, 11-13, 16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Davis, et al. (U.S. Patent Application Pub. No. 2025/0222963) in view of Tal, et al. (U.S. Patent Application Pub. No. 2022/0020230).
Regarding Claim 1, Davis teaches: A system for responding to events (Davis, Para. 0004-0006, 0038-0039 – “an autonomous driving system” in communication to a “mission control computing system” for “identifying an emergency situation”, or event, and “triggering a first responder mode of the autonomous vehicle” to “initiating one or more actions”), comprising:
a vehicle (Davis, Para. 0004-0006 – “an autonomous vehicle”);
a camera mounted to the vehicle (Davis, Para. 0004-0006, 0025, 0059 – “one or more sensors positioned on a body of the autonomous vehicle” including “one or more cameras” capturing “visual sensor data”),
a first electronic computing device mounted to the vehicle (Davis, Para. 0044 – the “autonomous vehicle 100 may further include an autonomy computing system 204”, or first electronic computing device), the first electronic computing device including:
at least one processor (Davis, Para. 0050-0051 – where the “autonomy computing system 204” may be “any suitable computing device 300” which “includes a processor 314”); and
memory encoding instructions which, when executed by the at least one processor, cause the first electronic computing device (Davis, Para. 0004-0006, 0050-0053 – “autonomy computing system 204” may be “any suitable computing device 300” including a “memory device 318” including “one or more devices that enable information, such as executable instructions” executable by the “processor 314”) to:
process, by the at least one processor, the “based on the data collected from sensors 206”, including the “cameras”, “autonomy computing system 204 is configured to perform analysis of the sensor data”) and, based thereon:
(i) detect an event in the “the autonomy computing system 204 and, more specifically, perception and understanding module 203 senses the environment surrounding autonomous vehicle 100 by gathering and interpreting sensor data” to “identify an emergency situation, presence or absence of a personnel from the emergency services, victims requiring assistance, damaged property, etc.”); and
(ii) assign a predefined event type to the event (Davis, Para. 0046 – “identify an emergency situation, presence or absence of a personnel from the emergency services, victims requiring assistance, damaged property, etc.” by “one or more machine-learning algorithms trained to identify an emergency situation”, such that the emergency situations, or events, are predefined by the machine-learning algorithms);
transmit first signals to a second electronic computing device that is remote from the vehicle, the first signals indicating an occurrence of the predefined event type (Davis, Para. 0038, 0055, 0060 –“autonomous driving system 200… is communicatively coupled with mission control computing system 224” that is a “server computer device 401”, or a second electronic computing device, where “the server computing device 401 may receive sensor data, from the autonomous vehicle 100, for identifying an emergency situation and triggering or enabling the first responder mode of the autonomous vehicle 100”; where when the autonomous vehicle determines “presence of the emergency situation and absence of the emergency vehicle at the site of the emergency situation”, the vehicle triggers “a first responder mode” and sends “data corresponding to the emergency situation”, or first signals, to the “computing system (e.g., a computing device at mission control…)”);
process, by the at least one processor, second signals transmitted from the second electronic computing device (Davis, Para. 0040, 0055, 0060-0061 – “the mission control computing system 224 may transmit control commands or data to the autonomous vehicle 100, such as navigation commands, and travel trajectories to the autonomous vehicle 100”, or second signals, where the “autonomous vehicle” performs “the one or more actions to assist or protect the people”); and
based on the second signals, cause a device associated with the vehicle to respond to the detected event with a response (Davis, Para. 0024, 0040, 0055, 0060-0062 – “one or more actions may be initiated 508 to assist or protect people or a property at the site of the emergency situation”, where the “instructions to… perform the one or more actions to assist… at the site of the emergency situation may be received from the computing system”, that is “a computing device at mission control”, i.e. “mission control computing system 224”; for example “pulling the autonomous vehicle in a driving lane or a shoulder”, “displaying one or more messages on one or more display devices… to alert a passerby”, “turn blinkers or hazard lights on”, “use a loudspeaker to inform of the emergency situation”, etc.).
While Davis teaches a camera mounted to the vehicle, Davis does not specifically teach the camera being configured to capture video footage.
However, Tal teaches the camera being configured to capture video footage (Tal, Para. 0045, 0064, 0075 – a “device 101” mounted to a “vehicle's 102a windshield or a portion of the body” or “attached to the dashboard” including “an imager 500”, “e.g. camera”, for obtaining “captured video containing the individual frames”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Davis to include the camera being configured to capture video footage, as taught by Tal, in order to obtain additional data from video footage to improve event detection and response accuracy.
In regards to Claim 3, Davis in view of Tal teaches the system of Claim 1, and Davis further teaches wherein the device associated with the vehicle includes one or more of a camera (Davis, Para. 0040 – “Mission control 224 may send commands to the autonomous vehicle to transmit sensor data associated with one or more specific sensors 206”, where sensors 206 include “cameras 212”), a smartphone, a controller that controls a movement of the vehicle or a part of the vehicle (Davis, Para. 0042-0044, 0061 – “autonomous vehicle 100 may further include a vehicle interface 220, which interfaces with an engine control unit (ECU) (not shown) or a MCU (not shown) of autonomous vehicle 100 to control the operation of the autonomous vehicle 100 such as acceleration, braking, signaling, and steering”; for example, “the one or more actions initiated 508 to assist or protect the people” include “pulling the autonomous vehicle in a driving lane or a shoulder”), a microphone (Davis, Para. 0039, 0040 – “Mission control 224 may send commands to the autonomous vehicle to transmit sensor data associated with one or more specific sensors 206”, where sensors 206 include “microphones or acoustic sensors 214”), and an electronic speaker (Davis, Para. 0024 – “use a loudspeaker to inform of the emergency situation”).
In regards to Claim 4, Davis in view of Tal teaches the system of Claim 1, and Davis in view of Tal further teaches further comprising a dedicated housing mounted to the vehicle and that houses the camera and the first electronic computing device (Tal, Para. 0045-0046 – “device 101 is mounted by a mounting component 103a to a body of a vehicle 102a (e.g. car, truck, bus, etc.), such that an imager 500 (e.g. camera…) has a viewpoint of the road surface 14”, where the device 101 “transmits the data 20 (containing information about the objects 12 identified from the images 16)” to a “server 107a”, such that the device 101 constitutes the first electronic computing device and includes a camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system including the above limitations of Davis in view of Tal to further include further comprising a dedicated housing mounted to the vehicle and that houses the camera and the first electronic computing device, as taught by Tal, in order to provide a single device to implement the system to reduce costs (Tal, Para. 0016)
In regards to Claim 5, Davis in view of Tal teaches the system of Claim 1, and Davis further teaches wherein the vehicle is an emergency vehicle (Davis, Para. 0020-0027 – where the “autonomous vehicle” has a “first responder mode”, or emergency mode, that may be “automatically enabled, by default”, such that the autonomous vehicle is operating as a first responder, or emergency vehicle, and may provide “emergency supplies” and “assist people”).
In regards to Claim 6, Davis in view of Tal teaches the system of Claim 1, and Davis further teaches wherein the first signals include metadata relating to an event corresponding to the event type (Davis, Para. 0031, 0046, 0059 – “the autonomous vehicle 100 may identify a scene of an accident, injured people, smoke, fire, distressed voice, damaged property” and where the autonomous vehicle transmits “a notification including a geolocation or an emergency situation identified based on analysis of the sensor data” to a “client computing device at the emergency services”).
In regards to Claim 7, Davis in view of Tal teaches the system of Claim 6, and Davis in view of Tal further teaches wherein the metadata includes a time stamp for the event in the video footage (Tal, Para. 0064, 0085 – additional information such as “date/time… can be associated to each incident image (e.g. image frame 16a,b,c,d)”, where each incident image frame is of the “captured video containing the individual frames”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system including the above limitations of Davis in view of Tal to further include wherein the metadata includes a time stamp for the event in the video footage, as taught by Tal, in order to obtain additional data to improve event detection and response accuracy.
Regarding Claim 8, Davis teaches: A system for responding to events (Davis, Para. 0004-0006, 0038-0039 – a “mission control computing system” for “identifying an emergency situation”, or event, and “triggering a first responder mode of the autonomous vehicle” to “initiating one or more actions”), comprising:
a first electronic computing device (Davis, Para. 0038, 0055, 0060 –“mission control computing system 224” that is a “server computer device 401”), including:
at least one processor (Davis, Para. 0055 – “server computer device 401 also includes a processor 405” which “may include one or more processing units); and
memory encoding instructions which, when executed by the at least one processor, cause the first electronic computing device (Davis, Para. 0055 – “server computer device 401 also includes a processor 405 for executing instructions… stored in a memory area 430”) to:
process, by the at least one processor, first signals transmitted from a second electronic computing device mounted to a vehicle that is remote from the first electronic computing device (Davis, Para. 0055, 0060 – where the “service computer device” receives “audio or visual data corresponding to the emergency situation”, where the server computer processes the data to determine “instructions to… perform the one or more actions to assist or protect the people or the property at the site of the emergency situation”), the first signals indicating an occurrence within a visual range of the vehicle of a predefined event type, the occurrence being determined from where the autonomous vehicle “senses the environment surrounding autonomous vehicle 100 by gathering and interpreting sensor data” to “identify an emergency situation, presence or absence of a personnel from the emergency services, victims requiring assistance, damaged property, etc.” and transmits “audio or visual data corresponding to the emergency situation” to the “server computer device”; where the sensors include “one or more cameras” capturing “visual sensor data”); and
generate second signals configured to cause a device associated with the vehicle to respond to the detected event with a response, the second signals being generated based on the predefined event type (Davis, Para. 0024, 0029, 0040, 0055, 0060-0062 – “one or more actions may be initiated 508 to assist or protect people or a property at the site of the emergency situation”, where the “instructions”, or second signals, “to… perform the one or more actions to assist… at the site of the emergency situation may be received from the computing system”, that is “a computing device at mission control”, i.e. “mission control computing system 224”; for example if “the accident has occurred in a single lane, the autonomous vehicle may stop in the lane of the accident behind the vehicles involved in the accident to protect them from other traffic and display appropriate message on the display device mounted on exterior of the autonomous vehicle”).
While Davis teaches the first signals indicating an occurrence within a visual range of the vehicle of a predefined event type, the occurrence being determined from footage, Davis does not specifically teach the occurrence being determined from video footage.
However, Tal teaches the occurrence being determined from video footage. (Tal, Para. 0042, 0045, 0064, 0075 – a “device 101” mounted to a “vehicle's 102a windshield or a portion of the body” or “attached to the dashboard” including “an imager 500”, “e.g. camera”, for obtaining “captured video containing the individual frames”, the device utilized for “identification and reporting of road related incidents/conditions/objects”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Davis to include the occurrence being determined from video footage, as taught by Tal, in order to obtain additional data from video footage to improve event detection and response accuracy.
In regards to Claim 11, Davis in view of Tal teaches the system of Claim 8, and Davis further teaches wherein the vehicle is an emergency vehicle (Davis, Para. 0020-0027 – where the “autonomous vehicle” has a “first responder mode”, or emergency mode, that may be “automatically enabled, by default”, such that the autonomous vehicle is operating as a first responder, or emergency vehicle, and may provide “emergency supplies” and “assist people”).
In regards to Claim 12, Davis in view of Tal teaches the system of Claim 8, and Davis further teaches wherein the instructions, when executed by the at least one processor, cause the first electronic computing device to record the predefined event type and metadata of an event corresponding to the predefined event type (Davis, Para. 0035, 0040 – “Mission control may store the sensor data received from the autonomous vehicle 100”, where mission control “may receive vehicle data from the autonomous vehicle 100, when the autonomous vehicle 100 determines, based upon analysis of the sensor data, that an emergency situation is present”; where the “the autonomous vehicle 100 may identify a scene of an accident, injured people, smoke, fire, distressed voice, damaged property”).
In regards to Claim 13, Davis in view of Tal teaches the system of Claim 8, and Davis further teaches wherein the first signals include metadata relating to an event corresponding to the event type (Davis, Para. 0031, 0046, 0059 – “the autonomous vehicle 100 may identify a scene of an accident, injured people, smoke, fire, distressed voice, damaged property” and where the autonomous vehicle transmits “a notification including a geolocation or an emergency situation identified based on analysis of the sensor data” to a “client computing device at the emergency services”).
Regarding Claim 16, Davis teaches: A system for responding to events (Davis, Para. 0004-0006, 0038-0039 – “an autonomous driving system” in communication to a “mission control computing system” for “identifying an emergency situation”, or event, and “triggering a first responder mode of the autonomous vehicle” to “initiating one or more actions”), comprising:
a vehicle (Davis, Para. 0004-0006 – “an autonomous vehicle”);
a visual sensor array mounted to the vehicle (Davis, Para. 0004-0006, 0025, 0059 – “one or more sensors positioned on a body of the autonomous vehicle” including “one or more cameras, one or more LiDAR sensors, one or more RADAR sensors, one or more ultrasound sensors, or one or more infrared sensors” capturing “visual sensor data”), ;
a first electronic computing device mounted to the vehicle (Davis, Para. 0044 – the “autonomous vehicle 100 may further include an autonomy computing system 204”, or first electronic computing device), the first electronic computing device including:
at least one processor (Davis, Para. 0050-0051 – where the “autonomy computing system 204” may be “any suitable computing device 300” which “includes a processor 314”); and
memory encoding instructions which, when executed by the at least one processor, cause the first electronic computing device (Davis, Para. 0004-0006, 0050-0053 – “autonomy computing system 204” may be “any suitable computing device 300” including a “memory device 318” including “one or more devices that enable information, such as executable instructions” executable by the “processor 314”) to:
process, by the at least one processor, the “based on the data collected from sensors 206”, including the “cameras”, “autonomy computing system 204 is configured to perform analysis of the sensor data”)
(i) detect an event “the autonomy computing system 204 and, more specifically, perception and understanding module 203 senses the environment surrounding autonomous vehicle 100 by gathering and interpreting sensor data” to “identify an emergency situation, presence or absence of a personnel from the emergency services, victims requiring assistance, damaged property, etc.”); and
(ii) assign a predefined event type to the event (Davis, Para. 0046 – “identify an emergency situation, presence or absence of a personnel from the emergency services, victims requiring assistance, damaged property, etc.” by “one or more machine-learning algorithms trained to identify an emergency situation”, such that the emergency situations, or events, are predefined by the machine-learning algorithms);
transmit first signals to a second electronic computing device that is remote from the vehicle, the first signals indicating an occurrence of the predefined event type (Davis, Para. 0038, 0055, 0060 –“autonomous driving system 200… is communicatively coupled with mission control computing system 224” that is a “server computer device 401”, or a second electronic computing device, where “the server computing device 401 may receive sensor data, from the autonomous vehicle 100, for identifying an emergency situation and triggering or enabling the first responder mode of the autonomous vehicle 100”; where when the autonomous vehicle determines “presence of the emergency situation and absence of the emergency vehicle at the site of the emergency situation”, the vehicle triggers “a first responder mode” and sends “data corresponding to the emergency situation”, or first signals, to the “computing system (e.g., a computing device at mission control…)”);
process, by the at least one processor, second signals transmitted from the second electronic computing device (Davis, Para. 0040, 0055, 0060-0061 – “the mission control computing system 224 may transmit control commands or data to the autonomous vehicle 100, such as navigation commands, and travel trajectories to the autonomous vehicle 100”, or second signals, where the “autonomous vehicle” performs “the one or more actions to assist or protect the people”); and
based on the second signals, cause a device associated with the vehicle to respond to the detected event with a response (Davis, Para. 0024, 0040, 0055, 0060-0062 – “one or more actions may be initiated 508 to assist or protect people or a property at the site of the emergency situation”, where the “instructions to… perform the one or more actions to assist… at the site of the emergency situation may be received from the computing system”, that is “a computing device at mission control”, i.e. “mission control computing system 224”; for example “pulling the autonomous vehicle in a driving lane or a shoulder”, “displaying one or more messages on one or more display devices… to alert a passerby”, “turn blinkers or hazard lights on”, “use a loudspeaker to inform of the emergency situation”, etc.).
While Davis teaches a visual sensor array mounted to the vehicle, Davis does not specifically teach the visual sensor array being configured to capture video footage. Furthermore, Davis does not teach process, by the at least one processor, the video footage to encode the video footage as a numerical data stream and, based thereon: (i) detect an event in the numerical data stream to provide a detected event.
However, Tal teaches the visual sensor array being configured to capture video footage (Tal, Para. 0045, 0064, 0075 – a “device 101” mounted to a “vehicle's 102a windshield or a portion of the body” or “attached to the dashboard” including “an imager 500”, “e.g. camera”, for obtaining “captured video containing the individual frames”), and process, by the at least one processor, the video footage to encode the video footage as a numerical data stream (Tal, Para. 0064-0069 – executing “image processing instructions” to obtain “digital data content… each with finite, discrete quantities of numeric representation for its color intensity or gray level that is an output from its two-dimensional functions fed as input by its spatial coordinates denoted with x, y on the x-axis and y-axis, respectively” from the “digital image[s]”, where the data can be “stored and sent separately, with association to the respective image”; where the individual digital image frames constitute the “captured video”) and, based thereon: (i) detect an event in the numerical data stream to provide a detected event (Tal, Para. 0022, 0042, 0069, 0115 – “execute image processing instructions for processing the digital images for automated detection of the incidents, generating object data based on the processing”, where object data is “referred to interchangeably as incident data”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Davis to include the camera being configured to capture video footage, and process, by the at least one processor, the video footage to encode the video footage as a numerical data stream and, based thereon: (i) detect an event in the numerical data stream to provide a detected event, as taught by Tal, in order to obtain additional data from video footage to improve event detection and response accuracy.
In regards to Claim 18, Davis in view of Tal teaches the system of Claim 16, and Davis further teaches wherein the device associated with the vehicle includes one or more of a camera (Davis, Para. 0040 – “Mission control 224 may send commands to the autonomous vehicle to transmit sensor data associated with one or more specific sensors 206”, where sensors 206 include “cameras 212”), a smartphone, a controller that controls a movement of the vehicle or a part of the vehicle (Davis, Para. 0042-0044, 0061 – “autonomous vehicle 100 may further include a vehicle interface 220, which interfaces with an engine control unit (ECU) (not shown) or a MCU (not shown) of autonomous vehicle 100 to control the operation of the autonomous vehicle 100 such as acceleration, braking, signaling, and steering”; for example, “the one or more actions initiated 508 to assist or protect the people” include “pulling the autonomous vehicle in a driving lane or a shoulder”), a microphone (Davis, Para. 0039, 0040 – “Mission control 224 may send commands to the autonomous vehicle to transmit sensor data associated with one or more specific sensors 206”, where sensors 206 include “microphones or acoustic sensors 214”), and an electronic speaker (Davis, Para. 0024 – “use a loudspeaker to inform of the emergency situation”).
In regards to Claim 19, Davis in view of Tal teaches the system of Claim 16, and Davis further teaches wherein the vehicle is an emergency vehicle (Davis, Para. 0020-0027 – where the “autonomous vehicle” has a “first responder mode”, or emergency mode, that may be “automatically enabled, by default”, such that the autonomous vehicle is operating as a first responder, or emergency vehicle, and may provide “emergency supplies” and “assist people”).
In regards to Claim 20, Davis in view of Tal teaches the system of Claim 16, and Davis further teaches wherein the first signals include metadata relating to an event corresponding to the event type (Davis, Para. 0031, 0046, 0059 – “the autonomous vehicle 100 may identify a scene of an accident, injured people, smoke, fire, distressed voice, damaged property” and where the autonomous vehicle transmits “a notification including a geolocation or an emergency situation identified based on analysis of the sensor data” to a “client computing device at the emergency services”); but Davis does not teach wherein the metadata includes a time stamp for the event in the video footage.
However, Tal teaches wherein the metadata includes a time stamp for the event in the video footage (Tal, Para. 0064, 0085 – additional information such as “date/time… can be associated to each incident image (e.g. image frame 16a,b,c,d)”, where each incident image frame is of the “captured video containing the individual frames”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system including the above limitations of Davis in view of Tal to further include wherein the metadata includes a time stamp for the event in the video footage, as taught by Tal, in order to obtain additional data to improve event detection and response accuracy.
Claim(s) 2, 9-10, 14-15, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Davis in view of Tal, and further in view of Gupta, et al. (U.S. Patent Application Pub. No. 2018/0349715).
In regards to Claim 2, Davis in view of Tal teaches the system of Claim 1, but Davis in view of Tal does not teach wherein the second electronic computing device does not receive the video footage.
However, Gupta teaches wherein the second electronic computing device does not receive the video footage (Gupta, Para. 0051-0052 – where a “smartphone” configured as an “imaging device” for a vehicle is configured to “send indications of high-priority events”, to “a remote analysis, monitoring, and/or coordination device”, “while storing low-priority data (e.g., video data) locally”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the system including the above limitations of Davis in view of Tal to include wherein the second electronic computing device does not receive the video footage, as taught by Gupta, in order to reduce the amount of data transmitted and utilize low bandwidth (Gupta, Para. 0052).
In regards to Claim 9, Davis in view of Tal teaches the system of Claim 9, but Davis in view of Tal does not teach wherein the second electronic computing device does not receive the video footage.
However, Gupta teaches wherein the second electronic computing device does not receive the video footage (Gupta, Para. 0051-0052 – where a “smartphone” configured as an “imaging device” for a vehicle is configured to “send indications of high-priority events”, to “a remote analysis, monitoring, and/or coordination device”, “while storing low-priority data (e.g., video data) locally”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the system including the above limitations of Davis in view of Tal to include wherein the second electronic computing device does not receive the video footage, as taught by Gupta, in order to reduce the amount of data transmitted and utilize low bandwidth (Gupta, Para. 0052).
In regards to Claim 10, Davis in view of Tal and Gupta teaches the system of Claim 9, and Davis further teaches wherein the device associated with the vehicle includes one or more of a camera (Davis, Para. 0040 – “Mission control 224 may send commands to the autonomous vehicle to transmit sensor data associated with one or more specific sensors 206”, where sensors 206 include “cameras 212”), a smartphone, a controller that controls a movement of the vehicle or a part of the vehicle (Davis, Para. 0042-0044, 0061 – “autonomous vehicle 100 may further include a vehicle interface 220, which interfaces with an engine control unit (ECU) (not shown) or a MCU (not shown) of autonomous vehicle 100 to control the operation of the autonomous vehicle 100 such as acceleration, braking, signaling, and steering”; for example, “the one or more actions initiated 508 to assist or protect the people” include “pulling the autonomous vehicle in a driving lane or a shoulder”), a microphone (Davis, Para. 0039, 0040 – “Mission control 224 may send commands to the autonomous vehicle to transmit sensor data associated with one or more specific sensors 206”, where sensors 206 include “microphones or acoustic sensors 214”), and an electronic speaker (Davis, Para. 0024 – “use a loudspeaker to inform of the emergency situation”).
Regarding Claim 14, Davis teaches: A system for responding to events (Davis, Para. 0004-0006, 0038-0039 – “an autonomous driving system” in communication to a “mission control computing system” for “identifying an emergency situation”, or event, and “triggering a first responder mode of the autonomous vehicle” to “initiating one or more actions”), comprising:
a vehicle (Davis, Para. 0004-0006 – “an autonomous vehicle”);
a response device associated with the vehicle (Davis, Para. 0023-0024, 0029, 0042 – “a vehicle interface 220, which interfaces with an engine control unit (ECU)… or a MCU… of autonomous vehicle 100 to control the operation of the autonomous vehicle 100” in an “emergency situation”; where response devices further include “blinkers or hazard lights”, “a loudspeaker”, a “digital display device”, etc.);
a camera mounted to the vehicle (Davis, Para. 0004-0006, 0025, 0059 – “one or more sensors positioned on a body of the autonomous vehicle” including “one or more cameras” capturing “visual sensor data”),
a first electronic computing device mounted to the vehicle (Davis, Para. 0044 – the “autonomous vehicle 100 may further include an autonomy computing system 204”, or first electronic computing device) and including at least one first processor (Davis, Para. 0050-0051 – where the “autonomy computing system 204” may be “any suitable computing device 300” which “includes a processor 314”);
a second electronic computing device that is remote from the vehicle and including at least one second processor (Davis, Para. 0055 – “server computer device 401”, or second electronic computing device, “includes a processor 405” which “may include one or more processing units),
the first electronic computing device including memory encoding first instructions (Davis, Para. 0004-0006, 0050-0053 – “autonomy computing system 204” may be “any suitable computing device 300” including a “memory device 318” including “one or more devices that enable information, such as executable instructions” executable by the “processor 314”) and the second electronic computing device including memory encoding second instructions (Davis, Para. 0055 – “server computer device 401 also includes a processor 405 for executing instructions… stored in a memory area 430”) which, when executed by the at least one first processor and the at least one second processor, respectively, cause the first electronic computing device and the second electronic computing device to:
process, by the at least one processor, the “based on the data collected from sensors 206”, including the “cameras”, “autonomy computing system 204 is configured to perform analysis of the sensor data”) and, based thereon:
(i) detect, by the at least one first processor, an event in the “the autonomy computing system 204 and, more specifically, perception and understanding module 203 senses the environment surrounding autonomous vehicle 100 by gathering and interpreting sensor data” to “identify an emergency situation, presence or absence of a personnel from the emergency services, victims requiring assistance, damaged property, etc.”); and
(ii) assign, by the at least one first processor, a predefined event type to the event (Davis, Para. 0046 – “identify an emergency situation, presence or absence of a personnel from the emergency services, victims requiring assistance, damaged property, etc.” by “one or more machine-learning algorithms trained to identify an emergency situation”, such that the emergency situations, or events, are predefined by the machine-learning algorithms);
generate, by the first electronic computing device, first signals indicating an occurrence of the predefined event (Davis, Para. 0038, 0055, 0060 –“autonomous driving system 200… is communicatively coupled with mission control computing system 224” to transmit “data corresponding to the emergency situation”, or first signals, such that the signals are generated);
transmit, by the first electronic computing device, the first signals to the second electronic computing device (Davis, Para. 0038, 0055, 0060 –“autonomous driving system 200… is communicatively coupled with mission control computing system 224” that is a “server computer device 401”, where “the server computing device 401 may receive sensor data, from the autonomous vehicle 100, for identifying an emergency situation and triggering or enabling the first responder mode of the autonomous vehicle 100”; where when the autonomous vehicle determines “presence of the emergency situation and absence of the emergency vehicle at the site of the emergency situation”, the vehicle triggers “a first responder mode” and sends “data corresponding to the emergency situation”, or first signals, to the “computing system (e.g., a computing device at mission control…)”);
generate, by the second electronic computing device, second signals based on the predefined event type (Davis, Para. 0024, 0029, 0040, 0055, 0060-0062 – “one or more actions may be initiated 508 to assist or protect people or a property at the site of the emergency situation”, where the “instructions”, or second signals, “to… perform the one or more actions to assist… at the site of the emergency situation may be received from the computing system”, that is “a computing device at mission control”, i.e. “mission control computing system 224”; for example if “the accident has occurred in a single lane, the autonomous vehicle may stop in the lane of the accident behind the vehicles involved in the accident to protect them from other traffic and display appropriate message on the display device mounted on exterior of the autonomous vehicle”);
transmit, by the second electronic computing device, the second signals (Davis, Para. 0024, 0029, 0040, 0055, 0060-0062 – “instructions”, or second signals, “to… perform the one or more actions to assist… at the site of the emergency situation may be received from the computing system”, that is “a computing device at mission control”, by the “autonomous vehicle”);
process, by the at least one first processor, the second signals (Davis, Para. 0040, 0055, 0060-0061 – “the mission control computing system 224 may transmit control commands or data to the autonomous vehicle 100, such as navigation commands, and travel trajectories to the autonomous vehicle 100”, or second signals, where the “autonomous vehicle” performs “the one or more actions to assist or protect the people”); and
based on the second signals, generate, by the first electronic computing device, third signals, wherein the response device is configured to receive the third signals and, based on the third signals, to respond to the detected event with a response (Davis, Para. 0024, 0040-0044, 0055, 0060-0062 – “one or more actions may be initiated 508 to assist or protect people or a property at the site of the emergency situation”, where the “instructions to… perform the one or more actions to assist… at the site of the emergency situation may be received from the computing system”, that is “a computing device at mission control”, i.e. “mission control computing system 224”; for example “autonomous vehicle 100 may further include a vehicle interface 220, which interfaces with an engine control unit (ECU) (not shown) or a MCU (not shown) of autonomous vehicle 100 to control the operation of the autonomous vehicle 100 such as acceleration, braking, signaling, and steering”; for example, “a vehicle interface 220, which interfaces with an engine control unit (ECU)… or a MCU… of autonomous vehicle 100 to control the operation of the autonomous vehicle 100” to trigger “the one or more actions initiated 508 to assist or protect the people” such as “pulling the autonomous vehicle in a driving lane or a shoulder”, such that third signals are generated to trigger the one or more actions); and
While Davis teaches a camera mounted to the vehicle, Davis does not specifically teach the camera being configured to capture video footage. Additionally Davis does not teach wherein the video footage is not provided to the second electronic computing device.
However, Tal teaches the camera being configured to capture video footage (Tal, Para. 0045, 0064, 0075 – a “device 101” mounted to a “vehicle's 102a windshield or a portion of the body” or “attached to the dashboard” including “an imager 500”, “e.g. camera”, for obtaining “captured video containing the individual frames”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Davis to include the camera being configured to capture video footage, as taught by Tal, in order to obtain additional data from video footage to improve event detection and response accuracy.
Davis in view of Tal does not teach wherein the video footage is not provided to the second electronic computing device.
However, Gupta teaches wherein the second electronic computing device does not receive the video footage (Gupta, Para. 0051-0052 – where a “smartphone” configured as an “imaging device” for a vehicle is configured to “send indications of high-priority events”, to “a remote analysis, monitoring, and/or coordination device”, “while storing low-priority data (e.g., video data) locally”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the system including the above limitations of Davis in view of Tal to include wherein the second electronic computing device does not receive the video footage, as taught by Gupta, in order to reduce the amount of data transmitted and utilize low bandwidth (Gupta, Para. 0052).
In regards to Claim 15, Davis in view of Tal and Gupta teaches the system of Claim 14, and Davis in view of Tal and Gupta further teaches further comprising a dedicated housing mounted to the vehicle and that houses the camera and the first electronic computing device (Tal, Para. 0045-0046 – “device 101 is mounted by a mounting component 103a to a body of a vehicle 102a (e.g. car, truck, bus, etc.), such that an imager 500 (e.g. camera…) has a viewpoint of the road surface 14”, where the device 101 “transmits the data 20 (containing information about the objects 12 identified from the images 16)” to a “server 107a”, such that the device 101 constitutes the first electronic computing device and includes a camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system including the above limitations of Davis in view of Tal and Gupta to further include further comprising a dedicated housing mounted to the vehicle and that houses the camera and the first electronic computing device, as taught by Tal, in order to provide a single device to implement the system to reduce costs (Tal, Para. 0016)
In regards to Claim 17, Davis in view of Tal teaches the system of Claim 16, but Davis in view of Tal does not teach wherein the second electronic computing device does not receive the video footage.
However, Gupta teaches wherein the second electronic computing device does not receive the video footage (Gupta, Para. 0051-0052 – where a “smartphone” configured as an “imaging device” for a vehicle is configured to “send indications of high-priority events”, to “a remote analysis, monitoring, and/or coordination device”, “while storing low-priority data (e.g., video data) locally”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the system including the above limitations of Davis in view of Tal to include wherein the second electronic computing device does not receive the video footage, as taught by Gupta, in order to reduce the amount of data transmitted and utilize low bandwidth (Gupta, Para. 0052).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kumar, et al. (U.S. Patent Application Pub. No. 2021/0166565) teaches controlling autonomous vehicles to provide automated emergency response functions, including a computing platform for receiving vehicle data associated with a vehicle from an on-board vehicle monitoring system associated with the vehicle.
Della Penna (U.S. Patent Application Pub. No. 2019/0220011) teaches recording event data to identify and resolve anomalies associated with control of driverless vehicles including “receiving event data via a communications network from an autonomous vehicle, identifying a computed vehicular drive parameter, extracting sensor data associated with the event, detecting application of control signals, analyzing the control signals, the sensor data, and the subset of computed vehicular drive parameters to identify a type of event, and generating update executable instructions responsive to the type of event”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HELEN LI whose telephone number is (703)756-4719. The examiner can normally be reached Monday through Friday, from 9am to 5pm eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at (571) 272-7298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.L./Examiner, Art Unit 3665
/HUNTER B LONSBERRY/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights