DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see Page 7, filed June 10, 2026, with respect to 101 rejections have been fully considered and are persuasive. The 101 rejections have been withdrawn.
Applicant’s arguments, see Page 7, filed June 10, 2026, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 102 and/or 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Colafrancesco et al. (US 12548564; hereinafter Colafrancesco).
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-3, 8-16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Harvey (US 20250104708) in view of Colafrancesco et al. (US 12548564; hereinafter Colafrancesco).
In regards to claim 1, Harvey discloses of a vehicle computing system for controlling functionality of a vehicle (“A vehicle chatbot of a smart vehicle assistant engages in a conversation with a user associated with a vehicle, for instance to provide insurance information and range extension tips associated with vehicle operations. The vehicle chatbot may also engage in a conversation with an external entity in the event of a collision, to provide information to the external entity on behalf of vehicle occupants. The smart vehicle assistant may also cause the vehicle to autonomously drive to a location following a collision. A responder dispatched to respond to the collision may use a smart responder assistant that includes a responder chatbot. The responder chatbot may engage in a conversation with the responder to obtain information identifying the vehicle and/or damage to the vehicle, and may provide the responder with information about the vehicle and recommendations regarding how to extract occupants of the vehicle.” (Abstract)) comprising:
a control circuit (“In various embodiments, a hardware module may be implemented mechanically or electronically. For example, a hardware module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) to perform certain operations). A hardware module may also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.” (Para 0201)) configured to:
receive, from a first sensor associated with a vehicle, a user prompt from a user associated with the vehicle, the user prompt indicative of a statement or a question (“A chatbot, such as the vehicle chatbot 120 or the responder chatbot 122, may be based upon a generative Artificial Intelligence (AI) system that may generate natural language text and/or audio responses to input data, such that a user may converse with the chatbot naturally by asking free-form questions or making other natural language statements, and receiving corresponding natural language responses generated by the chatbot instead of, or in addition to, prewritten responses or predetermined information. The chatbot may generate natural language output that expresses information conversationally via serious and/or humorous statements, that responds to statements and/or questions input by the user most recently and/or earlier during a conversation, that poses questions to the user, and/or that otherwise converses with the user.” (Para 0075), “In some examples, the smart vehicle assistant 102 may be executed at least in part via one or more computing systems that are integrated into the vehicle 106. For example, the vehicle 106 may have one or more on-board processors that may execute one or more elements of the smart vehicle assistant 102. In these examples, a user inside the vehicle 106, such as a driver or other occupant, may use the smart vehicle assistant 102 via a dashboard display of the vehicle 106, integrated speakers and/or microphones of the vehicle 106, and/or other elements of the vehicle 106.” (Para 0058));
access, from the first or a second sensor associated with the vehicle, sensor data associated with a surrounding environment of the vehicle, the sensor data comprising at least of (i) an image of the user, (ii) weather data, (iii) a location of the vehicle, or (iv) a timestamp associated with the user prompt (“The vehicle 106 may have one or more sensors 110 that are configured to capture corresponding types of sensor data, user input, or other input data. The sensors 110 may include accelerometers and/or other motion sensors, Global Positioning System (GPS) sensors and/or other location sensors, sensors associated with a transmission and/or braking system of the vehicle 106, cameras and/or other image-based sensors, Light Detection and Ranging (LiDAR) sensors, microphones, proximity sensors, weight sensors, seatbelt sensors, seat pressure sensors, payload sensors, and/or other types of sensors. Sensor data, user input, and/or other input data captured by the sensors 110 may be provided to an on-board computing system of the vehicle 106, for instance such that the on-board computing system may perform autonomous or semi-autonomous operations based upon received sensor data. In some examples, as described herein, sensor data, user input, and/or other input data captured by the sensors 110 may also, or alternately, be provided to the smart vehicle assistant 102, such that the smart vehicle assistant 102 may operate based upon the sensor data, user input, and/or other input data.” (Para 0056), “he additional data 146 may include one or more other types of information, such as weather data, traffic data, map data, image and/or audio data associated with collisions of vehicles, image and/or audio data associated with occupants of vehicles before, during, and/or after collisions, steering and driving data, and/or other types of data. The additional data 146 may be maintained in one or more databases or other data repositories, for instance in one or more databases maintained by an insurance company, an operator of the model training system 134 and/or a provider of the smart vehicle assistant 102 and/or the smart responder assistant 104.” (Para 0093), “The range data 142 may include information about how far vehicles powered by batteries are able to travel based upon State of Charge (SoC) levels of the batteries and/or other factors. For example, the range data 142 may include historical data indicating how SoC levels of vehicle batteries change over time, and/or how far vehicles have been able to travel based upon power from such vehicle batteries, in association with travel speeds, travel routes, traffic patterns along the travel routes, capabilities of vehicles, and/or other factors. The range data 142 may also include example scripts for communicating tips regarding extending travel ranges and/or battery SoC levels to users of the vehicle chatbot 120. The range data 142 may be maintained in one or more databases or other data repositories, for instance in one or more databases maintained by manufacturers of vehicles and/or batteries, an operator of the model training system 134, and/or a provider of the smart vehicle assistant 102.” (Para 0091));
generate, using a context engine configured to determine context data associated with at least one of the user or the vehicle, a modified user prompt based on the user prompt and the sensor data(“For example, as described further below, the smart vehicle assistant 102 may provide output associated with the vehicle 106 to occupants of the vehicle 106 and/or to other entities. Such output may include insurance information, battery range information, collision response information, and/or other types of information associated with the vehicle 106. The smart vehicle assistant 102 may also, or alternately, cause the vehicle 106 to perform actions autonomously in certain situations. For example, if the vehicle 106 is involved in a collision, and the smart vehicle assistant 102 determines that occupants of the vehicle 106 are unresponsive following the collision, the smart vehicle assistant 102 may cause the vehicle 106 to autonomously drive to a hospital or other destination.” (Para 0052), “The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Para 0103), see also Para 0078, 0102, and 0105) …
input the modified user prompt into a machine-learned model configured to generate, based on the modified user prompt, a user response (“The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Para 0103), “For example, as described further below, the smart vehicle assistant 102 may provide output associated with the vehicle 106 to occupants of the vehicle 106 and/or to other entities. Such output may include insurance information, battery range information, collision response information, and/or other types of information associated with the vehicle 106. The smart vehicle assistant 102 may also, or alternately, cause the vehicle 106 to perform actions autonomously in certain situations. For example, if the vehicle 106 is involved in a collision, and the smart vehicle assistant 102 determines that occupants of the vehicle 106 are unresponsive following the collision, the smart vehicle assistant 102 may cause the vehicle 106 to autonomously drive to a hospital or other destination.” (Para 0052), “In other examples, a user interface 118 may include a non-visual interface, such as an audio-based interface. Accordingly, the smart vehicle assistant 102 and/or the smart responder assistant 104 may present or convey information to users via audio with or without also displaying the information visually via a screen. As an example, the smart vehicle assistant 102 may be an audio-based system that may receive user input as audio voice input captured by a microphone of the vehicle 106 or a user device, and that May audibly present corresponding output voice data via speakers of the vehicle 106 or the user device. Accordingly, in these examples, a user of the smart vehicle assistant 102 may have a voice-based audio conversation with the smart vehicle assistant 102 instead of, or in addition to, interacting with the smart vehicle assistant 102 via a screen or other visual interface. Similarly, a user of the smart responder assistant 104 may have a voice-based audio conversation with the smart responder assistant 104 instead of, or in addition to, interacting with the smart responder assistant 104 via a screen or other visual interface.” (Para 0068), see also Para 0051); and
control, based on the user response, one or more hardware components of the vehicle to perform a vehicle function corresponding to the statement or the question, the vehicle function comprising at least one of: (v) a vehicle comfort function; (vi) a vehicle staging function; (vii) a vehicle climate function; (viii) a vehicle navigation function; (ix) a drive style function; (x) a vehicle parking function; or (xi) a vehicle entertainment function (“The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Para 0103), “For example, as described further below, the smart vehicle assistant 102 may provide output associated with the vehicle 106 to occupants of the vehicle 106 and/or to other entities. Such output may include insurance information, battery range information, collision response information, and/or other types of information associated with the vehicle 106. The smart vehicle assistant 102 may also, or alternately, cause the vehicle 106 to perform actions autonomously in certain situations. For example, if the vehicle 106 is involved in a collision, and the smart vehicle assistant 102 determines that occupants of the vehicle 106 are unresponsive following the collision, the smart vehicle assistant 102 may cause the vehicle 106 to autonomously drive to a hospital or other destination.” (Para 0052), “In other examples, a user interface 118 may include a non-visual interface, such as an audio-based interface. Accordingly, the smart vehicle assistant 102 and/or the smart responder assistant 104 may present or convey information to users via audio with or without also displaying the information visually via a screen. As an example, the smart vehicle assistant 102 may be an audio-based system that may receive user input as audio voice input captured by a microphone of the vehicle 106 or a user device, and that May audibly present corresponding output voice data via speakers of the vehicle 106 or the user device. Accordingly, in these examples, a user of the smart vehicle assistant 102 may have a voice-based audio conversation with the smart vehicle assistant 102 instead of, or in addition to, interacting with the smart vehicle assistant 102 via a screen or other visual interface. Similarly, a user of the smart responder assistant 104 may have a voice-based audio conversation with the smart responder assistant 104 instead of, or in addition to, interacting with the smart responder assistant 104 via a screen or other visual interface.” (Para 0068), see also Para 0051).
However, Harvey does not specifically disclose of wherein the modified user prompt comprises additional tokens that supplements the user prompt, the additional tokens associated with the context data, and the context data providing one or more conditions associated with the user prompt.
Colafrancesco, in the same field of endeavor, teaches of wherein the modified user prompt comprises additional tokens that supplements the user prompt, the additional tokens associated with the context data, and the context data providing one or more conditions associated with the user prompt (“With continued reference to FIG. 2, the command script 208 output by the script generator 204 may include one or more ambiguities (e.g., an unknown or uncertain parameter value). For example, the command script 208 may include a placeholder for a target device to be controlled that could be one or more different devices. As another example, the command script 208 may specify an ambiguous configuration for an unidentified target device, such as a particular setting, media content to be played, and/or the like. The command script 208 is input to the context engine 210, which may include software and/or hardware configured to generate one or more refined command scripts 212 based on the raw command script 208 and contextual data. The context engine 210 may include, for example, a software function executed by a control device or a remote server computer. The context engine 210 outputs a refined command script 212 that is input to a command signal generator 214.” (Column 10 lines 7-24), “At step 414, ambiguities in the command script are attempted to be resolved based on the configuration data and/or contextual data. Step 414 may be a post-processing step performed after the command script is initially generated, as shown in FIG. 4, or in other examples may be implemented as part of a model (e.g., as a decoder model) used to generate the command script. In some non-limiting embodiments, a command script may be generated at step 406 that includes a placeholder (e.g., a token) for an unknown value. Such a token may be inserted into the command script during its initial generation if a device, location, command, and/or other parameter cannot be identified in the natural language text string. For example, if the text string is “turn light on” and there are multiple lighting devices in the configuration database or otherwise recognized by the control device, a script may be generated with the token “{unknown}” in place of a device identifier (e.g., “power on type light {unknown}”). At step 414, the configuration data may be analyzed again to determine if additional information is available for resolving the ambiguity and for replacing the token with a device or a group of devices. For example, if the lights identified in the configuration data are in one location, it may be inferred that all lights should be turned on and the token may be replaced with device identifiers for each of the lighting devices associated with that location in the configuration database.” (Column 13 lines 5-31), see also claim 1).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the user prompt, as taught by Harvey, to include being modified to include additional tokens that supplements the user prompt with context data, as taught by Colafrancesco, with a reasonable expectation of success in order to resolve ambiguities in the user prompt (Colafrancesco Column 13 lines 5-31).
In regards to claim 2, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 1, wherein the context engine is configured to:
analyze the user prompt from the user (“In some examples, the model training system 134 may be at least partially separate from the smart vehicle assistant 102 and/or the smart responder assistant 104, and may execute to train and/or re-train instance of the vehicle chatbot 120 and/or the responder chatbot 122. A trained instance of the vehicle chatbot 120 may accordingly be deployed in the smart vehicle assistant 102, and a trained instance of the responder chatbot 122 may accordingly be deployed in the smart responder assistant 104. The model training system 134 may train a chatbot, such as the vehicle chatbot 120 or the responder chatbot 122, to generate conversational statements and/or other output during a conversation proactively and/or in response to user questions or statements. The chatbot may generate such statements or other output based upon information that was in a training dataset at the time the chatbot was trained and/or based upon other information that may be accessed by the chatbot.” (Harvey Para 0080), “In examples in which user input is audio-based voice data, a chatbot and/or other elements of the smart vehicle assistant 102 or the smart responder assistant 104 may use voice-to-text systems, Natural Language Processing (NLP), and/or other types of audio processing to interpret the audio-based voice data provided by the user. In other examples in which user input is text-based, a chatbot and/or other elements of the smart vehicle assistant 102 or the smart responder assistant 104 may similarly use NLP and/or other types of text processing systems to interpret text provided by a user.” (Harvey Para 0076));
based on analyzing the user prompt, access user preference data associated with the user, the user preference data associated with the one or more conditions (“Accordingly, a user may inquire about a current travel range of the vehicle 106 via the vehicle chatbot 120, and the vehicle chatbot 120 may present information about the current travel range that is generated by the range predictor 124. In some examples, the vehicle chatbot 120 may ask the user questions about when and where the user plans to travel via the vehicle 106, what time the user wants to arrive at a destination, whether the user wants to avoid tolls, heavy traffic, accidents, and/or other elements along a route, and/or other information, such that the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may suggest a route for the user and determine a corresponding travel range to be presented via the vehicle chatbot 120. In other examples, the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may obtain information about a current or planned travel route via a GPS system of the vehicle or a connected user device, such that the range predictor 124 may generate range predictions that may be presented to the user via the vehicle chatbot 120.” (Harvey Para 0102), “Accordingly, based upon information about a type of the battery 108 of the vehicle 106, a current SoC of the battery 108, a particular travel route, current or expected traffic and/or weather conditions along the particular travel route, a current or expected travel speed of the vehicle 106 along the particular travel route, a historical driving profile of the driver of the vehicle 106, and/or other factors, the range predictor 124 may predict how far the vehicle 106 may travel and/or how much of the SoC of the battery 108 will be used during travel. Similarly, the range predictor 124 may predict how changes, such as alternate routes with different geographies, traffic patterns, weather conditions, and/or other factors that differ relative to factors associated with current travel route, adjustments to increase or decrease travel speeds of the vehicle 106, and/or other changes relative to current or expected operations of the vehicle 106, would change how far the vehicle 106 may travel and/or how much of the SoC of the battery 108 would be used during travel.” (Harvey Para 0105)); and
generate, based on the user prompt and the user preference data, the context data, wherein the context data is indicative of one or more user preferences associated with the user prompt (“Accordingly, a user may inquire about a current travel range of the vehicle 106 via the vehicle chatbot 120, and the vehicle chatbot 120 may present information about the current travel range that is generated by the range predictor 124. In some examples, the vehicle chatbot 120 may ask the user questions about when and where the user plans to travel via the vehicle 106, what time the user wants to arrive at a destination, whether the user wants to avoid tolls, heavy traffic, accidents, and/or other elements along a route, and/or other information, such that the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may suggest a route for the user and determine a corresponding travel range to be presented via the vehicle chatbot 120. In other examples, the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may obtain information about a current or planned travel route via a GPS system of the vehicle or a connected user device, such that the range predictor 124 may generate range predictions that may be presented to the user via the vehicle chatbot 120.” (Harvey Para 0102), “Accordingly, based upon information about a type of the battery 108 of the vehicle 106, a current SoC of the battery 108, a particular travel route, current or expected traffic and/or weather conditions along the particular travel route, a current or expected travel speed of the vehicle 106 along the particular travel route, a historical driving profile of the driver of the vehicle 106, and/or other factors, the range predictor 124 may predict how far the vehicle 106 may travel and/or how much of the SoC of the battery 108 will be used during travel. Similarly, the range predictor 124 may predict how changes, such as alternate routes with different geographies, traffic patterns, weather conditions, and/or other factors that differ relative to factors associated with current travel route, adjustments to increase or decrease travel speeds of the vehicle 106, and/or other changes relative to current or expected operations of the vehicle 106, would change how far the vehicle 106 may travel and/or how much of the SoC of the battery 108 would be used during travel.” (Harvey Para 0105), “The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Harvey Para 0103), see also Harvey Para 0052).
In regards to claim 3, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 2, wherein the context engine is configured:
concatenate the context data with one or more supplemental topics, the one or more supplemental topics comprising additional information associated with the user preference data (“Accordingly, a user may inquire about a current travel range of the vehicle 106 via the vehicle chatbot 120, and the vehicle chatbot 120 may present information about the current travel range that is generated by the range predictor 124. In some examples, the vehicle chatbot 120 may ask the user questions about when and where the user plans to travel via the vehicle 106, what time the user wants to arrive at a destination, whether the user wants to avoid tolls, heavy traffic, accidents, and/or other elements along a route, and/or other information, such that the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may suggest a route for the user and determine a corresponding travel range to be presented via the vehicle chatbot 120. In other examples, the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may obtain information about a current or planned travel route via a GPS system of the vehicle or a connected user device, such that the range predictor 124 may generate range predictions that may be presented to the user via the vehicle chatbot 120.” (Harvey Para 0102), “Accordingly, based upon information about a type of the battery 108 of the vehicle 106, a current SoC of the battery 108, a particular travel route, current or expected traffic and/or weather conditions along the particular travel route, a current or expected travel speed of the vehicle 106 along the particular travel route, a historical driving profile of the driver of the vehicle 106, and/or other factors, the range predictor 124 may predict how far the vehicle 106 may travel and/or how much of the SoC of the battery 108 will be used during travel. Similarly, the range predictor 124 may predict how changes, such as alternate routes with different geographies, traffic patterns, weather conditions, and/or other factors that differ relative to factors associated with current travel route, adjustments to increase or decrease travel speeds of the vehicle 106, and/or other changes relative to current or expected operations of the vehicle 106, would change how far the vehicle 106 may travel and/or how much of the SoC of the battery 108 would be used during travel.” (Harvey Para 0105), “The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Harvey Para 0103), see also Harvey Para 0052); and
input the user prompt and the one or more supplemental topics into the machine-learned model, wherein the machine-learned model is configured to generate the user response (“The range predictor 124 may be a component of the vehicle chatbot 120, or may be a separate machine learning model, a separate rules-based model, or another separate system that may interact with users via the vehicle chatbot 120. For example, the range predictor 124 may be a machine learning model that based upon convolutional neural networks, recurrent neural networks, other types of neural networks, nearest-neighbor algorithms, regression analysis, deep learning algorithms, Gradient Boosted Machines (GBMs), Random Forest algorithms, and/or other types of artificial intelligence or machine learning frameworks. The model training system 134 may train the range predictor 124 based upon one or more types of data, such as the vehicle data 140, the range data 142, and/or the additional data 146. For example, the model training system 134 may train the range predictor 124 based upon historical data associated with battery types, battery SoC levels, travel routes, traffic levels, weather conditions, and/or other factors that correspond with known travel ranges indicated in the historical data.” (Harvey Para 0104), “One or more models associated with the vehicle chatbot 120, other elements of the smart vehicle assistant 102, the responder chatbot 122, and/or other elements of the smart responder assistant 104 may be trained by a model training system 134 using supervised learning, reinforcement learning, and/or other machine learning techniques. For example, one or more models associated with a chatbot may be trained, by the model training system 134, based upon a training dataset. As discussed further below, a training dataset used by the model training system 134 to train a chatbot may be based upon one or more types of information that may be provided and/or maintained by one or more data sources 116. The data sources 116 may include insurance policy data 136, collision response data 138, vehicle data 140, range data 142, responder data 144, and/or additional data 146. Accordingly, the chatbot may be trained to provide information indicated by, and/or derived from, one or more data sources 116 during conversations with users, and/or to steer the conversations towards such information as described further below.” (Harvey Para 0078)).
In regards to claim 8, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 1, wherein the user prompt is received from a user computing device (“The smart vehicle assistant 102 and the smart responder assistant 104 may be executed by one or more computing systems, as discussed further below. An exemplary architecture of a computing system that may execute one or more elements of the smart vehicle assistant 102 or the smart responder assistant 104 is shown in FIG. 5, and is discussed further with respect to that figure.” (Harvey Para 0057), “FIG. 1 shows an exemplary computing environment 100 associated with at least one of a smart vehicle assistant 102 or a smart responder assistant 104. The smart vehicle assistant 102 may be configured to assist one or more occupants of a vehicle 106, and/or other individuals or entities in association with the vehicle 106. The smart responder assistant 104 may assist one or more response entities, such emergency services personnel, during a response to a collision, accident, or other incident involving the vehicle 106. The smart vehicle assistant 102 and the smart responder assistant 104 may be computer-implemented systems that may receive data, such as user input, sensor data, and/or other data, and that may provide output to users and/or other elements proactively and/or in response to received input.” (Harvey Para 0051)).
In regards to claim 9, Harvey discloses of the vehicle computing system of claim 1, wherein the user prompt is received from a vehicle interface located within the vehicle and physically coupled to the vehicle “FIG. 1 shows an exemplary computing environment 100 associated with at least one of a smart vehicle assistant 102 or a smart responder assistant 104. The smart vehicle assistant 102 may be configured to assist one or more occupants of a vehicle 106, and/or other individuals or entities in association with the vehicle 106. The smart responder assistant 104 may assist one or more response entities, such emergency services personnel, during a response to a collision, accident, or other incident involving the vehicle 106. The smart vehicle assistant 102 and the smart responder assistant 104 may be computer-implemented systems that may receive data, such as user input, sensor data, and/or other data, and that may provide output to users and/or other elements proactively and/or in response to received input.” (Harvey Para 0051), “In some examples, the smart vehicle assistant 102 may be executed at least in part via one or more computing systems that are integrated into the vehicle 106. For example, the vehicle 106 may have one or more on-board processors that may execute one or more elements of the smart vehicle assistant 102. In these examples, a user inside the vehicle 106, such as a driver or other occupant, may use the smart vehicle assistant 102 via a dashboard display of the vehicle 106, integrated speakers and/or microphones of the vehicle 106, and/or other elements of the vehicle 106.” (Harvey Para 0058)).
In regards to claim 10, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 1, wherein the vehicle function comprises at least one of:
emitting an audio response (“In other examples, a user interface 118 may include a non-visual interface, such as an audio-based interface. Accordingly, the smart vehicle assistant 102 and/or the smart responder assistant 104 may present or convey information to users via audio with or without also displaying the information visually via a screen. As an example, the smart vehicle assistant 102 may be an audio-based system that may receive user input as audio voice input captured by a microphone of the vehicle 106 or a user device, and that May audibly present corresponding output voice data via speakers of the vehicle 106 or the user device. Accordingly, in these examples, a user of the smart vehicle assistant 102 may have a voice-based audio conversation with the smart vehicle assistant 102 instead of, or in addition to, interacting with the smart vehicle assistant 102 via a screen or other visual interface. Similarly, a user of the smart responder assistant 104 may have a voice-based audio conversation with the smart responder assistant 104 instead of, or in addition to, interacting with the smart responder assistant 104 via a screen or other visual interface.” (Harvey Para 0068));
updating a user interface within the vehicle (“In other examples, a user interface 118 may include a non-visual interface, such as an audio-based interface. Accordingly, the smart vehicle assistant 102 and/or the smart responder assistant 104 may present or convey information to users via audio with or without also displaying the information visually via a screen. As an example, the smart vehicle assistant 102 may be an audio-based system that may receive user input as audio voice input captured by a microphone of the vehicle 106 or a user device, and that May audibly present corresponding output voice data via speakers of the vehicle 106 or the user device. Accordingly, in these examples, a user of the smart vehicle assistant 102 may have a voice-based audio conversation with the smart vehicle assistant 102 instead of, or in addition to, interacting with the smart vehicle assistant 102 via a screen or other visual interface. Similarly, a user of the smart responder assistant 104 may have a voice-based audio conversation with the smart responder assistant 104 instead of, or in addition to, interacting with the smart responder assistant 104 via a screen or other visual interface.” (Harvey Para 0068));;
providing an entertainment suggestion; or
providing a destination suggestion (“The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Harvey Para 0103), “In some examples, the vehicle chatbot 120 may also, or alternately, provide information about battery charging stations along a planned route and/or potential alternate routes, such as locations of the battery charging stations, costs associated with using the battery charging stations, and/or other information. For instance, as the user is driving the vehicle 106, the user may notice that the battery 108 of the vehicle 106 should soon be recharged, and may ask the vehicle chatbot 120 where the nearest battery charging station is located.” (Harvey Para 0109)).
In regards to claim 11, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 1, wherein the control circuit is configured to:
access the sensor data (“The vehicle 106 may have one or more sensors 110 that are configured to capture corresponding types of sensor data, user input, or other input data. The sensors 110 may include accelerometers and/or other motion sensors, Global Positioning System (GPS) sensors and/or other location sensors, sensors associated with a transmission and/or braking system of the vehicle 106, cameras and/or other image-based sensors, Light Detection and Ranging (LiDAR) sensors, microphones, proximity sensors, weight sensors, seatbelt sensors, seat pressure sensors, payload sensors, and/or other types of sensors. Sensor data, user input, and/or other input data captured by the sensors 110 may be provided to an on-board computing system of the vehicle 106, for instance such that the on-board computing system may perform autonomous or semi-autonomous operations based upon received sensor data. In some examples, as described herein, sensor data, user input, and/or other input data captured by the sensors 110 may also, or alternately, be provided to the smart vehicle assistant 102, such that the smart vehicle assistant 102 may operate based upon the sensor data, user input, and/or other input data.” (Harvey Para 0056), “he additional data 146 may include one or more other types of information, such as weather data, traffic data, map data, image and/or audio data associated with collisions of vehicles, image and/or audio data associated with occupants of vehicles before, during, and/or after collisions, steering and driving data, and/or other types of data. The additional data 146 may be maintained in one or more databases or other data repositories, for instance in one or more databases maintained by an insurance company, an operator of the model training system 134 and/or a provider of the smart vehicle assistant 102 and/or the smart responder assistant 104.” (Harvey Para 0093), “The range data 142 may include information about how far vehicles powered by batteries are able to travel based upon State of Charge (SoC) levels of the batteries and/or other factors. For example, the range data 142 may include historical data indicating how SoC levels of vehicle batteries change over time, and/or how far vehicles have been able to travel based upon power from such vehicle batteries, in association with travel speeds, travel routes, traffic patterns along the travel routes, capabilities of vehicles, and/or other factors. The range data 142 may also include example scripts for communicating tips regarding extending travel ranges and/or battery SoC levels to users of the vehicle chatbot 120. The range data 142 may be maintained in one or more databases or other data repositories, for instance in one or more databases maintained by manufacturers of vehicles and/or batteries, an operator of the model training system 134, and/or a provider of the smart vehicle assistant 102.” (Harvey Para 0091)); and
based on the sensor data, generate an automated user prompt, wherein the automated user prompt is associated with a predicted user prompt from the user (“If the collision responder 128 detects that the vehicle 106 has been involved in a collision, the detection of the collision may prompt the vehicle chatbot 120 to ask questions to occupants of the vehicle 106. For instance, the vehicle chatbot 120 may ask occupants whether they are hurt and/or are in need of medical attention due to the collision, ask the occupants whether they are trapped in the vehicle 106 or may safely get out of the vehicle 106, and/or ask other questions that may help determine the state of the occupants following the collision. In some examples, if the occupants do not or cannot respond to such questions from the vehicle chatbot 120, the collision responder 128 may determine that the occupants are unconscious and/or may be in need of medical attention or other emergency services.” (Harvey Para 0113), (“The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Harvey Para 0103)).
In regards to claim 12, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 11, wherein the control circuit is configured to:
implement the vehicle function in response to the automated user prompt (“If the collision responder 128 detects that the vehicle 106 has been involved in a collision, the detection of the collision may prompt the vehicle chatbot 120 to ask questions to occupants of the vehicle 106. For instance, the vehicle chatbot 120 may ask occupants whether they are hurt and/or are in need of medical attention due to the collision, ask the occupants whether they are trapped in the vehicle 106 or may safely get out of the vehicle 106, and/or ask other questions that may help determine the state of the occupants following the collision. In some examples, if the occupants do not or cannot respond to such questions from the vehicle chatbot 120, the collision responder 128 may determine that the occupants are unconscious and/or may be in need of medical attention or other emergency services.” (Harvey Para 0113), (“The vehicle chatbot 120 may also, or alternately, provide the user with tips on how to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106, for instance by suggesting an alternate travel route, by suggesting that the vehicle 106 travel at reduced speeds, and/or by suggesting other adjustments to operations of the vehicle 106 that, based upon output of the range predictor 124, is expected to extend the range of the vehicle 106 and/or preserve battery life of the vehicle 106. The vehicle chatbot 120 may provide such tips to a user proactively during a conversation, and/or in response to user questions or statements during a conversation indicating that the user may be interested in extending the range of the vehicle 106 and/or preserving battery life of the vehicle 106.” (Harvey Para 0103), “If the vehicle is involved in a collision, the vehicle chatbot may also initiate a 911 call or other emergency communication session on behalf of vehicle occupants who may be unconscious or otherwise unable to engage in the emergency communication session. For example, the vehicle chatbot may engage in a natural language conversation with a 911 operator, to provide information to the 911 operator and/or respond to questions from the 911 operator. In some situations, if the vehicle is an autonomous vehicle and a self-diagnosis indicates that the vehicle is still able to drive autonomously following the collision, the smart vehicle assistant may direct the vehicle to autonomously drive to a hospital or other location, so that occupants of the vehicle may obtain medical services or other assistance.” (Harvey Para 0049)).
In regards to claim 13, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 1, wherein the one or more conditions associated with the user prompt comprises at least one of (i) a cabin temperature, (ii) a comfort setting, or (iii) a navigation preset (“If the vehicle is involved in a collision, the vehicle chatbot may also initiate a 911 call or other emergency communication session on behalf of vehicle occupants who may be unconscious or otherwise unable to engage in the emergency communication session. For example, the vehicle chatbot may engage in a natural language conversation with a 911 operator, to provide information to the 911 operator and/or respond to questions from the 911 operator. In some situations, if the vehicle is an autonomous vehicle and a self-diagnosis indicates that the vehicle is still able to drive autonomously following the collision, the smart vehicle assistant may direct the vehicle to autonomously drive to a hospital or other location, so that occupants of the vehicle may obtain medical services or other assistance.” (Harvey Para 0049), “Accordingly, a user may inquire about a current travel range of the vehicle 106 via the vehicle chatbot 120, and the vehicle chatbot 120 may present information about the current travel range that is generated by the range predictor 124. In some examples, the vehicle chatbot 120 may ask the user questions about when and where the user plans to travel via the vehicle 106, what time the user wants to arrive at a destination, whether the user wants to avoid tolls, heavy traffic, accidents, and/or other elements along a route, and/or other information, such that the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may suggest a route for the user and determine a corresponding travel range to be presented via the vehicle chatbot 120. In other examples, the vehicle chatbot 120 or other elements of the smart vehicle assistant 102 may obtain information about a current or planned travel route via a GPS system of the vehicle or a connected user device, such that the range predictor 124 may generate range predictions that may be presented to the user via the vehicle chatbot 120.” (Harvey Para 0102), see also Harvey Para 0056).
In regards to claims 14 and 20, the claims recite analogous limitations to claim 1, and is therefore rejected on the same premise.
In regards to claims 15-16, the claims recite analogous limitations to claims 2-3, respectively, and are therefore rejected on the same premise.
Claim(s) 4-7 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Harvey in view of Colafrancesco, as applied to claim 1 above, further in view of Tzirkel-Hancock et al. (US 20170287476; hereinafter Tzirkel-Hancock; already of record).
In regards to claim 4, Harvey in view of Colafrancesco teaches of the vehicle computing system of claim 2.
However, Harvey in view of Colafrancesco does not specifically disclose of wherein the context engine is configured to:
determine, based on the user prompt the sensor data, and the user preference data, sentiment data associated with the user; and
generate based on the sentiment data, the modified user prompt.
Tzirkel-Hancock, in the same field of endeavor, teaches of wherein the context engine is configured to:
determine, based on the user prompt the sensor data, and the user preference data, sentiment data associated with the user (“In various embodiments, the vehicle 12 includes a context data acquisition module 26 that communicates with sensors or other systems of the vehicle 12 to capture the context data. The context data indicates a level or mode of automation of the vehicle 12, a vehicle state (e.g., parked, static, moving, in a maneuver, etc.), visibility conditions, road conditions (e.g., rainy, foggy, rough, busy, etc.), driving type (e.g., city, freeway, country roads, etc.), driver state (e.g., distracted or focused as indicated by camera, aware of the car situation or not aware, slurred speech, emotion in speech, etc.), etc. As can be appreciated, these examples of context data and events are merely some examples, as the list may be exhaustive. The disclosure is not limited to the present examples. In various embodiments, the context data acquisition module 26 captures context data and evaluates the context data in realtime.” (Para 0015), see also Para 0021); and
generate based on the sentiment data, the modified user prompt (“In various embodiments, the vehicle 12 includes a context data acquisition module 26 that communicates with sensors or other systems of the vehicle 12 to capture the context data. The context data indicates a level or mode of automation of the vehicle 12, a vehicle state (e.g., parked, static, moving, in a maneuver, etc.), visibility conditions, road conditions (e.g., rainy, foggy, rough, busy, etc.), driving type (e.g., city, freeway, country roads, etc.), driver state (e.g., distracted or focused as indicated by camera, aware of the car situation or not aware, slurred speech, emotion in speech, etc.), etc. As can be appreciated, these examples of context data and events are merely some examples, as the list may be exhaustive. The disclosure is not limited to the present examples. In various embodiments, the context data acquisition module 26 captures context data and evaluates the context data in realtime.” (Para 0015), “The context data acquisition module 26 then communicates the context data to the HMI module 16. In response, the HMI module may optionally alter or add information to the data, and communicate the context data to the speech system 10 through the API 24. The speech system 10 is then updated based on the context data.” (Para 0016), and “Upon completion of speech processing by the speech system 10, the speech system 10 provides a dialog prompt, and a delivery method back to the HMI module 16 of the vehicle 12. The dialog prompt and the delivery method are then further processed by, for example, the HMI module 16 to deliver the prompt to the user or schedule an action by a system of the vehicle 12. By adjusting the delivery method based on the context data, the efficiency of communicating with the user via the speech system 10 is improved during various driving scenarios.” (Para 0017), see also Para 0021).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the modified user prompt, as taught by Harvey in view of Colafrancesco, to include to be based on sentient data associated with the user, as taught by Tzirkel-Hancock, with a reasonable expectation of success in order to have the efficiency of communicating with the user via the speech system 10 is improved during various driving scenarios (Tzirkel-Hancock Para 0017).
In regards to claim 5, Harvey in view of Colafrancesco in view of Tzirkel-Hancock teaches of the vehicle computing system of claim 4, wherein the sentiment data comprises at least one of (i) a mood, (ii) a feeling, or (iii) a tone of the user (“In various embodiments, the vehicle 12 includes a context data acquisition module 26 that communicates with sensors or other systems of the vehicle 12 to capture the context data. The context data indicates a level or mode of automation of the vehicle 12, a vehicle state (e.g., parked, static, moving, in a maneuver, etc.), visibility conditions, road conditions (e.g., rainy, foggy, rough, busy, etc.), driving type (e.g., city, freeway, country roads, etc.), driver state (e.g., distracted or focused as indicated by camera, aware of the car situation or not aware, slurred speech, emotion in speech, etc.), etc. As can be appreciated, these examples of context data and events are merely some examples, as the list may be exhaustive. The disclosure is not limited to the present examples. In various embodiments, the context data acquisition module 26 captures context data and evaluates the context data in realtime.” (Tzirkel-Hancock Para 0015)).
The motivation for combining Harvey, Colafrancesco, and Tzirkel-Hancock is the same as that recited for claim 4 above.
In regards to claim 6, Harvey in view of Colafrancesco in view of Tzirkel-Hancock teaches of the vehicle computing system of claim 1, wherein the control circuit is configured to:
generate voice analysis data for the user prompt, wherein the voice analysis data is indicative of a sentiment of the user (“In various embodiments, the vehicle 12 includes a context data acquisition module 26 that communicates with sensors or other systems of the vehicle 12 to capture the context data. The context data indicates a level or mode of automation of the vehicle 12, a vehicle state (e.g., parked, static, moving, in a maneuver, etc.), visibility conditions, road conditions (e.g., rainy, foggy, rough, busy, etc.), driving type (e.g., city, freeway, country roads, etc.), driver state (e.g., distracted or focused as indicated by camera, aware of the car situation or not aware, slurred speech, emotion in speech, etc.), etc. As can be appreciated, these examples of context data and events are merely some examples, as the list may be exhaustive. The disclosure is not limited to the present examples. In various embodiments, the context data acquisition module 26 captures context data and evaluates the context data in realtime.” (Tzirkel-Hancock Para 0015)).
The motivation for combining Harvey, Colafrancesco, and Tzirkel-Hancock is the same as that recited for claim 4 above.
In regards to claim 7, Harvey in view of Colafrancesco in view of Tzirkel-Hancock teaches of the vehicle computing system of claim 6, wherein the one or more conditions associated with the user prompt comprise the sentiment of the user (“In various embodiments, the vehicle 12 includes a context data acquisition module 26 that communicates with sensors or other systems of the vehicle 12 to capture the context data. The context data indicates a level or mode of automation of the vehicle 12, a vehicle state (e.g., parked, static, moving, in a maneuver, etc.), visibility conditions, road conditions (e.g., rainy, foggy, rough, busy, etc.), driving type (e.g., city, freeway, country roads, etc.), driver state (e.g., distracted or focused as indicated by camera, aware of the car situation or not aware, slurred speech, emotion in speech, etc.), etc. As can be appreciated, these examples of context data and events are merely some examples, as the list may be exhaustive. The disclosure is not limited to the present examples. In various embodiments, the context data acquisition module 26 captures context data and evaluates the context data in realtime.” (Tzirkel-Hancock Para 0015), “The context data acquisition module 26 then communicates the context data to the HMI module 16. In response, the HMI module may optionally alter or add information to the data, and communicate the context data to the speech system 10 through the API 24. The speech system 10 is then updated based on the context data.” (Tzirkel-Hancock Para 0016), and “Upon completion of speech processing by the speech system 10, the speech system 10 provides a dialog prompt, and a delivery method back to the HMI module 16 of the vehicle 12. The dialog prompt and the delivery method are then further processed by, for example, the HMI module 16 to deliver the prompt to the user or schedule an action by a system of the vehicle 12. By adjusting the delivery method based on the context data, the efficiency of communicating with the user via the speech system 10 is improved during various driving scenarios.” (Tzirkel-Hancock Para 0017), see also Tzirkel-Hancock Para 0021).
The motivation for combining Harvey, Colafrancesco, and Tzirkel-Hancock is the same as that recited for claim 4 above.
In regards to claims 17-19, the claims recite analogous limitations to claims 4-6, respectively, and are therefore rejected on the same premise.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kyle J Kingsland whose telephone number is (571)272-3268. The examiner can normally be reached Monday-Friday from 8:00-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn can be reached at (571) 272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE J KINGSLAND/ Primary Examiner, Art Unit 3663
Prosecution Insights