DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-3, 6-13, 16-21 and 23-25 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-3, 6-13, 16-21 and 23-25 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 11, 18, and 20 all describe, “generating a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions and to output the response via the interface to the voice assistant system”.
The scope of this limitation is indefinite because it isn’t clear as to how the combined response can be generated based on the output of (only) some of the plurality of actions while the combined response also comprises an output of each (or all) of the plurality of actions.
The examiner recommends an amendment along the lines of, “generating a combined response in dependence on an output of
The dependent claims are rejected for being indefinite due to inheriting and not correcting the indefiniteness issue from respective independent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-13, 16-21 and 23-25 are rejected under 35 U.S.C. 103 as unpatentable over Ding (CN 202120538745 U), in view of Ni et al. (US 11,450,314), hereinafter “Ni”, and further in view of Carbune et al. (US 2022/0272055), hereinafter “Carbune”.
Regarding claim 1:
Ding teaches:
A control system for a voice assistant system for implementing actions associated with a vehicle, the control system comprising one or more controllers, the control system comprising: an interface configured to receive a request from the voice assistant system, wherein the request is indicative of a task requested by a user of the voice assistant system; and
[page 1, paragraph 1, line 1-5] The utility model relates to a system for realizing vehicle control based on sound source positioning, which comprises a main control unit and a voice pickup unit, wherein the voice pickup unit is used for picking up voice information, and the signal output end of the voice pickup unit is connected with a voice assistant unit; the voice assistant unit is used for identifying sound source positioning and voice instructions, and the signal output end of the voice assistant unit is connected with the main control unit; and the central control unit is used for controlling according to the voice instruction information and the sound source positioning information output by the main control unit, and the signal input end of the central control unit is connected with the main control unit.
Ding also teaches:
[and processing means configured to determine whether the request is for an individual action or for a task that corresponds to a plurality of actions, wherein based upon a determination that the request is for the individual action the processing means causes the control system to perform the individual action and based upon a determination that the request is for the task the processing means is arranged to execute, in dependence on the request, a task handler corresponding to the task, wherein the task handler is arranged to cause the control system to perform the plurality of actions associated with the task,] to generate a response in dependence on an output of at least some of the plurality of actions and to output the response via the interface to the voice assistant system.
([page 4, paragraph 2, line 7-12]...After the CAN signal is received by the vehicle control module, if the driving position of the driver is found to be open, the main driving window is opened, a command is executed and fed back to be executed when the condition is met, a corresponding state signal is fed back to the main control unit MCU when the condition is not met, and the MCU makes a corresponding prompt through the loudspeaker according to the feedback of the signal (if the main driving window is opened).)
Ding doesn’t teach, but Ni teaches:
processing means configured to, prior to execution of the task, determine whether the request is for an individual action or for a task that corresponds to a plurality of actions, the plurality of actions at least including a first action and a second, succeeding action, wherein based upon a determination that the request is for the individual action the processing means causes the control system to perform the individual action and based upon a determination that the request is for the task the processing means is arranged to execute a task handler corresponding to the task, wherein the task handler is arranged to cause the control system to perform the plurality of actions associated with the task, to generate a response in dependence on an output of at least some of the plurality of actions and to output the response via the interface to the voice assistant system
(See col. 6 lns. 23-47: “Implementations set forth herein relate to systems, methods, and apparatus for enabling shortcut commands for an automated assistant. The shortcut commands can be command phrases that are spoken by a user to invoke multiple commands that are available through the automated assistant. For example, a user that operates a computing device can access an automated assistant through the computing device using a spoken command phrase. The command phrase can cause the automated assistant to perform particular actions specified by the user. A user can request the automated assistant perform a variety of different actions in response to a command phrase, and/or consolidate multiple command phrases into a single command phrase. For example, the automated assistant can map a new command phrase to multiple existing command phrases, so that the automated assistant can execute the actions associated with the existing command phrases in response to receiving the new command phrase. Alternatively, the new command phrase can be mapped to actions that are already correlated to the existing command phrases, in order that the automated assistant can perform the actions in response to receiving the new command phrase. In this way, the user does not need to recite each of the existing command phrases to cause the actions to be performed but, rather, can provide a single command phrase to cause the actions to be performed.”
Also see col. 12 lns. 18-30: “In some implementations, the server device 112 can operate a command engine 114 that manages a database of commands that are accessible to the assistant application 118. The database can include entries that correlate command phrases and actions. The command phrases can be speakable commands that, when spoken by a user to the assistant interface 110, can cause the assistant application 118 to perform one or more different actions. In some implementations, the command engine 114 can also modify the database to include shortcut command phrases, when requested by a user, and/or when otherwise tasked with creating a shortcut command phrase by a separate entity (e.g., a developer).”
Further, see col. 13 ln. 49 – col. 14 ln. 16: “In some implementations, the user can request that the assistant application 118 use certain slot value and/or provide a follow up inquiry regarding a particular slot value. For example, the user can provide the command, “Assistant, when I say ‘get ready for movie night,’ you should open my movie application on my phone, order a pizza, and turn on the entertainment system.” In response, the assistant application 118 can identify the slots necessary to fulfill the command “order a pizza,” and provide a feedback response to the user. The feedback response can be, for example, “Ok, when I order the pizza, what size pizza and what toppings would you like?” Thereafter, the user can either request that the assistant application 118 always provide the same order (e.g., “Always order a large pizza with pepperonis.”), or always inquire about some of the slot values (e.g., “Always order a large pizza but ask me about the toppings each time.”). Thereafter, should the user elect to have the assistant application 118 always inquire about one or more of the slot values, the assistant application 118 can prompt the user for those slot values in response to the command phrase, while auto-populating other slot values that were specified by the user. For example, when the user subsequently provides the shortcut command phrase “get ready for movie night,” the assistant application 118 can open the movie application, turn on the entertainment system, and ask the user what toppings they would like on the pizza (but auto-populate the specified “large” slot value for a “pizza size” slot). In this way, the user is able to provide a shortened command phrase (“get ready for movie night”), and in response the assistant application 118 automatically performs a plurality of actions (transmits a command to open the movie application and transmits a command to turn on the entertainment system) and auto-populates a slot value (“large”) for a slot for another command.”
Finally, see col. 15 lns. 18-37: “The user 202 can respond to the automated assistant application's query of “Ok, what can I do for you?” with additional dialog 208 that identifies actions to take to execute the shortcut command phrase. For instance, the additional dialog 208 can include “Open my reading app′, identify the latest couple books I've accessed, and place all my devices in silent mode.” In response, the automated assistant can perform the actions identified and also generate an entry in a database that correlates the actions to the shortcut command phrase of “it's time to read.” The actions to be performed by the automated assistant can include accessing, over a network 212, a reading application 216 that is hosted by a remote device 214. Providing the shortcut command phrase can cause the automated assistant to provide user login data to the reading application 216 in order that the latest viewed books can be identified and announced back to the user 202. The automated assistant can thereafter perform an action of providing a response that identifies the latest viewed books (e.g., “You last opened ‘Catcher in the Rye’ and ‘6 Easy Pieces.’”).”).
Ding and Ni are considered to be analogous to the claimed invention because they are in the same field of virtual assistants. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ding to incorporate the teachings of Ni to propose adding determining if the request is for a single or multiple tasks. Doing so would allow for defining the tasks before execution and running multiple tasks automatically, which improves the user’s experience (see Ni, col. 1 lns. 17-37).
Ding in view of Ni doesn’t describe a system or method including processing means configured to generate a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions and to output the response via the interface to the voice assistant system, wherein the processing means is further configured to identify whether the individual action and at least one additional individual action are invoked within a predetermined period of time and identify whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times and, responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generate a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked.
However, Carbune describes a system and method including
processing means configured to generate a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions and to output the response via the interface to the voice assistant system (See FIG. 5A and ¶ [0099]: “For example, and as depicted in FIG. 5A, the automated assistant can cause the assistant input device 106.sub.1 to render synthesized speech 552A2 by saying “Hi John, I noticed that you are eating breakfast”. Further, assume the automated assistant is sufficiently confident that the user will ask the assistant to provide weather and traffic updates (e.g., based on the corresponding predicted measures associated with the weather and traffic actions satisfy a threshold measure). Accordingly, the automated assistant can automatically perform a weather action and a traffic action based on the user making and/or eating breakfast, and cause synthesized speech 552A3 of “The weather today is a beautiful sunny and 75 degrees” and synthesized speech 552A4 of “Traffic on the way into the office is moderate” to be audibly presented to the user via the assistant input device 106.sub.1 (e.g., as part of a single utterance or multiple utterances). Further assume the automated assistant is sufficiently confident that the user will ask the assistant to start his or her car (e.g., based on the corresponding predicted measures associated with the car start action satisfies a threshold measure). However, the automated assistant may not automatically cause the user's car to be started when the synthesized speech 552A3 and 554A4 to satisfy the other actions is provided. Rather, the automated assistant may cause synthesized speech 552A5 of “I'll start your car for you a few minutes before you typically leave” to be visually and/or audibly presented to the user via the assistant input device 106.sub.1 and subsequently start the user's car before the user leaves. This temporal difference between performance of the weather and traffic actions and performance of the car start action can be learned by the ambient sensing ML model in implementations where training instances include temporal data associated with one or more of the temporally corresponding actions.”),
wherein the processing means is further configured to identify whether the individual action and at least one additional individual action are invoked within a predetermined period of time (¶ [0076]: “Further assume that, as the user 101 is cooking and/or eating, the user 101 provides a spoken utterance 354A1 of “Assistant what's the weather?” at a first time (e.g., time=t1) that invokes the automated assistant 120 causes it to retrieve and present weather information to the user 101, provides a spoken utterance 354A2 of “How's traffic?” at a second time (e.g., time=t2) that causes the automated assistant 120 to retrieve and present traffic information to the user 101, and provides a spoken utterance 354A3 of “Assistant. Start my car” at a third time (e.g., time=t3) that invokes the automated assistant 120 and causes it to automatically start a car of the user 101. In this example, actions associated with the spoken utterances 354A1 (e.g., a weather action), 354A2 (e.g., a traffic action), and 354A3 (e.g., a car start action) can each be considered temporally corresponding actions for the determined ambient state. The actions associated with the spoken utterances 354A1, 354A2, and/or 354A3 can be considered temporally corresponding since they are received within a threshold duration of time of obtaining the audio data utilized to determine the ambient state.” Also see ¶ [0079].) and
identify whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times (¶ [0083]: “At block 262, the system determines whether one or more conditions are satisfied. If, at an iteration of block 262, the system determines that one or more of the conditions are not satisfied, the system continues monitoring for satisfaction of one or more of the conditions at block 262. The one or more conditions can include, for example, that the assistant device is charging, that the assistant device has at least a threshold state of charge, that a temperature of the assistant device (based on one or more on-device temperature sensors) is less than a threshold, that the assistant device is not being held by a user, temporal condition(s) associated with the assistant device(s) (e.g., between a particular time period, every N hours, where N is a positive integer, and/or other temporal condition(s) associated with the assistant device), whether the ambient sensing ML model has been trained based on a threshold number of training instances [emphasis added], and/or other condition(s).”
Also see ¶ [0084]: “Moreover, while the operations of block 262 are depicted as occurring between blocks 260 and block 264, it should be understood that is for the sake of example and is not meant to be limiting. For example, the method 200 may employ multiple instances of block 262 prior to performing the operations of one or more other blocks included in the method 200. For instance, the system may store one or more instances of the sensor data, and withhold from performance of the operations of blocks 254, 256, 258, and 260 until one or more of the conditions are satisfied. Also, for instance, the system may perform the operations of blocks 252, 254, 256, and 258, but withhold from training the ambient sensing ML model until one or more of the conditions are satisfied (e.g., such as whether a threshold quantity of training instances is available for training the ambient sensing ML model) [emphasis added].”) and,
responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generate a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked (¶ [0085]: “If, at an iteration of block 262, the system determines that one or more of the conditions are satisfied, the system proceeds to block 264. At block 264, the system causes the trained ambient sensing ML model to be utilized in generating one or more suggested actions based on one or more additional instances of the sensor data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in Ding in view of Ni a system and method including processing means configured to generate a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions and to output the response via the interface to the voice assistant system, wherein the processing means is further configured to identify whether the individual action and at least one additional individual action are invoked within a predetermined period of time and identify whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times and, responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generate a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked, as taught by Carbune, in order to enable the system to automatically create routines that carry out a series of actions that a user typically requests in specific situations, in order to improve the user’s experience by automatically providing the user with desired information, which avoids the cumbersome process of requesting actions on an individual basis.
Regarding claim 2: (ORIGINAL)
Ding further teaches:
The control system according to claim 1, wherein the processing means is arranged to communicate with the vehicle to perform at least some of the plurality of actions.
[page 4, paragraph 1]: The intelligent control system comprises a central control unit (vehicle control module) for controlling according to voice instruction information and sound source positioning information output by a main control unit, wherein the signal input end of the central control unit is connected with the main control unit through a CAN (controller area network) circuit. The central control unit is used for controlling an air conditioner, a window or a door on the main driving side or the auxiliary driving side.
Regarding claim 3: (ORIGINAL)
Ding further teaches:
The control system according to claim 2, wherein at least some of the plurality of actions each correspond to a request for data from the vehicle and the processing means is arranged to communicate with the vehicle to obtain the data and to generate the response in dependence on the data.
[page 4, paragraph 2, line 7-12] The voice recognition module sends the voice recognition instruction information and the sound source positioning information to the main control unit, and the main control unit transmits the control signal to the central control unit (vehicle control module) through the CAN line. After the CAN signal is received by the vehicle control module, if the driving position of the driver is found to be open [requests data], the main driving window is opened, a command is executed and fed back [generate the response in dependence on the data] to be executed when the condition is met, a corresponding state signal is fed back to the main control unit MCU when the condition is not met, and the MCU makes a corresponding prompt through the loudspeaker according to the feedback of the signal (if the main driving window is opened).)
Regarding claim 6: (ORIGINAL)
Ding further teaches:
The control system according to claim 2, wherein at least some of the plurality of actions each correspond to a request for the vehicle to perform a respective function, and the processing means is arranged to communicate with the vehicle to cause the vehicle to perform the respective functions.
[page 4, paragraph 1]: The intelligent control system comprises a central control unit (vehicle control module) for controlling according to voice instruction information and sound source positioning information output by a main control unit, wherein the signal input end of the central control unit is connected with the main control unit through a CAN (controller area network) circuit. The central control unit is used for controlling an air conditioner, a window or a door on the main driving side or the auxiliary driving side.)
Regarding claim 7: (ORIGINAL)
Ding further teaches:
The control system according to claim 6, wherein the processing means is arranged to receive data indicative of the function being performed by the vehicle and to generate the response in dependence thereon.
[paragraph 4, line 7-12] The voice recognition module sends the voice recognition instruction information and the sound source positioning information to the main control unit, and the main control unit transmits the control signal to the central control unit (vehicle control module) through the CAN line. After the CAN signal is received by the vehicle control module, if the driving position of the driver is found to be open, the main driving window is opened, a command is executed and fed back to be executed when the condition is met [Data indicative of function being performed], a corresponding state signal is fed back to the main control unit MCU when the condition is not met, and the MCU makes a corresponding prompt through the loudspeaker according to the feedback of the signal [generate response in dependence thereon] (if the main driving window is opened).
Regarding claim 8: (ORIGINAL)
Ding further teaches:
The control system according to claim 6, wherein one or more of the functions comprises an instruction to cause the vehicle to actuate an aperture of the vehicle.
[page 4, paragraph 1] The intelligent control system comprises a central control unit (vehicle control module) for controlling according to voice instruction information and sound source positioning information output by a main control unit, wherein the signal input end of the central control unit is connected with the main control unit through a CAN (controller area network) circuit. The central control unit is used for controlling an air conditioner, a window or a door [actuate an aperture of the vehicle] on the main driving side or the auxiliary driving side.)
Regarding claim 9: (ORIGINAL)
Ding in view of Ni in view of Carbune teaches:
A system, comprising:
a control system according to claim 1 (see the rejection of claim 1); and
Ding don’t teach, but Ni teaches:
a server for a voice assistant system arranged to provide the request to the control system indicative of the task requested by the user and to receive the response from the control system (See FIG. 1 and col. 10 lns. 52-58: “Turning now to the figures, FIG. 1 illustrates a system 100 for providing an automated assistant capable of being responsive to shortcut command phrases that can cause the automated assistant to perform multiple different actions. The automated assistant can operate as part of an assistant application 118 that is provided at a computing device 102 or a remote device 124, such as a server device 112.”
Also see col. 11 lns. 23-34: “As another example, a storage entry correlating a shortcut command phrase to one or more actions can be generated through user interface input provided by a user utilizing the assistant interface 110 of the computing device 102 (through interaction with server device 112). Thereafter, the user can provide the shortcut command phrase at a separate computing device, and the storage entry accessed to perform the one or more actions in response. As yet another example, the shortcut command phrase can cause the assistant application 118 to transmit command(s) that cause the state of one or more peripheral devices (e.g., IoT devices) to be altered.”
Finally, see col. 11 lns. 46-64: “The computing device 102 can be in communication with the remote device 124 over a network 122, such as the internet. The computing device 102 can offload computational tasks to the server device 112 in order to conserve computational resources at the computing device 102. For instance, the server device 112 can host the assistant application 118 and the computing device 102 can transmit inputs received at the assistant interface 110 to the server device 112. However, in some implementations, the assistant application 118 can be hosted at the computing device 102. In various implementations, all or less than all aspects of the assistant application 118 can be implemented on the computing device 102. In some of those implementations, aspects of the assistant application 118 are implemented via a local assistant application of the computing device 102 and interface with the server device 112 that implements other aspects of the assistant application 118. The server device 112 can optionally serve a plurality of users and their associated assistant applications via multiple threads.”).
See the rationale to modify Ding based on Ni in the rejection of claim 1, as it is equally applicable here.
Regarding claim 10: (ORIGINAL)
Ding doesn’t teach, but Ni teaches
The system of claim 9, wherein the server is arranged to communicate the request to the control system according to a predetermined routine (See col. 11 lns. 23-34: “As another example, a storage entry correlating a shortcut command phrase to one or more actions can be generated through user interface input provided by a user utilizing the assistant interface 110 of the computing device 102 (through interaction with server device 112). Thereafter, the user can provide the shortcut command phrase at a separate computing device, and the storage entry accessed to perform the one or more actions in response. As yet another example, the shortcut command phrase can cause the assistant application 118 to transmit command(s) that cause the state of one or more peripheral devices (e.g., IoT devices) to be altered.”).
See the rationale to modify Ding based on Ni in the rejection of claim 1, as it is equally applicable here.
Regarding claim 11: (CURRENTLY AMENDED)
A computer-implemented method for a voice assistant system for implementing actions associated with a vehicle, the method comprising:
receiving a request from a voice assistant system, wherein the request is indicative of a task requested by a user of the voice assistant system;
determining, prior to execution of the task, whether the request is for an individual action or for a task that corresponds to a plurality of actions, the plurality of actions at least include a first action and a second, succeeding action, wherein based upon a determination that the request is for the individual action, performing the individual action and based upon a determination that the request is for the task, executing a task handler corresponding to the task and performing the plurality of actions associated with the task, generating a combined response in dependence on an output of at least some of the plurality of actions comprising and output of each of the plurality of actions, and further identifying whether the individual action and at least one additional individual action are invoked within a predetermined period of time and identifying whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times and, responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generating a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked; and
outputting the response to the voice assistant system.
Claim 11 is a computer-implemented method claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale.
Regarding claim 12: (ORIGINAL)
The method according to claim 11, comprising communicating with the vehicle to perform at least some of the plurality of actions.
Claim 12 is a method claim with limitations similar to the limitations of Claim 2 and is rejected under similar rationale.
Regarding claim 13: (ORIGINAL)
The method according to claim 12, wherein at least some of the plurality of actions each correspond to a request for data from the vehicle and the method comprises communicating with the vehicle to obtain the data and generating the response in dependence on the data.
Claim 13 is a method claim with limitations similar to the limitations of Claim 3 and is rejected under similar rationale.
Regarding claim 16: (ORIGINAL)
The method according to claim 12, wherein at least some of the plurality of actions each correspond to a request for the vehicle to perform a respective function, and the method comprises communicating with the vehicle to cause the vehicle to perform the respective functions.
Claim 16 is a method claim with limitations similar to the limitations of Claim 6 and is rejected under similar rationale.
Regarding claim 17: (ORIGINAL)
Ding in view of Ni and further in view of Carbune teaches:
A non-transitory storage medium containing computer readable instructions which, when executed by a computer, are arranged to perform the method according to claim 11.
Ding: [page 4, paragraph 2, line 7-12] The voice recognition module sends the voice recognition instruction information and the sound source positioning information to the main control unit, and the main control unit transmits the control signal to the central control unit (vehicle control module) through the CAN line. After the CAN signal is received by the vehicle control module, if the driving position of the driver is found to be open, the main driving window is opened, a command is executed and fed back to be executed when the condition is met, a corresponding state signal is fed back to the main control unit MCU[this unit is a control unit that would have a non-transitory storage medium containing computer readable instructions] when the condition is not met, and the MCU makes a corresponding prompt through the loudspeaker according to the feedback of the signal (if the main driving window is opened).
See rejections of claims 1 and 11.
Regarding claim 18: (CURRENTLY AMENDED)
Ding teaches:
A control system associated with a voice assistant system, the control system comprising one or more controllers, the control system comprising:
processing means arranged to:
determine a plurality of actions associated with a vehicle performed within [a predetermined period of time in response to a request from a voice assistant system,] the request indicative of a task; and
[page 1, paragraph 1, line 1-5] The utility model relates to a system for realizing vehicle control [i.e. perform tasks of controlling something] based on sound source positioning, which comprises a main control unit and a voice pickup unit, wherein the voice pickup unit is used for picking up voice information, and the signal output end of the voice pickup unit is connected with a voice assistant unit; the voice assistant unit is used for identifying sound source positioning and voice instructions [i.e. perform tasks], and the signal output end of the voice assistant unit is connected with the main control unit; and the central control unit is used for controlling according to the voice instruction information and the sound source positioning information output by the main control unit, and the signal input end of the central control unit is connected with the main control unit.
Ding doesn’t teach but Ni teaches:
determine a plurality of actions performed within a predetermined period of time in response to a request from a voice assistant system, the request indicative of a task (See col. 6 lns. 23-47: “Implementations set forth herein relate to systems, methods, and apparatus for enabling shortcut commands for an automated assistant. The shortcut commands can be command phrases that are spoken by a user to invoke multiple commands that are available through the automated assistant. For example, a user that operates a computing device can access an automated assistant through the computing device using a spoken command phrase. The command phrase can cause the automated assistant to perform particular actions specified by the user. A user can request the automated assistant perform a variety of different actions in response to a command phrase, and/or consolidate multiple command phrases into a single command phrase. For example, the automated assistant can map a new command phrase to multiple existing command phrases, so that the automated assistant can execute the actions associated with the existing command phrases in response to receiving the new command phrase. Alternatively, the new command phrase can be mapped to actions that are already correlated to the existing command phrases, in order that the automated assistant can perform the actions in response to receiving the new command phrase. In this way, the user does not need to recite each of the existing command phrases to cause the actions to be performed but, rather, can provide a single command phrase to cause the actions to be performed.”); and
determine, prior to execution of the task, whether the request from the voice assistant system is for an individual action or for a task that corresponds to the plurality of actions, wherein based upon a determination that the request is for the individual action the processing means causes the control system to perform the individual action and based upon a determination that the request is for the task the processing means generates a task handler corresponding to the task, wherein the task handler is arranged to cause the control system to perform the determined plurality of actions in response to the request from the voice assistant system and generates a response in dependence on an output of at least some of the plurality of actions and to output the response to the voice assistant system (See col. 12 lns. 18-30: “In some implementations, the server device 112 can operate a command engine 114 that manages a database of commands that are accessible to the assistant application 118. The database can include entries that correlate command phrases and actions. The command phrases can be speakable commands that, when spoken by a user to the assistant interface 110, can cause the assistant application 118 to perform one or more different actions. In some implementations, the command engine 114 can also modify the database to include shortcut command phrases, when requested by a user, and/or when otherwise tasked with creating a shortcut command phrase by a separate entity (e.g., a developer).”
Further, see col. 13 ln. 49 – col. 14 ln. 16: “In some implementations, the user can request that the assistant application 118 use certain slot value and/or provide a follow up inquiry regarding a particular slot value. For example, the user can provide the command, “Assistant, when I say ‘get ready for movie night,’ you should open my movie application on my phone, order a pizza, and turn on the entertainment system.” In response, the assistant application 118 can identify the slots necessary to fulfill the command “order a pizza,” and provide a feedback response to the user. The feedback response can be, for example, “Ok, when I order the pizza, what size pizza and what toppings would you like?” Thereafter, the user can either request that the assistant application 118 always provide the same order (e.g., “Always order a large pizza with pepperonis.”), or always inquire about some of the slot values (e.g., “Always order a large pizza but ask me about the toppings each time.”). Thereafter, should the user elect to have the assistant application 118 always inquire about one or more of the slot values, the assistant application 118 can prompt the user for those slot values in response to the command phrase, while auto-populating other slot values that were specified by the user. For example, when the user subsequently provides the shortcut command phrase “get ready for movie night,” the assistant application 118 can open the movie application, turn on the entertainment system, and ask the user what toppings they would like on the pizza (but auto-populate the specified “large” slot value for a “pizza size” slot). In this way, the user is able to provide a shortened command phrase (“get ready for movie night”), and in response the assistant application 118 automatically performs a plurality of actions (transmits a command to open the movie application and transmits a command to turn on the entertainment system) and auto-populates a slot value (“large”) for a slot for another command.”
Finally, see col. 15 lns. 18-37: “The user 202 can respond to the automated assistant application's query of “Ok, what can I do for you?” with additional dialog 208 that identifies actions to take to execute the shortcut command phrase. For instance, the additional dialog 208 can include “Open my reading app′, identify the latest couple books I've accessed, and place all my devices in silent mode.” In response, the automated assistant can perform the actions identified and also generate an entry in a database that correlates the actions to the shortcut command phrase of “it's time to read.” The actions to be performed by the automated assistant can include accessing, over a network 212, a reading application 216 that is hosted by a remote device 214. Providing the shortcut command phrase can cause the automated assistant to provide user login data to the reading application 216 in order that the latest viewed books can be identified and announced back to the user 202. The automated assistant can thereafter perform an action of providing a response that identifies the latest viewed books (e.g., “You last opened ‘Catcher in the Rye’ and ‘6 Easy Pieces.’”).”).
Ding and Ni are considered to be analogous to the claimed invention because they are in the same field of virtual assistants. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ding to incorporate the teachings of Ni to propose adding determining if the request is for a single or multiple tasks. Doing so would allow for defining the tasks before execution and running multiple tasks automatically, which improves the user’s experience (see Ni, col. 1 lns. 17-37).
Ding in view of Ramnani doesn’t describe a system or method including processing means that generates a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions and to output the response via the interface to the voice assistant system, wherein the processing means is further configured to identify whether the individual action and at least one additional individual action are invoked within a predetermined period of time and identify whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times and, responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generate a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked.
However, Carbune describes a system and method including
processing means that generates a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions and to output the response via the interface to the voice assistant system (See FIG. 5A and ¶ [0099]: “For example, and as depicted in FIG. 5A, the automated assistant can cause the assistant input device 106.sub.1 to render synthesized speech 552A2 by saying “Hi John, I noticed that you are eating breakfast”. Further, assume the automated assistant is sufficiently confident that the user will ask the assistant to provide weather and traffic updates (e.g., based on the corresponding predicted measures associated with the weather and traffic actions satisfy a threshold measure). Accordingly, the automated assistant can automatically perform a weather action and a traffic action based on the user making and/or eating breakfast, and cause synthesized speech 552A3 of “The weather today is a beautiful sunny and 75 degrees” and synthesized speech 552A4 of “Traffic on the way into the office is moderate” to be audibly presented to the user via the assistant input device 106.sub.1 (e.g., as part of a single utterance or multiple utterances). Further assume the automated assistant is sufficiently confident that the user will ask the assistant to start his or her car (e.g., based on the corresponding predicted measures associated with the car start action satisfies a threshold measure). However, the automated assistant may not automatically cause the user's car to be started when the synthesized speech 552A3 and 554A4 to satisfy the other actions is provided. Rather, the automated assistant may cause synthesized speech 552A5 of “I'll start your car for you a few minutes before you typically leave” to be visually and/or audibly presented to the user via the assistant input device 106.sub.1 and subsequently start the user's car before the user leaves. This temporal difference between performance of the weather and traffic actions and performance of the car start action can be learned by the ambient sensing ML model in implementations where training instances include temporal data associated with one or more of the temporally corresponding actions.”),
wherein the processing means is further configured to identify whether the individual action and at least one additional individual action are invoked within a predetermined period of time (¶ [0076]: “Further assume that, as the user 101 is cooking and/or eating, the user 101 provides a spoken utterance 354A1 of “Assistant what's the weather?” at a first time (e.g., time=t1) that invokes the automated assistant 120 causes it to retrieve and present weather information to the user 101, provides a spoken utterance 354A2 of “How's traffic?” at a second time (e.g., time=t2) that causes the automated assistant 120 to retrieve and present traffic information to the user 101, and provides a spoken utterance 354A3 of “Assistant. Start my car” at a third time (e.g., time=t3) that invokes the automated assistant 120 and causes it to automatically start a car of the user 101. In this example, actions associated with the spoken utterances 354A1 (e.g., a weather action), 354A2 (e.g., a traffic action), and 354A3 (e.g., a car start action) can each be considered temporally corresponding actions for the determined ambient state. The actions associated with the spoken utterances 354A1, 354A2, and/or 354A3 can be considered temporally corresponding since they are received within a threshold duration of time of obtaining the audio data utilized to determine the ambient state.” Also see ¶ [0079].) and
identify whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times (¶ [0083]: “At block 262, the system determines whether one or more conditions are satisfied. If, at an iteration of block 262, the system determines that one or more of the conditions are not satisfied, the system continues monitoring for satisfaction of one or more of the conditions at block 262. The one or more conditions can include, for example, that the assistant device is charging, that the assistant device has at least a threshold state of charge, that a temperature of the assistant device (based on one or more on-device temperature sensors) is less than a threshold, that the assistant device is not being held by a user, temporal condition(s) associated with the assistant device(s) (e.g., between a particular time period, every N hours, where N is a positive integer, and/or other temporal condition(s) associated with the assistant device), whether the ambient sensing ML model has been trained based on a threshold number of training instances [emphasis added], and/or other condition(s).”
Also see ¶ [0084]: “Moreover, while the operations of block 262 are depicted as occurring between blocks 260 and block 264, it should be understood that is for the sake of example and is not meant to be limiting. For example, the method 200 may employ multiple instances of block 262 prior to performing the operations of one or more other blocks included in the method 200. For instance, the system may store one or more instances of the sensor data, and withhold from performance of the operations of blocks 254, 256, 258, and 260 until one or more of the conditions are satisfied. Also, for instance, the system may perform the operations of blocks 252, 254, 256, and 258, but withhold from training the ambient sensing ML model until one or more of the conditions are satisfied (e.g., such as whether a threshold quantity of training instances is available for training the ambient sensing ML model) [emphasis added].”) and,
responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generate a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked (¶ [0085]: “If, at an iteration of block 262, the system determines that one or more of the conditions are satisfied, the system proceeds to block 264. At block 264, the system causes the trained ambient sensing ML model to be utilized in generating one or more suggested actions based on one or more additional instances of the sensor data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in Ding in view of Ni a system and method including processing means that generates a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions and to output the response via the interface to the voice assistant system, wherein the processing means is further configured to identify whether the individual action and at least one additional individual action are invoked within a predetermined period of time and identify whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times and, responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generate a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked, as taught by Carbune, in order to enable the system to automatically create routines that carry out a series of actions that a user typically requests in specific situations, in order to improve the user’s experience by automatically providing the user with desired information, which avoids the cumbersome process of requesting actions on an individual basis.
Regarding claim 19: (ORIGINAL)
Ding further teaches:
The control system according to claim 18, wherein generating the task handler comprises defining an interface for the task handler to receive data associated with each of the plurality of actions.
[page 4, paragraph 2, line 7-12] The voice recognition module sends the voice recognition instruction information and the sound source positioning information to the main control unit, and the main control unit transmits the control signal to the central control unit (vehicle control module) through the CAN line. After the CAN signal is received by the vehicle control module, if the driving position of the driver is found to be open, the main driving window is opened, a command is executed and fed back to be executed when the condition is met, a corresponding state signal is fed back to the main control unit MCU [the MCU assumes the interface role for the task handler] when the condition is not met, and the MCU makes a corresponding prompt through the loudspeaker according to the feedback of the signal (if the main driving window is opened).
Regarding claim 20: (CURRENTLY AMENDED)
Ding teaches:
A method associated with a voice assistant system, the method comprising:
determining a plurality of actions associated with a vehicle performed within a predetermined period of time in response to a request from a voice assistant system, the request indicative of a task requested by a user of the voice assistant system;
[page 1, paragraph 1, line 1-5] The utility model relates to a system for realizing vehicle control based on sound source positioning, which comprises a main control unit and a voice pickup unit, wherein the voice pickup unit is used for picking up voice information, and the signal output end of the voice pickup unit is connected with a voice assistant unit; the voice assistant unit is used for identifying sound source positioning and voice instructions, and the signal output end of the voice assistant unit is connected with the main control unit; and the central control unit is used for controlling according to the voice instruction information and the sound source positioning information output by the main control unit, and the signal input end of the central control unit is connected with the main control unit.
Ding also teaches:
generating a response in dependence on an output of at least some of the plurality of actions,
[page 4, paragraph 2, line 7-12]...After the CAN signal is received by the vehicle control module, if the driving position of the driver is found to be open, the main driving window is opened, a command is executed and fed back to be executed when the condition is met, a corresponding state signal is fed back to the main control unit MCU when the condition is not met, and the MCU makes a corresponding prompt through the loudspeaker according to the feedback of the signal (if the main driving window is opened).)
Ding doesn’t teach, but Ni teaches:
processing means configured to, prior to execution of the task, determine whether the request is for an individual action or for a task that corresponds to a plurality of actions, the plurality of actions at least including a first action and a second, succeeding action, wherein based upon a determination that the request is for the individual action the processing means causes the control system to perform the individual action and based upon a determination that the request is for the task the processing means is arranged to execute a task handler corresponding to the task, wherein the task handler is arranged to cause the control system to perform the plurality of actions associated with the task, to generate a response in dependence on an output of at least some of the plurality of actions and to output the response via the interface to the voice assistant system
(See col. 6 lns. 23-47: “Implementations set forth herein relate to systems, methods, and apparatus for enabling shortcut commands for an automated assistant. The shortcut commands can be command phrases that are spoken by a user to invoke multiple commands that are available through the automated assistant. For example, a user that operates a computing device can access an automated assistant through the computing device using a spoken command phrase. The command phrase can cause the automated assistant to perform particular actions specified by the user. A user can request the automated assistant perform a variety of different actions in response to a command phrase, and/or consolidate multiple command phrases into a single command phrase. For example, the automated assistant can map a new command phrase to multiple existing command phrases, so that the automated assistant can execute the actions associated with the existing command phrases in response to receiving the new command phrase. Alternatively, the new command phrase can be mapped to actions that are already correlated to the existing command phrases, in order that the automated assistant can perform the actions in response to receiving the new command phrase. In this way, the user does not need to recite each of the existing command phrases to cause the actions to be performed but, rather, can provide a single command phrase to cause the actions to be performed.”
Also see col. 12 lns. 18-30: “In some implementations, the server device 112 can operate a command engine 114 that manages a database of commands that are accessible to the assistant application 118. The database can include entries that correlate command phrases and actions. The command phrases can be speakable commands that, when spoken by a user to the assistant interface 110, can cause the assistant application 118 to perform one or more different actions. In some implementations, the command engine 114 can also modify the database to include shortcut command phrases, when requested by a user, and/or when otherwise tasked with creating a shortcut command phrase by a separate entity (e.g., a developer).”
Further, see col. 13 ln. 49 – col. 14 ln. 16: “In some implementations, the user can request that the assistant application 118 use certain slot value and/or provide a follow up inquiry regarding a particular slot value. For example, the user can provide the command, “Assistant, when I say ‘get ready for movie night,’ you should open my movie application on my phone, order a pizza, and turn on the entertainment system.” In response, the assistant application 118 can identify the slots necessary to fulfill the command “order a pizza,” and provide a feedback response to the user. The feedback response can be, for example, “Ok, when I order the pizza, what size pizza and what toppings would you like?” Thereafter, the user can either request that the assistant application 118 always provide the same order (e.g., “Always order a large pizza with pepperonis.”), or always inquire about some of the slot values (e.g., “Always order a large pizza but ask me about the toppings each time.”). Thereafter, should the user elect to have the assistant application 118 always inquire about one or more of the slot values, the assistant application 118 can prompt the user for those slot values in response to the command phrase, while auto-populating other slot values that were specified by the user. For example, when the user subsequently provides the shortcut command phrase “get ready for movie night,” the assistant application 118 can open the movie application, turn on the entertainment system, and ask the user what toppings they would like on the pizza (but auto-populate the specified “large” slot value for a “pizza size” slot). In this way, the user is able to provide a shortened command phrase (“get ready for movie night”), and in response the assistant application 118 automatically performs a plurality of actions (transmits a command to open the movie application and transmits a command to turn on the entertainment system) and auto-populates a slot value (“large”) for a slot for another command.”
Finally, see col. 15 lns. 18-37: “The user 202 can respond to the automated assistant application's query of “Ok, what can I do for you?” with additional dialog 208 that identifies actions to take to execute the shortcut command phrase. For instance, the additional dialog 208 can include “Open my reading app′, identify the latest couple books I've accessed, and place all my devices in silent mode.” In response, the automated assistant can perform the actions identified and also generate an entry in a database that correlates the actions to the shortcut command phrase of “it's time to read.” The actions to be performed by the automated assistant can include accessing, over a network 212, a reading application 216 that is hosted by a remote device 214. Providing the shortcut command phrase can cause the automated assistant to provide user login data to the reading application 216 in order that the latest viewed books can be identified and announced back to the user 202. The automated assistant can thereafter perform an action of providing a response that identifies the latest viewed books (e.g., “You last opened ‘Catcher in the Rye’ and ‘6 Easy Pieces.’”).”).
Ding and Ni are considered to be analogous to the claimed invention because they are in the same field of virtual assistants. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ding to incorporate the teachings of Ni to propose adding determining if the request is for a single or multiple tasks. Doing so would allow for defining the tasks before execution and running multiple tasks automatically, which improves the user’s experience (see Ni, col. 1 lns. 17-37).
Ding in view of Ni doesn’t describe a system or method including generating a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions, and further identifying whether the individual action and at least one additional individual action are invoked within a predetermined period of time and identifying whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times and, responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generating a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked.
However, Carbune describes a system and method including
generating a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions (See FIG. 5A and ¶ [0099]: “For example, and as depicted in FIG. 5A, the automated assistant can cause the assistant input device 106.sub.1 to render synthesized speech 552A2 by saying “Hi John, I noticed that you are eating breakfast”. Further, assume the automated assistant is sufficiently confident that the user will ask the assistant to provide weather and traffic updates (e.g., based on the corresponding predicted measures associated with the weather and traffic actions satisfy a threshold measure). Accordingly, the automated assistant can automatically perform a weather action and a traffic action based on the user making and/or eating breakfast, and cause synthesized speech 552A3 of “The weather today is a beautiful sunny and 75 degrees” and synthesized speech 552A4 of “Traffic on the way into the office is moderate” to be audibly presented to the user via the assistant input device 106.sub.1 (e.g., as part of a single utterance or multiple utterances). Further assume the automated assistant is sufficiently confident that the user will ask the assistant to start his or her car (e.g., based on the corresponding predicted measures associated with the car start action satisfies a threshold measure). However, the automated assistant may not automatically cause the user's car to be started when the synthesized speech 552A3 and 554A4 to satisfy the other actions is provided. Rather, the automated assistant may cause synthesized speech 552A5 of “I'll start your car for you a few minutes before you typically leave” to be visually and/or audibly presented to the user via the assistant input device 106.sub.1 and subsequently start the user's car before the user leaves. This temporal difference between performance of the weather and traffic actions and performance of the car start action can be learned by the ambient sensing ML model in implementations where training instances include temporal data associated with one or more of the temporally corresponding actions.”), and
further identifying whether the individual action and at least one additional individual action are invoked within a predetermined period of time (¶ [0076]: “Further assume that, as the user 101 is cooking and/or eating, the user 101 provides a spoken utterance 354A1 of “Assistant what's the weather?” at a first time (e.g., time=t1) that invokes the automated assistant 120 causes it to retrieve and present weather information to the user 101, provides a spoken utterance 354A2 of “How's traffic?” at a second time (e.g., time=t2) that causes the automated assistant 120 to retrieve and present traffic information to the user 101, and provides a spoken utterance 354A3 of “Assistant. Start my car” at a third time (e.g., time=t3) that invokes the automated assistant 120 and causes it to automatically start a car of the user 101. In this example, actions associated with the spoken utterances 354A1 (e.g., a weather action), 354A2 (e.g., a traffic action), and 354A3 (e.g., a car start action) can each be considered temporally corresponding actions for the determined ambient state. The actions associated with the spoken utterances 354A1, 354A2, and/or 354A3 can be considered temporally corresponding since they are received within a threshold duration of time of obtaining the audio data utilized to determine the ambient state.” Also see ¶ [0079].) and
identifying whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times (¶ [0083]: “At block 262, the system determines whether one or more conditions are satisfied. If, at an iteration of block 262, the system determines that one or more of the conditions are not satisfied, the system continues monitoring for satisfaction of one or more of the conditions at block 262. The one or more conditions can include, for example, that the assistant device is charging, that the assistant device has at least a threshold state of charge, that a temperature of the assistant device (based on one or more on-device temperature sensors) is less than a threshold, that the assistant device is not being held by a user, temporal condition(s) associated with the assistant device(s) (e.g., between a particular time period, every N hours, where N is a positive integer, and/or other temporal condition(s) associated with the assistant device), whether the ambient sensing ML model has been trained based on a threshold number of training instances [emphasis added], and/or other condition(s).”
Also see ¶ [0084]: “Moreover, while the operations of block 262 are depicted as occurring between blocks 260 and block 264, it should be understood that is for the sake of example and is not meant to be limiting. For example, the method 200 may employ multiple instances of block 262 prior to performing the operations of one or more other blocks included in the method 200. For instance, the system may store one or more instances of the sensor data, and withhold from performance of the operations of blocks 254, 256, 258, and 260 until one or more of the conditions are satisfied. Also, for instance, the system may perform the operations of blocks 252, 254, 256, and 258, but withhold from training the ambient sensing ML model until one or more of the conditions are satisfied (e.g., such as whether a threshold quantity of training instances is available for training the ambient sensing ML model) [emphasis added].”) and,
responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generate a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked (¶ [0085]: “If, at an iteration of block 262, the system determines that one or more of the conditions are satisfied, the system proceeds to block 264. At block 264, the system causes the trained ambient sensing ML model to be utilized in generating one or more suggested actions based on one or more additional instances of the sensor data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in Ding in view of Ni a system and method including generating a combined response in dependence on an output of at least some of the plurality of actions comprising an output of each of the plurality of actions, and further identifying whether the individual action and at least one additional individual action are invoked within a predetermined period of time and identifying whether the individual action and at least one additional individual action are performed in temporal proximity more than a predetermined number of times and, responsive to identifying that the individual action and the at least one additional individual action are invoked within the predetermined period of time and identifying that the individual action and the at least one additional individual action are performed in temporal proximity more than the predetermined number of times, generating a new task handler associated with a task that is configured to perform the individual action and at least one additional individual action when the new task handler is invoked, as taught by Carbune, in order to enable the system to automatically create routines that carry out a series of actions that a user typically requests in specific situations, in order to improve the user’s experience by automatically providing the user with desired information, which avoids the cumbersome process of requesting actions on an individual basis.
Regarding claim 21: (PREVIOUSLY PRESENTED)
Ding doesn’t teach but Ni teaches the additional limitation:
The control system according to claim 1, wherein the plurality of actions are predetermined (see col. 13 ln. 49 – col. 14 ln. 16: “In some implementations, the user can request that the assistant application 118 use certain slot value and/or provide a follow up inquiry regarding a particular slot value. For example, the user can provide the command, “Assistant, when I say ‘get ready for movie night,’ you should open my movie application on my phone, order a pizza, and turn on the entertainment system.” In response, the assistant application 118 can identify the slots necessary to fulfill the command “order a pizza,” and provide a feedback response to the user. The feedback response can be, for example, “Ok, when I order the pizza, what size pizza and what toppings would you like?” Thereafter, the user can either request that the assistant application 118 always provide the same order (e.g., “Always order a large pizza with pepperonis.”), or always inquire about some of the slot values (e.g., “Always order a large pizza but ask me about the toppings each time.”). Thereafter, should the user elect to have the assistant application 118 always inquire about one or more of the slot values, the assistant application 118 can prompt the user for those slot values in response to the command phrase, while auto-populating other slot values that were specified by the user. For example, when the user subsequently provides the shortcut command phrase “get ready for movie night,” the assistant application 118 can open the movie application, turn on the entertainment system, and ask the user what toppings they would like on the pizza (but auto-populate the specified “large” slot value for a “pizza size” slot). In this way, the user is able to provide a shortened command phrase (“get ready for movie night”), and in response the assistant application 118 automatically performs a plurality of actions (transmits a command to open the movie application and transmits a command to turn on the entertainment system) and auto-populates a slot value (“large”) for a slot for another command.”).
Ding and Ni are considered to be analogous to the claimed invention because they are in the same field of virtual assistants. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ding to incorporate the teachings of Ni to propose adding determining if the request is for a single or multiple tasks. Doing so would allow for defining the tasks before execution.
Regarding claims 23 and 25, Ding in view of Ni doesn’t describe the control system according to claim 1, or the method according to claim 11, wherein each of the plurality of actions is independently invokable by the voice assistant system, and wherein the processing means is further configured to store data indicative of an invocation time of each of the plurality of actions.
However, Carbune describes the control system according to claim 1, and the method according to claim 11, wherein each of the plurality of actions is independently invokable by the voice assistant system, and wherein the processing means is further configured to store data indicative of an invocation time of each of the plurality of actions (See ¶ [0076]: “Further assume that, as the user 101 is cooking and/or eating, the user 101 provides a spoken utterance 354A1 of “Assistant what's the weather?” at a first time (e.g., time=t1) that invokes the automated assistant 120 causes it to retrieve and present weather information to the user 101, provides a spoken utterance 354A2 of “How's traffic?” at a second time (e.g., time=t2) that causes the automated assistant 120 to retrieve and present traffic information to the user 101, and provides a spoken utterance 354A3 of “Assistant. Start my car” at a third time (e.g., time=t3) that invokes the automated assistant 120 and causes it to automatically start a car of the user 101. In this example, actions associated with the spoken utterances 354A1 (e.g., a weather action), 354A2 (e.g., a traffic action), and 354A3 (e.g., a car start action) can each be considered temporally corresponding actions for the determined ambient state. The actions associated with the spoken utterances 354A1, 354A2, and/or 354A3 can be considered temporally corresponding since they are received within a threshold duration of time of obtaining the audio data utilized to determine the ambient state. Notably, the threshold duration of time for identifying temporally corresponding actions can vary. For instance, assume that the spoken utterance 354A1 is received when the user begins cooking breakfast, and that the spoken utterance 354A2 is received as a follow-up utterance to the spoken utterance 354A1 such that the user needs not re-invoke the automated assistant 120 (e.g., via a hotword, software button, hardware button, gesture-based invocation, etc.) at the second time. Further assume the third time when the spoken utterance 354A3 is received is after the user 101 finishes eating 20 minutes later. In some instances, the third time can still be considered temporally corresponding because the sensor data indicates a breakfast ambient state is still applicable to the user 101 and/or the environment of the user 101. Thus, any user-initiated performed before, during, and/or after the instance of the sensor data is obtained can be considered a temporally corresponding action for the ambient state.”).
See the rationale in claim 1 to modify Ding in view of Ni based on the teachings of Carbune, as it is equally applicable here.
Regarding claim 24, Ding in view of Ni doesn’t describe the control system according to claim 1, wherein the predetermined period of time commences upon invocation of the individual action.
However, Carbune describes the control system according to claim 1, wherein the predetermined period of time commences upon invocation of the individual action (See ¶ [0076]: “Further assume that, as the user 101 is cooking and/or eating, the user 101 provides a spoken utterance 354A1 of “Assistant what's the weather?” at a first time (e.g., time=t1) that invokes the automated assistant 120 causes it to retrieve and present weather information to the user 101, provides a spoken utterance 354A2 of “How's traffic?” at a second time (e.g., time=t2) that causes the automated assistant 120 to retrieve and present traffic information to the user 101, and provides a spoken utterance 354A3 of “Assistant. Start my car” at a third time (e.g., time=t3) that invokes the automated assistant 120 and causes it to automatically start a car of the user 101. In this example, actions associated with the spoken utterances 354A1 (e.g., a weather action), 354A2 (e.g., a traffic action), and 354A3 (e.g., a car start action) can each be considered temporally corresponding actions for the determined ambient state. The actions associated with the spoken utterances 354A1, 354A2, and/or 354A3 can be considered temporally corresponding since they are received within a threshold duration of time of obtaining the audio data utilized to determine the ambient state. Notably, the threshold duration of time for identifying temporally corresponding actions can vary. For instance, assume that the spoken utterance 354A1 is received when the user begins cooking breakfast, and that the spoken utterance 354A2 is received as a follow-up utterance to the spoken utterance 354A1 such that the user needs not re-invoke the automated assistant 120 (e.g., via a hotword, software button, hardware button, gesture-based invocation, etc.) at the second time. Further assume the third time when the spoken utterance 354A3 is received is after the user 101 finishes eating 20 minutes later. In some instances, the third time can still be considered temporally corresponding because the sensor data indicates a breakfast ambient state is still applicable to the user 101 and/or the environment of the user 101. Thus, any user-initiated performed before, during, and/or after the instance of the sensor data is obtained can be considered a temporally corresponding action for the ambient state.”
Also see ¶ [0079]: “Further assume that, as the user 101 is outside, the user 101 pairs a mobile device with Bluetooth headphones at a first time (e.g., time=t1) as indicated by 352B1, provides a spoken utterance 35462 of “Assistant, lock the back door” at a second time (e.g., time=t2) that causes the automated assistant 120 to lock the back door via the smart lock 185.sub.4, and initiates tracking a run with a 3P software application at a third time (e.g., time=t3) as indicated by 352B3. In this example, actions associated with the pairing 35461 (e.g., a pairing action), the spoken utterance 35462 (e.g., a door lock action), and the 3P software application 354A3 (e.g., a run tracking action) can each be considered temporally corresponding actions for the determined ambient state. These actions can be considered temporally corresponding since they are received within a threshold duration of time of obtaining the motion data utilized to determine the ambient state and/or the user 101 leaving the primary dwelling. Notably, the threshold duration of time for identifying temporally corresponding actions can vary. For instance, and in contrast with FIG. 3A, the first time, the second time, and the third time can be relatively shorter since the actions indicated by 352B1, 352B2, and/or 352B3 are generally performed within a shorter duration of time. Thus, any user-initiated performed before, during, and/or after the instance of the sensor data is obtained can be considered a temporally corresponding action for the ambient state.”).
See the rationale in claim 1 to modify Ding in view of Ni based on the teachings of Carbune, as it is equally applicable here.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniel C Washburn whose telephone number is (571)272-5551. The examiner can normally be reached Monday-Friday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL C WASHBURN/Supervisory Patent Examiner, Art Unit 2657
Prosecution Insights