DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Amendment filed on June 18, 2026. Claims 1-5, 9-12, and 14-47 are pending in the case. Claims 1-5, 9, 11, 12, 14-16, and 18-23 are amended. Claims 6-8 and 13 are cancelled. Claims 24-47 are new. Claims 1, 22, and 23 are the independent claims.
This action is final.
Applicant’s Response
In the amendment filed on June 18, 2026, Applicant amended the claims and provided arguments in response to the rejections of the claims under 35 USC 103 in the previous office action.
Response to Argument/Amendment
Applicant’s arguments in response to the rejection of the claims under 35 USC 103 is acknowledged, and Applicant’s associated arguments have been fully considered. Applicant argues that Kannan, Trufinescu, Gada, Novitchenko, and Gray are silent with regard to the newly recited limitations “wherein the context of the electronic device includes a content displayed on a display of the electronic device that is referenced by the request and the content displayed on the display of the electronic device includes a plurality of objects associated with applications,” and determining, based on the context referenced by the request, a first textual representation of a first utterance for performing a first task utilizing a first application associated with a first object included in the content displayed on the display of the electronic device and a second textual representation of a second utterance for performing a second task utilizing a second application associated with a second object included in the content displayed on the display of the electronic device, wherein the second task is different from the first task and the second application is different from the first application,” in the amended independent claims.
Applicant’s argument is persuasive, and the rejection under 35 USC 103 is withdrawn. However, new grounds of rejection are provided below.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102€, (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 1, 11, 12, 14, 18, 19-23, 28-30, 33-35, 40-42, and 45-47 are rejected under 35 U.S.C. 103 as being unpatentable over Kannan et al. (US 20160189717 A1) in view of Trufinescu et al. (US 20180131643 A1), further in view of Gada et al. (US 20200380985 A1), further in view of Krishnan et al. (US 20200356339 A1).
With respect to claims 1, 22, and 23, Kannan teaches an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method; a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for performing a method; and the method, comprising: at an electronic device with one or more processors and memory (e.g. paragraph 0078, computing system including processing units and memory; paragraph s0098-0099, computer-executable instructions stored on computer-readable storage media and executed on computing device; memory/storage):
receiving a user input including a request for potential tasks executable by a digital assistant and a context of the electronic device (e.g. paragraph 0028, extensible digital personal assistant not limited to operating system features/services and is extended to support third party voice enabled applications; paragraph 0041, range of tasks that can be performed on behalf of user by digital assistant extended to include commands for performing available tasks of third party voice enabled applications; paragraph 0049, user is trying to discover which tasks a particular application can perform; paragraph 0051, user providing voice input as shown at 325 of Fig. 3; user voice request in this example is “Which tasks can <application> perform?” as depicted at 330; paragraph 0065, Fig. 6, at 610 a digital voice input is received by a voice-controlled digital personal assistant; paragraph 0066, at 620, determining user voice request for available tasks capable of being performed by voice-enabled application; i.e. receiving a request for potential tasks which the assistant can perform, including tasks which the assistant can perform via another application, where the request additionally includes contextual information identifying a particular application which the user is requesting commands for);
in response to receiving the request:
determining, based on the context referenced by the request, a textual representation of an utterance for performing a task (e.g. paragraph 0040, voice command data structure identifying voice enabled applications and supported commands along with associated voice command variations and examples; paragraph 0052, Fig. 3, assistant determining which tasks the particular application is capable of performing, including example voice commands along with voice examples illustrating how the user can use a given voice command as shown at 340; controlling which voice commands are displayed, which variations etc.; paragraph 0067, at 630, available tasks capable of being performed by voice-enabled application are identified; paragraph 0067, associated voice command variations and voice command examples; i.e. determining which voice commands and variations are available for the particular application (as provided with the request as contextual information) and how the commands/variations should be presented in textual form on a user interface); and
providing the textual representation of the utterance for performing the task in an affordance displayed over a user interface on the display of the electronic device (e.g. paragraph 0052, Fig. 3, third graphical user interface displayed listing tasks at 340 including example voice commands/variations; paragraph 0068, at 640, response provided to the user via GUI identifying available tasks, such as displayed list of tasks along with associated voice command variations and voice command examples).
Kannan does not explicitly disclose wherein the context of the electronic device includes a content displayed on a display of the electronic device that is referenced by the request. However, Trufinescu teaches wherein the context of the electronic device includes a content displayed on a display of the electronic device that is referenced by the request (e.g. paragraph 0024, Fig. 3, user clicking on graphical selector for MOVIE1, and session state of application user interface changed to present specific content for MOVIE1; paragraph 0030, Fig. 3, query 48 from user includes word or phrase and the bot client program is unable to resolve the meaning of the word or phrase within threshold confidence level, and in response, sending context request to application program; example query 48A includes the phrase “When was this movie made?”; paragraph 0032, determining whether query is directed to content related to the session state of the application program; paragraph 0033, sending context request to application program; application program sending current context data, which may include current content 38 presented via the application user interface of the application program; paragraph 0034, Fig. 3, query is “When was this movie made?” and context data includes content for the Movie 1 that is currently being presented by the application user interface).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan and Trufinescu in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), to incorporate the teachings of Trufinescu (directed to application context aware chatbots) to include the capability to include, in the context, content displayed on a display of the electronic device that is referenced by the request (as taught by Trufinescu). One of ordinary skill would have been motivated to perform such a modification in order to enable contextual information about application programs to be communicated to and leveraged by chatbots to process user queries with greater effectiveness and the functionality of the application program and chatbot are extended as described in Trufinescu (paragraph 0010).
Kannan and Trufinescu do not explicitly disclose that the textual representation of the utterance for performing the task is determined by utilizing the content displayed on the display of the electronic device and that the providing of the textual representation of the utterance for performing the task is performed by utilizing the content displayed on the display of the electronic device. However, Gada teaches that the textual representation of the utterance for performing the task is determined by utilizing the content displayed on the display of the electronic device and that the providing of the textual representation of the utterance for performing the task is performed by utilizing the content displayed on the display of the electronic device (e.g. paragraph 0291-0292, describing Fig. 8C-D, receiving voice input of “Show me what to say” and in response displaying UI of Fig. 8D, in which suggestion interface 802B is displayed including visual cues/suggestions each of which corresponds to a valid voice command that is available while the device is in the context shown in Fig. 8D; visual cue 802B1 “Hide Grid” is displayed because current device context includes a displayed grid; i.e. the suggested utterance/voice command is determined and displayed to the user based on/utilizing the content currently displayed on the display of the device, such as providing a “Hide grid” command/utterance as a suggestion based on the device context including the device currently displaying the grid as content).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, and Gada in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources) and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Gada (directed to voice-based control of electronic devices) to include the capability to determine and provide the textual representation of the utterance/command for performing a suggested task by utilizing the content currently displayed on the display of the device (as taught by Gada). One of ordinary skill would have been motivated to perform such a modification in order to provide faster more efficient methods and interfaces for voice-based control of electronic devices, reducing cognitive burden on the user and producing a more efficient human-machine interface as described in Gada (paragraph 0005).
Kannan, Trufinescu, and Gada do not explicitly disclose
wherein the content displayed on the display of the electronic device includes a plurality of objects associated with the applications;
the textual representation of the utterance for performing the first task utilizing the content displayed is a first textual representation of a first utterance for performing a first task utilizing a first application associated with a first object included in the content displayed on the display of the electronic device;
determining a second textual representation of a second utterance for performing a second task utilizing a second application associated with a second object included in the content displayed on the display of the electronic device, wherein the second task is different from the first task and the second application is different from the first application;
providing the first textual representation and the second textual representation in an affordance displayed over a user interface on the display of the electronic device.
However, Krishnan teaches
wherein the content displayed on the display of the electronic device includes a plurality of objects associated with the applications (e.g. paragraph 0022, Fig. 1B, displaying a content related to a navigation application in a first area 110 of the GUI and content related to a messaging/communication application in a second areas 114 of the GUI; navigation application data rendered as first graphical element 112 in first area 110 and messaging application data rendered as second graphical element 116);
the textual representation of the utterance for performing the first task utilizing the content displayed is a first textual representation of a first utterance for performing a first task utilizing a first application associated with a first object included in the content displayed on the display of the electronic device (e.g. paragraph 0025, Fig. 1B, automated assistant providing suggestion data characterizing a first suggestion element 132 characterizing a spoken utterance that, when spoken, causes the automated assistance to initialize or perform a particular action/another application to perform the action; first suggestion element 132 characterizes spoken utterance such as “Assistant, what is my ETA?”; paragraph 0026, in response to user selecting/uttering input corresponding to suggestion element 132, assistant communicating estimated time of arrival to user; in this way, user can be put on notice that they can employ the automated assistant to initialize actions of the OEM vehicle applications 124 and/or the third-party applications 126, and/or otherwise perform actions based on information provided by the vehicle application 124 and/or the third-party applications 126; paragraph 0032, automated assistant generating suggestions for spoken utterances that can be provided to user for invoking the automated assistant to initialize performance of an action by another OEM application or third party application);
determining a second textual representation of a second utterance for performing a second task utilizing a second application associated with a second object included in the content displayed on the display of the electronic device, wherein the second task is different from the first task and the second application is different from the first application (e.g. paragraph 0006, suggestion element for invoking messaging application; paragraph 0025, Fig. 1B, automated assistant providing suggestion data characterizing a second suggestion element 134 which characterizes a different spoken utterance that, when spoken, causes the automated assistant to perform a different particular action; paragraph 0026, second suggestion element 134 characterizes spoken utterance such as “Assistant, reply to Jane”; in response to user selecting/uttering input corresponding to suggestion element 134, assistant prompting user for additional content for message; in this way, user can be put on notice that they can employ the automated assistant to initialize actions of the OEM vehicle applications 124 and/or the third-party applications 126, and/or otherwise perform actions based on information provided by the vehicle application 124 and/or the third-party applications 126);
providing the first textual representation and the second textual representation in an affordance displayed over a user interface on the display of the electronic device (e.g. suggestion elements 132 and 134, as cited above and shown in Fig. 1B).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, and Krishnan in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Trufinescu (directed to application context aware chatbots), and Gada (directed to voice-based control of electronic devices), to incorporate the teachings of Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices) to include the capability to utilize context information including multiple objects associated with respective applications in order to determine first and second utterances, each for performing a different corresponding task utilizing a different corresponding application associated with a corresponding object on the screen. One of ordinary skill would have been motivated to perform such a modification in order to provide improved reliability while also improving user privacy and data security, and in order to provide useful suggestions characterizing actions that are relevant to content provided in a GUI as described in Krishnan (paragraph 0003, 0033, 0036).
With respect to claims 11, 28, and 40, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed, and Kannan further teaches wherein the one or more programs further include instructions for: determining a third textual representation of a third utterance for performing a third task, wherein the third task is not performed with the content displayed on the display of the electronic device (e.g. paragraph 0040, voice command data structure identifying voice enabled applications and supported commands along with associated voice command variations and examples; paragraph 0052, Fig. 3, assistant determining which tasks the particular application is capable of performing, including example voice commands along with voice examples illustrating how the user can use a given voice command as shown at 340; controlling which voice commands are displayed, which variations etc.; paragraph 0067, at 630, available tasks capable of being performed by voice-enabled application are identified; paragraph 0067, associated voice command variations and voice command examples; as shown in Fig. 3 at 324/340, the determined list of tasks for the particular application includes at three different tasks along with three different corresponding commands/utterances for performing the respective tasks, such as a first task to add a movie to a queue in the application along with a corresponding command, a second task to play a movie in the application along with a corresponding command, and a third task to search for a movie in the application, where these tasks are performed using content of the corresponding application, which is not currently displayed on the screen).
With respect to claim 12, 29, and 41, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed, and Kannan further teaches wherein determining, based on the context referenced by the request, the first textual representation of the first utterance for performing the first task further comprises: determining the first application for performing the first task by identifying an application included in the request (e.g. paragraph 0049, user is trying to discover which tasks a particular application can perform; paragraph 0051, user providing voice input as shown at 325 of Fig. 3; user voice request in this example is “Which tasks can <application> perform?” as depicted at 330; paragraph 0066, at 620, determining user voice request for available tasks capable of being performed by specific voice-enabled application; paragraph 0067, identifying tasks capable of being performed by the specific application).
With respect to claims 14, 30, and 42, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed, and Krishnan further teaches wherein an order of the first textual representation of the first utterance and the second textual representation of the second utterance is based on a position of the first object and a position of the second object displayed on the display of the electronic device (e.g. paragraph 0021, Fig. 1B, GUI with different areas corresponding to different applications; paragraph 0025, providing first and second suggestion elements 132 and 134; i.e. as shown in Fig. 1B, first suggestion element 132 is displayed in the GUI area associated with the navigation application, adjacent to the first object 112 associated with the navigation application and second suggestion element 134 is displayed in the GUI area associated with the communication/messaging application, adjacent to the second object 116 associated with the communication/messaging application).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, and Krishnan in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Trufinescu (directed to application context aware chatbots), and Gada (directed to voice-based control of electronic devices), to incorporate the teachings of Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices) to include the capability to display the representations of the first and second utterances based on respective positions of corresponding objects associated with respective applications. One of ordinary skill would have been motivated to perform such a modification in order to provide improved reliability while also improving user privacy and data security, and in order to provide useful suggestions characterizing actions that are relevant to content provided in a GUI as described in Krishnan (paragraph 0003, 0033, 0036).
With respect to claims 18, 33, and 45, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed, and Kannan further teaches wherein the one or more programs further include instructions for: receiving a user input including the first utterance; and in response to receiving the user input including the first utterance, performing the first task (e.g. paragraph 0056, Fig. 4, as depicted at 440, user provides voice input for voice command; in response the assistant determines the voice command, compares to available commands, and then presents results of the command to the user as shown at 445).
With respect to claim 19, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claim 18 as previously discussed, and Kannan further teaches wherein the one or more programs further include instructions for:
receiving a second user input including the request for potential tasks executable by the digital assistant and the context of the electronic device (e.g. paragraph 0028, extensible digital personal assistant not limited to operating system features/services and is extended to support third party voice enabled applications; paragraph 0041, range of tasks that can be performed on behalf of user by digital assistant extended to include commands for performing available tasks of third party voice enabled applications; paragraph 0049, user is trying to discover which tasks a particular application can perform; paragraph 0051, user providing voice input as shown at 325 of Fig. 3; user voice request in this example is “Which tasks can <application> perform?” as depicted at 330; paragraph 0065, Fig. 6, at 610 a digital voice input is received by a voice-controlled digital personal assistant; paragraph 0066, at 620, determining user voice request for available tasks capable of being performed by voice-enabled application; i.e. receiving a request for potential tasks which the assistant can perform, including tasks which the assistant can perform via another application, where the request additionally includes contextual information identifying a particular application which the user is requesting commands for; i.e. as shown in Fig. 2 at 240, multiple different applications are included on the device (i.e. Netflix, Hulu Plus, Amazon Instant Video), such that the user can provide a different/second request as shown in Fig. 3 regarding which tasks are performable in a different application from that specified in the first request, or can provide another request regarding the same application at a subsequent time; in addition, the context may be a context in which a user is asking about available functionalities in third party applications in general);
in response to receiving the second user input including the request for potential tasks executable by the digital assistant in the context of the electronic device:
determining, based on the context referenced by the request, a textual representation of another utterance different from the first utterance (e.g. paragraph 0040, voice command data structure identifying voice enabled applications and supported commands along with associated voice command variations and examples; paragraph 0052, Fig. 3, assistant determining which tasks the particular application is capable of performing, including example voice commands along with voice examples illustrating how the user can use a given voice command as shown at 340; controlling which voice commands are displayed, which variations etc.; paragraph 0067, at 630, available tasks capable of being performed by voice-enabled application are identified; paragraph 0067, associated voice command variations and voice command examples; i.e. determining which voice commands and variations are available for the particular application (as provided with the request as contextual information) and how the commands/variations should be presented in textual form on a user interface; where determined voice commands/variations are those which apply to the second/different application and are therefore different/other than the received utterance corresponding to a command for the first application); and
providing another textual representation of the another utterance different from the first utterance (e.g. paragraph 0052, Fig. 3, third graphical user interface displayed listing tasks at 340 including example voice commands/variations; paragraph 0068, at 640, response provided to the user via GUI identifying available tasks, such as displayed list of tasks along with associated voice command variations and voice command examples; where determined and displayed voice commands/variations are those which apply to the second/different application and are therefore different/other than the received utterance corresponding to a command for the first application).
With respect to claims 20, 34, and 46, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed, and Kannan further teaches wherein the one or more programs further include instructions for: detecting a user input selecting the first utterance; and in response to detecting the user input selecting the first utterance performing the first task (e.g. paragraph 0056, Fig. 4, as depicted at 440, user provides voice input for voice command; in response the assistant determines the voice command, compares to available commands, and then presents results of the command to the user as shown at 445).
With respect to claims 21, 35, and 47, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 20, 34, and 46 as previously discussed, and Kannan further teaches wherein the one or more programs further include instructions for: after performing the first task, displaying a result of the first task on the display of the electronic device (e.g. paragraph 0056, Fig. 4, as depicted at 440, user provides voice input for voice command; in response the assistant determines the voice command, compares to available commands, and then presents results of the command to the user as shown at 445).
Claims 1, 2-5, 9, 10, 16, 17, 22-27, 32, 36-39, and 44 are rejected under 35 U.S.C. 103 as being unpatentable over Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan, further in view of Novitchenko et al. (US 20200380973 A1).
With respect to claims 1, 22, and 23, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches the limitations of the claims as previously discussed. Assuming arguendo that Kannan does not explicitly disclose that the textual representation of the utterance is provided in an affordance displayed over a user interface on a screen of the electronic device, Novitchenko teaches that the textual representation of the utterance is provided in an affordance displayed over a user interface on a screen of the electronic device (e.g. paragraph 0287, Fig. 9A, suggestion affordance 912 overlaying the interface).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to display the textual representations of the utterances in an affordance which is displayed over a user interface (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claims 2, 24, and 36 Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed. Kannan does not explicitly disclose wherein determining, based on the context included in the request, the first textual representation of the first utterance for performing the first task utilizing the first application associated with the first object included in the content displayed on the display of the electronic device further comprises: selecting the first utterance based on a usage history associated with a user providing the user input and the digital assistant.
However, Novitchenko teaches wherein determining, based on the context included in the request, the first textual representation of the first utterance for performing the first task utilizing the first application associated with the first object included in the content displayed on the display of the electronic device further comprises: selecting the first utterance based on a usage history associated with a user providing the user input and the digital assistant (e.g. paragraph 0243, determining based on context data a task that may be performed by a digital assistant; determining whether suggestion criteria associated with the determined task are satisfied; if suggestion criteria satisfied, device providing suggestion indicating that the determined task may be performed using the digital assistant; paragraph 0248, context data including previous state of the device, such as user inputs that the device previously received, previous tasks performed on the device, whether the tasks previously performed were performed using a digital assistant of the electronic device, suggestions previously provided, user inputs previously received in response to suggestions, etc.; paragraph 0254, determining based on the context data a task that may be performed by digital assistant in response to natural language expression; paragraph 0269, determining whether suggestion criteria satisfied based on context data associated with user of the electronic device; personalizing suggestions based on users current and past usage of the electronic device; paragraph 0272, providing suggestion indicating that determined task may be performed using the digital assistant; paragraph 0274, suggestion includes indication as to how user can use the digital assistant to perform the digital task, such as a natural-language expression including trigger that, when provided to the device, causes the electronic device to initiate dialog with the digital assistant; i.e. where selection of the suggested task includes selection of the expression associated with the suggested task, and this selection is based on user’s usage history).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to select the task and associated utterance for display to the user based on usage history associated with the user and the digital assistant (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claims 3, 25, and 37, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan, further in view of Novitchenko teaches all of the limitations of claim 2, 24, and 36 as previously discussed, and Novitchenko further teaches wherein the first utterance is selected based on a number of times the first utterance is included in the usage history associated with the user and the digital assistant (e.g. paragraph 0249-0251, context data including number of time the user has previously performed particular task using the digital assistant; number of times the device has provided suggestion indicating particular task may be performed by digital assistant to the user; user inputs in previously received in response to suggestion (including via speech input/utterance); paragraph 0264, suggestion criteria include requirement that the determined task has not been performed on the electronic device using digital assistant; task never performed using digital assistant; i.e. where the task is performed using the digital assistant in response to a corresponding speech/utterance input by the user, the context data indicating a number of times the task has been invoked via the assistant by the user includes/indicates a number of times which the corresponding speech/utterance has been provided in the usage history of the device, and this number of task invocations/utterances is utilized as context data for selecting the task as a suggestion, and as a suggestion criteria for determining whether to present the task suggestion to the user).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to select the task and associated utterance for display to the user based on usage history, including context/suggestion criteria information indicating a number of times the task was invoked by the user using the associated speech/utterance, associated with the user and the digital assistant (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claim 4, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan, further in view of Novitchenko teaches all of the limitations of claim 3 as previously discussed, and Novitchenko further teaches wherein the first utterance is selected when the number of times the first utterance is included in the usage history is zero (e.g. paragraph 0249-0251, context data including number of time the user has previously performed particular task using the digital assistant; number of times the device has provided suggestion indicating particular task may be performed by digital assistant to the user; user inputs in previously received in response to suggestion (including via speech input/utterance); paragraph 0264, suggestion criteria include requirement that the determined task has not been performed on the electronic device using digital assistant; task never performed using digital assistant; i.e. where the task is performed using the digital assistant in response to a corresponding speech/utterance input by the user, the context data indicating a number of times the task has been invoked via the assistant by the user includes/indicates a number of times which the corresponding speech/utterance has been provided in the usage history of the device, and this number of task invocations/utterances is utilized as context data for selecting the task as a suggestion, and as a suggestion criteria for determining whether to present the task suggestion to the user; it is noted that context/suggestion criteria indicating that the task has never been invoked (and the corresponding utterance therefore has never been used) indicates that the utterance has been included in the usage history zero times).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to select the task and associated utterance for display to the user based on usage history, including context/suggestion criteria information indicating a number of times the task was invoked by the user using the associated speech/utterance, including never/zero times, associated with the user and the digital assistant (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claims 5, 26, and 38, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed. Kannan does not explicitly disclose wherein determining, based on the context referenced by the request, a textual representation of an utterance for performing a task utilizing the content displayed on the display of the display device further comprises: determining content displayed on the display of the electronic device; and determining the textual representation of the utterance for performing the task based on the content displayed on the screen of the electronic device.
However, Novitchenko teaches wherein determining, based on the context referenced by the request, a textual representation of an utterance for performing a task utilizing the content displayed on the display of the display device further comprises: determining content displayed on the display of the electronic device; and determining the textual representation of the utterance for performing the task based on the content displayed on the screen of the electronic device (e.g. paragraph 0243, determining based on context data a task that may be performed by a digital assistant; determining whether suggestion criteria associated with the determined task are satisfied; if suggestion criteria satisfied, device providing suggestion indicating that the determined task may be performed using the digital assistant; paragraph 0245, context data indicating application currently open on the device; paragraph 0254, determining task based on context data; context data indicating that a clock application is open on the electronic device, determining a task based on the clock application; paragraph 0258, determining search screen interface is open on the electronic device and receives text input “clock” in search field; determining user is attempting to search for and open clock application; determining task of setting an alarm or timer in the clock application; paragraph 0272, providing suggestion indicating that determined task may be performed using the digital assistant; paragraph 0274, suggestion includes indication as to how user can use the digital assistant to perform the digital task, such as a natural-language expression including trigger that, when provided to the device, causes the electronic device to initiate dialog with the digital assistant; i.e. where the determined context data includes content displayed on the screen such as of a currently opened application or currently entered search input text, and the task is determined based on this context data, including an associated natural language expression/utterance, and this information is displayed to the user as a suggestion).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to select the task and associated utterance for display to the user based on content currently displayed on the screen (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
As previously discussed, Trufinescu teaches that the content is content that is referenced by the request (e.g. paragraph 0024, Fig. 3, user clicking on graphical selector for MOVIE1, and session state of application user interface changed to present specific content for MOVIE1; paragraph 0030, Fig. 3, query 48 from user includes word or phrase and the bot client program is unable to resolve the meaning of the word or phrase within threshold confidence level, and in response, sending context request to application program; example query 48A includes the phrase “When was this movie made?”; paragraph 0032, determining whether query is directed to content related to the session state of the application program; paragraph 0033, sending context request to application program; application program sending current context data, which may include current content 38 presented via the application user interface of the application program; paragraph 0034, Fig. 3, query is “When was this movie made?” and context data includes content for the Movie 1 that is currently being presented by the application user interface).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Novitchenko, Gada, Krishnan, and Trufinescu in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization), to incorporate the teachings of Trufinescu (directed to application context aware chatbots) to include the capability to include, in the context, content displayed on a display of the electronic device that is referenced by the request (as taught by Trufinescu). One of ordinary skill would have been motivated to perform such a modification in order to enable contextual information about application programs to be communicated to and leveraged by chatbots to process user queries with greater effectiveness and the functionality of the application program and chatbot are extended as described in Trufinescu (paragraph 0010).
As previously discussed, Gada teaches that the textual representation of the utterance for performing the task is determined by utilizing content displayed on the display of the electronic device (e.g. paragraph 0291-0292, describing Fig. 8C-D, receiving voice input of “Show me what to say” and in response displaying UI of Fig. 8D, in which suggestion interface 802B is displayed including visual cues/suggestions each of which corresponds to a valid voice command that is available while the device is in the context shown in Fig. 8D; visual cue 802B1 “Hide Grid” is displayed because current device context includes a displayed grid; i.e. the suggested utterance/voice command is determined and displayed to the user based on/utilizing the content currently displayed on the display of the device, such as providing a “Hide grid” command/utterance as a suggestion based on the device context including the device currently displaying the grid as content).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Novitchenko, Krishnan, and Gada in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Gada (directed to voice-based control of electronic devices) to include the capability to determine and provide the textual representation of the utterance/command for performing a suggested task by utilizing the content currently displayed on the display of the device (as taught by Gada). One of ordinary skill would have been motivated to perform such a modification in order to provide faster more efficient methods and interfaces for voice-based control of electronic devices, reducing cognitive burden on the user and producing a more efficient human-machine interface as described in Gada (paragraph 0005).
With respect to claim 9, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claim 1 as previously discussed. Kannan does not explicitly disclose wherein the second textual representation of the second utterance for performing the second task is further determined based on other contextual data. However, Novitchenko teaches wherein the second textual representation of the second utterance for performing the second task is further determined based on other contextual data (e.g. paragraph 0245-0253, context data including context data based on current state of device and context data based on previous state of device; context data indicating software application currently open; context data indicating current value or setting of electronic device such as wireless communication protocol/setting, device mode of the device, etc.; context data including physical state of the device such as current speed, acceleration, directional movement, location, orientation, temperature, or signal strength of the device; context data indicating previous user inputs, previous tasks performed, whether tasks performed with assistant, previous suggestions, previous responses to suggestions; user specific context data; context data indicating previous tasks, numbers of times performed, whether digital assistant used, etc.; number of times suggestion of task provided, responses, user inputs, etc. in response to suggestions, etc.; context data including frequency of user behaviors, context data associated with additional devices associated with the user; i.e. the device may provide multiple different suggested tasks (including corresponding utterances), and the tasks (and corresponding utterances) may be determined based on various different contextual data).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to provide the first and second tasks based on various different contextual data (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claim 10, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan, further in view of Novitchenko teaches all of the limitations of claim 9 as previously discussed, and Novitchenko further teaches wherein the other contextual data includes data recently accessed on the electronic device (e.g. paragraph 0207, contextual information including software and hardware states of device at the time user request is received; paragraph 0245, currently open application; paragraphs 0249-0250, context data indicating number of times user has performed task within time period/amount of time elapsed since user lasted used assistant to perform the task, such as within the previous week, month, etc.; paragraph 0253, context data indicating previous interactions with digital assistant of additional devices within predetermined period such as a week, month, etc.; paragraph 0254, context data indicating clock application currently open; paragraph 0258, context data indicating search screen interface is open, text input “clock” is in search field; i.e. where the user’s interaction with the device, such as via an application or digital assistant to perform a task, indicates data accessed on the device (i.e. such as the application, assistant, or other data associated with the task) within a particular time period, including a time period which may be considered “recent” (such as within the past week, month, etc.)).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to utilize, as context data, indications of data (associated with/including a program, task, assistant function, etc.) recently (within past week, month, etc.) accessed on the device (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claims 27 and 39, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 22 and 23 as previously discussed, and Novitchenko further teaches wherein the second textual representation of the second utterance for performing the second task is further determined based on data recently accessed on the electronic device (e.g. paragraph 0207, contextual information including software and hardware states of device at the time user request is received; paragraph 0245, currently open application; paragraphs 0249-0250, context data indicating number of times user has performed task within time period/amount of time elapsed since user lasted used assistant to perform the task, such as within the previous week, month, etc.; paragraph 0253, context data indicating previous interactions with digital assistant of additional devices within predetermined period such as a week, month, etc.; paragraph 0254, context data indicating clock application currently open; paragraph 0258, context data indicating search screen interface is open, text input “clock” is in search field; i.e. where the user’s interaction with the device, such as via an application or digital assistant to perform a task, indicates data accessed on the device (i.e. such as the application, assistant, or other data associated with the task) within a particular time period, including a time period which may be considered “recent” (such as within the past week, month, etc.)).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability to utilize, as context data, indications of data (associated with/including a program, task, assistant function, etc.) recently (within past week, month, etc.) accessed on the device (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claims 16, 32, and 44, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed, and Krishnan further teaches where the first textual representation of the first utterance and the second textual representation of the second utterance belong to a first plurality of textual representations (e.g. as shown in Fig. 1B, suggestion elements 132 and 134, displayed at a first time, appear to constitute first and second textual representations belonging to a second plurality of textual representations; Examiner notes that at a subsequent time, additional/other textual representations may be displayed in addition to, or instead of, suggestions elements 132 and 134, such as element 142 in Fig. 1C, element 216 in area 222 of Fig. 2A, etc.).
Kannan does not explicitly disclose wherein the one or more programs further include instructions for: detecting selection of a user interface object; and in response to detecting selection of the user interface object, displaying a second plurality of textual representations of utterances in the affordance. However, Novitchenko teaches wherein the one or more programs further include instructions for: detecting selection of a user interface object; and in response to detecting selection of the user interface object, displaying a second plurality of textual representations of utterances in the affordance (e.g. paragraph 0289, Fig. 9A, suggestion affordance 912 displayed in user interface is a selectable affordance; selecting suggestion affordance 912 may cause suggestion/tips software application to be opened to display an interface in which suggestions that have been previously provided by the device can be collected, categorized, and displayed).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability for the user to select an affordance (including an affordance in which the suggestion/utterance is presented) and, in response, display a plurality of previously provided suggestions (where these suggestions include corresponding textual representations of utterances) (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
With respect to claim 17, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan, further in view of Novitchenko teaches all of the limitations of claim 16 as previously discussed, and Novitchenko further teaches wherein the second plurality of textual representations of utterances are unrelated to the content displayed on the display of the electronic device (e.g. paragraph 0289, Fig. 9A, suggestion affordance 912 displayed in user interface is a selectable affordance; selecting suggestion affordance 912 may cause suggestion/tips software application to be opened to display an interface in which suggestions that have been previously provided by the device can be collected, categorized, and displayed; i.e. where at least some of the displayed previously provided suggestions/tips (including their associated utterances) may be related to various different applications (as shown in at least Figs. 9A-D), and therefore unrelated to at least some other content displayed on the screen).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Novitchenko in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Novitchenko (directed to voice assistant discoverability through on-device targeting and personalization) to include the capability for the user to select an affordance (including an affordance in which the suggestion/utterance is presented) and, in response, display a plurality of previously provided suggestions (where these suggestions include corresponding textual representations of utterances), where these suggestions/utterances may be related to various different applications and therefore may be unrelated to at least some content which is currently displayed on the screen (such as content of a current application) (as taught by Novitchenko). One of ordinary skill would have been motivated to perform such a modification in order to improve an electronic device’s ability to provide a suggestion indicating a task that may be performed by a digital assistant of the electronic device, and to increase the relevancy and usefulness of the suggestion and thus increase the likelihood that the user will engage with an learn from the suggestion, making the device more efficient, reducing power usage and improving battery life by enabling the user to user the device to perform tasks more quickly and efficiently as described in Novitchenko (paragraph 0010).
Claims 15, 31, and 43 is rejected under 35 U.S.C. 103 as being unpatentable over Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan, further in view of Gray (US 9990176 B1).
With respect to claim 15, Kannan in view of Trufinescu, further in view of Gada, further in view of Krishnan teaches all of the limitations of claims 1, 22, and 23 as previously discussed. Kannan does not explicitly disclose wherein an order of the first textual representation of the first utterance and the second textual representation of the second utterance is based on a number of times the first textual representation of the first utterance and the second textual representation of the second utterance were previously received by the digital assistant. However, Gray teaches wherein an order of the first textual representation of the first utterance and the second textual representation of the second utterance is based on a number of times the first textual representation of the first utterance and the second textual representation of the second utterance were previously received by the digital assistant (e.g. col. 36 lines 49-64, Fig. 5, frequent utterances module storing data corresponding to each utterance and arranging by which utterances were made most often; i.e. as shown in Fig. 5, the set of utterances may be ordered (such as in a numerical list) by frequency/number of times the utterance is received).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Kannan, Trufinescu, Gada, Krishnan, and Gray in front of him to have modified the teachings of Kannan (directed to discovering capabilities of third-party voice-enabled resources), Gada (directed to voice-based control of electronic devices), Krishnan (directed to automated assistant suggestions for third-party vehicle computing devices), and Trufinescu (directed to application context aware chatbots), to incorporate the teachings of Gray (directed to latency reduction for content playback, such as by an assistant in response to an utterance) to include the capability to order the textual representations of the utterances based on number of times they are historically received by the assistant (as taught by Gray). One of ordinary skill would have been motivated to perform such a modification in order to reduce amount of bandwidth consumed and provide responses to spoken utterances in a substantially instantaneous manner as described in Gray (abstract).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain,” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting in re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (GCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co, v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F,3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir, 2005): Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMY L STANLEY whose telephone number is (469)295-9105. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM CST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar, can be reached at telephone number (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/JEREMY L STANLEY/
Primary Examiner, Art Unit 2127
Prosecution Insights