Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of Korean Patent Application No. Korean Patent Application Nos. 10-2024-0019974 and 10-2024-0077175, filed in the Republic of Korea on February 8, 2024, and June 13, 2024. Claims 1-15 have been afforded the benefit of the February 8, 2024 filing date.
Information Disclosure Statement
The IDS dated May 8, 2026 has been considered and placed in the application file.
Response to Arguments
Applicant’s arguments, see Rejection Section 35 USC § 103, filed August 5, 2026, with respect to the rejection(s) of claim(s) 1-4, 6, 8, 9, 11-13, 15 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of newly found prior art.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims [1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 15] are rejected under 35 U.S.C. 103 as being unpatentable over Oktem (US 11,238,848 B2) in view of Chen (US 12,008,091 B2) and in further view Wang (WO2017059679A1).
Regarding claim 1, Oktem teaches an electronic device comprising:
a display;
[Column 24, lines 21-26 "The processor 802 can process instructions for execution within the computing device 800, including instructions stored in the memory 804 or on the storage device 806 to display graphical information for a graphical user interface (GUI) on an external input/output device, such as a display 816 coupled to the high-speed interface 808"].
a network interface configured to communicate with a server
[Column 28, lines 42-47 "The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network"]; … [The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network"].
a user input interface configured to receive speech uttered by a user
[Column 1, lines 46-47 "A speech-enabled device may be used by multiple different users"].
and a controller configured to: determine whether the speech uttered by the user is related to a user account logged in to the server
[Column 7, lines 14- 20 "To handle utterances from multiple users, the speech enabled device 125 may classify utterances as spoken by particular known users, and provide queries to the server that indicate the particular known users classified as speaking the utterances. A known user may be a user that is registered as a user of the system 100 and guest user may be a user that is not registered as a user of the system 100" where the known user(registered) based on speech generally refers to having a user account and the controller is part of the speech enabled device]; [Column 8, lines 6-8 "The authentication token of a user may be a data structure that serves as a log-in for an account of the user in the system 100"].
and in response to determining the speech uttered by the user is related to the user account logged in to the server, display a second response on the display indicating the user who uttered the speech is currently logged in to the server.
[Column 14, lines 51-60 "If the process 500 determines that the query includes a known user indication, then the process 500 logs in or performs another credential action using the authentication token for the indicated known user (516), retrieves data (518), and serves to a voice-enabled device (520). For example, the server 155 logs into an account of "Dad" using the authentication token for "Dad," retrieves a calendar of "Dad," and provides a response to the speech-enabled device 125 with a summary of appointments for "Dad" based on retrieved calendar"]; [Column 8, lines 6-8 " The authentication token of a user may be a data structure that serves as a log-in for an account of the user in the system 100"].
wherein the controller is further configured to: log in to a user account related to the user who has uttered the speech.
[Column 14, lines 51-60 "If the process 500 determines that the query includes a known user indication, then the process 500 logs in or performs another credential action using the authentication token for the indicated known user (516), retrieves data (518), and serves to a voice-enabled device (520). For example, the server 155 logs into an account of "Dad" using the authentication token for "Dad," retrieves a calendar of "Dad," and provides a response to the speech-enabled device 125 with a summary of appointments for "Dad" based on retrieved calendar"]; [Column 8, lines 6-8 " The authentication token of a user may be a data structure that serves as a log-in for an account of the user in the system 100"].
However, Oktem does not teach in response to determining the speech uttered by the user is unrelated to the user account logged in to the server, display a first response on the display including an indicator indicating the user who has uttered the speech is not the user logged in to the user account, wherein the controller is further configured to: log out of the user account logged in to the server based on the speech uttered by the user being unrelated to the user account logged in to the server,
But Chen teaches in response to determining the speech uttered by the user is unrelated to the user account logged in to the server, display a first response on the display including an indicator indicating the user who has uttered the speech is not the user logged in to the user account
[Column 5, lines 27-32 "Additionally, or alternatively, in response to the voice verification module 230 determining that the user-ID does not exist or the digital audio data does not match the voiceprint stored in the profile corresponding to the user-ID, 30 a rejection indication is transmitted to the device 110 in operation 428. Thus, the user is not authenticated" where the rejection indication is displayed to the device 110"]; [Column 5, lines 62-65 "In some implementations, the functions of the STT module 210, the NLU module 220, and voice verification module 230 are integrated within the device 110, e.g., processor 112 of user interface device 110 of FIG. 1"]; [Column 5, lines 44-48 "In operation 430, the device 110 outputs an indication that NLU module 220 may determine the intent is enrolling the voiceprint is not verified, or authenticated. For example, the device 110 may output an audio signal via a speaker, or a visual signal via a display, indicating that the user's verification failed, and requests that the user try again"];[Column 4, lines 67, column 5, lines 1-9 "For example, the voice verification module 230 compares the received user-ID data to a plurality identification codes corresponding to the plurality of user profiles stored in the database. In response to finding an identification code in the database that matches the received user-ID, the voice verification module 230 determines the user-ID exists. In response to not finding an identification code that matches the received user-ID, the voice verification module 230 determines the user-ID does not exist, or is not present in the database" where the user profiles are the user accounts.].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem with the teaching of Chen because generating different responses when voice identification is incorporated, as taught by varying responses based on user recognition, would improve system robustness and user experience.
However, Oktem in view of Chen do not teach wherein the controller is further configured to: log out of the user account logged in to the server based on the speech uttered by the user being unrelated to the user account logged in to the server,
But Wang teaches log out of the user account logged in to the server based on the speech uttered by the user being unrelated to the user account logged in to the server [Page 2, 4th paragraph, Summary of Invention “In this embodiment, the voiceprint matching is performed according to a certain frequency, and the current account is logged out when the voiceprint information does not match the preset voiceprint information, thereby improving the security of the account.“ [Page 3, Paragraph 4 ”When the first voiceprint information and the pre-stored preset voiceprint information associated with the at least two accounts do not match, log out of the at least two accounts.”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem in view of Chen in view Wang’s voice print based logout technique to log out of the current account when the received speech does not match that account’s associated voiceprints, and then use Oktem’s authentication token to log into the identified speaker’s account. This would protect the security of the user.
Regarding claim 2, Oktem does teach the electronic device of claim 1, further comprising: a memory configured to store a user list including at least one piece of user identification information related to a user account with a history of logging in to the server [Column 2, lines 47-55 "In some aspects, the subject matter described in this specification may be embodied in methods that may include the actions of storing authentication tokens corresponding to known users of a device, receiving an utterance from a speaker, classifying the utterance as spoken by a particular known user of the known users, and providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user" where storing authentication tokens are the user list in the memory (storing requires memory component) with the "known users of a device" (history of logged in users is kept) and the particular known user is the ID of the user].
wherein the controller is further configured to: transmit the user list to the server along with data including a speech signal related to the speech uttered by the user
[Column 2, lines 8- 14 "Simultaneously, the system is configured to permit processing of queries by parties not otherwise recognized (e.g., a guest user) by providing to the server either simply the utterance, the utterance and the identification ("id") of one or more possible speakers but without confirmation that a speaker has been identified, or an utterance and some other indication such as a device Id" where the ID of one or more speakers is in the user list sent to the server] ;
[Column 3, lines 47- 53 "In certain aspects, providing a query that includes a representation of the utterance and that does not indicate that the utterance was classified as spoken by a particular known user of the known users includes providing a query that includes the authentication tokens and no flag that indicates the utterance was classified as spoken by a particular known user of the known users" where the query has both authentication tokens (user list) and representation of the utterance( data including speech)] ;[Column 5, lines 58- 67, column 6, lines 1-5 "In another example, by having queries that include multiple or all authorization tokens for users stored on the speech-enabled device and indicating which authorization token corresponds to the speaker, or whether no authorization token corresponds to the speaker, the system may enable services to be shared between users of the shared speech-enabled device. For instance, if a server that processes queries receives a query with a pool of tokens that includes an authentication token for a particular user that is permitted to use a music streaming service, even if the query does not indicate that the particular user spoke the utterance, the server may still permit the music streaming service to be used in response to the query given the presence of the authentication token for the particular user" where the multiple authorization tokens is the user list sent to the server in the query].
and determine the speech uttered by the user is related to the user account logged in to the server based on a result of processing the speech signal received from the server - [Column 2, lines 56-62 "In some aspects, providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user includes providing a query that includes the authentication tokens and a flag that indicates the utterance was classified as spoken by the particular known user"].
Regarding claim 3, Oktem does teach the electronic device of claim 2, wherein the result of processing the speech signal includes first user identification information related to the speech uttered by the user, and wherein the controller is further configured to determine the speech uttered by the user is related to the user account logged in to the server by comparing the first user identification information with second user identification information related to the user account logged in to the server.
[Column 1, lines 57-67, Column 2, lines 1-2 "To handle utterances from multiple different users, a speech-enabled device may attempt to identify a speaker of an utterance as a particular user, and if successful, provide a query to a server with an indication corresponding to the identified particular user, and if not successful, provide a query to a server without an indication corresponding to the identified particular user. For example, the speech-enabled device may recognize an utterance as spoken by "John Doe" based on recognizing the utterance as matching a preestablished speech pattern from "John Doe," and, in response, provide a query to the server that includes both an audio representation of the utterance and an indication that the utterance was spoken by "John Doe." where comparing the first user id information (voice print) with the second (pre-established speech pattern) is happening in the example shown].
However, Oktem doesn’t teach processing the speech signal includes first user identification information related to speech and then comparing the first user identification information with second user identification information related to the user account logged in to the server.
But Chen teaches processing the speech signal includes first user identification information related to speech - [Column 12, lines 23-30 "In one form of the method, the method further includes extracting the user identification code from the text derived from the voice audio, wherein comparing includes comparing the user identification code against the database, and wherein determining whether the user profile exists includes attempting to retrieve from the database the user profile that matches the user identification code stored in the database"].
comparing the first user identification information with second user identification information related to the user account logged in to the server
[Column 4, lines 59- 7, column 5, lines 1-6 "In operation 422, the user-ID and digital audio data are transmitted to the voice verification module 230. The voice verification module 230 determines whether a user profile corresponding to the user-ID extracted from the text exists, or is present, among a plurality of user profiles stored in a database ( e.g., database 126). Each of the user profiles stored in the database includes an identification code and one or more voiceprints of a corresponding user for voice verification. For example, the voice verification module 230 compares the received user-ID data to a plurality identification codes corresponding to the plurality of user profiles stored in the database. In response to finding an identification code in the database that matches the received user-ID, the voice verification module 230 determines the user-ID exists" where the second ID is one of the plurality of identification ids stored in the database and the user profiles are the user accounts in the database]; [Column 5, lines 14 -21 "In response to the captured voice audio substantially matching the stored voiceprint corresponding to the user-ID, the voice verification module 230 transmits an acceptance indication to the device 110 in operation 424. In response to the acceptance indication, the device 110 provides the user access to a resource. In operation 426, the device 110 outputs an indication that the voiceprint is verified and that the user is authenticated" where the authentication is part of the user account].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem with the teaching of Chen because using extracted user id from speech and comparing it with stored user information would provide reliable user authentication and would improve system security.
Regarding claim 4, Chen teaches the electronic device of claim 3, wherein the controller is further configured to: compare the first user identification information with the second user identification information based on the electronic device being preset to use voice-related identification information with respect to the first user identification information, and
[Column 4, lines 59- 7, column 5, lines 1-6 "In operation 422, the user-ID and digital audio data are transmitted to the voice verification module 230. The voice verification module 230 determines whether a user profile corresponding to the user-ID extracted from the text exists, or is present, among a plurality of user profiles stored in a database ( e.g., database 126). Each of the user profiles stored in the database includes an identification code and one or more voiceprints of a corresponding user for voice verification. For example, the voice verification module 230 compares the received user-ID data to a plurality identification codes corresponding to the plurality of user profiles stored in the database. In response to finding an identification code in the database that matches the received user-ID, the voice verification module 230 determines the user-ID Exists" where the second ID is one of the plurality of identification ids stored in the db.]; [Column 4, lines 10-12 "The voice verification module 230 is configured to compare digital audio data derived from a captured audio signal to pre-recorded voiceprints of one or more users" where pre-recorded is the preset voiceprints].
generate the second response based on the electronic device being not preset to use the voice-related identification information with respect to the first user identification information
[Column 5, lines 26-36 "Additionally, or alternatively, in response to the voice verification module 230 determining that the user-ID does not exist or the digital audio data does not match the voiceprint stored in the profile corresponding to the user-ID, 30 a rejection indication is transmitted to the device 110 in operation 428. Thus, the user is not authenticated. For example, the voice verification module 230 may determine that the user-ID does not exist and may transmit a rejection indication without further analyzing the digital audio data against the plurality of voiceprints stored in the database" where the rejection indication can be the second response without further analyzing the audio data (no preset used in voice-relation id information)].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem with the teaching of Chen because using extracted user id from speech and comparing it with stored user information would provide reliable user authentication and would improve system security.
Regarding claim 5, Oktem in view of Chen in view of Wang teaches the electronic device of claim 1, wherein the indicator corresponds to indicating that the user account related to the user who has uttered the speech is currently logged in.
Oktem teaches
[Column 14, lines 51-60 "If the process 500 determines that the query includes a known user indication, then the process 500 logs in or performs another credential action using the authentication token for the indicated known user (516), retrieves data (518), and serves to a voice-enabled device (520). For example, the server 155 logs into an account of "Dad" using the authentication token for "Dad," retrieves a calendar of "Dad," and provides a response to the speech-enabled device 125 with a summary of appointments for "Dad" based on retrieved calendar"]; [Column 8, lines 6-8 " The authentication token of a user may be a data structure that serves as a log-in for an account of the user in the system 100"].
However, Oktem does not teach indicator corresponds to indicating that the user has uttered the speech.
But Chen teaches indicator corresponds to indicating that the user has uttered the speech.
[Column 5, lines 10- 26 "In response to determining the user profile corresponding to the user-ID exists, the voice verification module 230 obtains the user profile containing the stored voiceprint corresponding to the user-ID and compares the digital audio data of the captured voice audio to the stored voiceprint. In response to the captured voice audio substantially matching the stored voiceprint corresponding to the user-ID, the voice verification module 230 transmits an acceptance indication to the device 110 in operation In response to the acceptance indication, the device 110 provides the user access to a resource. In operation 426, the device 110 outputs an indication that the voiceprint is verified and that the user is authenticated. For example, the device 110 may output an audible signal via a speaker, and/or visual signal via a display, indicating that the user's identity is verified. In some implementations, the device may indicate that access has been granted" where acceptance indication can be a second response indicating the user is currently verified and has access to the resources and is authenticated].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem with Chen to display that the identified speaker’s account is currently logged in, thereby confirming successful login and informing the user which account is active.
Regarding claim 6, Oktem teaches the electronic device of claim 1, wherein the controller is further configured to: determine whether the speech uttered by the user requires identification of the user based on a result of performing natural language processing on the speech uttered by the user
[Column 1, lines 57- 59 "…A speech-enabled device may attempt to identify a speaker of an utterance as a particular user…"]; [Column 11, lines 64 -67 " The process 200 includes receiving an utterance from a speaker (210). For example, the speech-enabled device 125 may receive an utterance of "OK Computer, what is on my calendar" from a known user "Dad" through a microphone of the speech-enabled device 125. The process 200 includes classifying the utterance as spoken by a particular known user of the known users (215). For example, the speech-enabled device 125 may classify an utterance as spoken by the known user "Dad" where receiving sound waves from user and identifying the speaker is where Natural language processing begins. ]; [Column 1, lines 16-17 "This specification generally relates to natural language processing"].
determine whether the speech uttered by the user is related to the user account logged in to the server based on the speech uttered by the user requiring the identification of the user
[Column 10, lines 13-23 "The server 155 may receive a query from the speech enabled device 125 and provide a response to the query to the speech-enabled device 125. For example, the server 155 may receive a query "OK Computer, what is on my calendar" and in response, provide the response, "Dad, you have one meeting today at 1 PM with John." The server 155 may provide a response to a query based on determining whether the query includes an indication of a known user and whether the query includes a question related to personal information, as shown in FIGS. lA-lC and the accompanying Description"];
[Column 2, lines 56-62 "In some aspects, providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user includes providing a query that includes the authentication tokens and a flag that indicates the utterance was classified as spoken by the particular known user" where the authentication tokens are related to user logged in account].
and generate the second response based on the speech uttered by the user not requiring the identification of the user
[Column 14, lines 1-6 "For example, the server 155 may determine that as no particular known user is the speaker for a query, then the query is from a guest user. The process 400 includes in response to determining that the query is not from a known user, determining whether the query is non-personal (415)"];
[Column 14, lines 23-29 "The process 400 includes in response to determining that the query is non-personal, attempting to provide a response to the query ( 420). For example, in response to determining that the query "What time is it" is non-personal, the server 155 may attempt to determine the current time and provide a response of "It's 11 AM" to the speech-enabled device 125" where a response provided is the second response based on speech not requiring user identification].
Regarding claim 8, Oktem teaches the electronic device of claim 6, wherein the controller is further configured to: determine the speech uttered by the user requires user identification based on the speech uttered by the user being related to a first application linked to the user account - [Column 1, lines 48-52 "Users of the speech-enabled device may ask questions that are related to the user's personal information. For example, John and Jane Doe may separately ask the speech enabled device what's scheduled for their respective day"];
[Column 2, lines 15-20 "A server may use the indication of the particular user, or lack of such an indication, to generate a response to the query. For example, the server may determine, based on receipt of the indication that the utterance was spoken by John Doe, that it has authority to provide appropriate personal information in response to the query.]; [Column 3, lines 39-47 "In some aspects, the subject matter described in this specification may be embodied in methods that may include the actions of storing authentication tokens corresponding to known users of a device, receiving an utterance from a speaker, classifying the utterance as spoken by a particular known user of the known users, and providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user”].
and determine the speech uttered by the user does not require the user identification of the user based on the speech uttered by the user being related to a second application not linked to the user account - [column 3, lines 53-59 " In some implementations, providing a query that includes a representation of the utterance and that does not indicate that the utterance was classified as spoken by a particular known user of the known users includes providing a query that does not include an authentication token of any of the known users"];
[Column 1, lines 53-56 " Users other than John and Jane Doe (i.e., "guests") may also ask the speech-enabled device questions that are not related to the user's personal information. For example, a guest user may ask the speech-enabled device what is the current time"]; [Column 2, lines 28-33 " In yet another example, the server may determine that even though a query corresponding to an utterance ( e.g., "What time is it") does not indicate a particular user, the query is not related to personal information and a reply may be provided (e.g., "It is now 11 AM")to the speech-enabled device" where the reply is related to a second application(time) where authentication is not required].
Regarding claim 9, Oktem teaches the electronic device of claim 6, wherein the controller is further configured to: execute an application linked to the user account
[Column 1, lines 48-52 "Users of the speech-enabled device may ask questions that are related to the user's personal information. For example, John and Jane Doe may separately ask the speech enabled device what's scheduled for their respective day"];
[Column 2, lines 15-24 "A server may use the indication of the particular user, or lack of such an indication, to generate a response to the query. For example, the server may determine, based on receipt of the indication that the utterance was spoken by John Doe, that it has authority to provide appropriate personal information in response to the query. In the example request for what is on the schedule for the day, this means the server may provide to the speech enabled device for vocalization a listing or summary of appointments for John Doe" where application linked to user account based on utterance is executed to provide user the responses he needs]; [Column 3, lines 39-47 "In some aspects, the subject matter described in this specification may be embodied in methods that may include the actions of storing authentication tokens corresponding to known users of a device, receiving an utterance from a speaker, classifying the utterance as spoken by a particular known user of the known users, and providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user”].
determine whether the speech uttered by the user is related to the user account logged in to the server based on the speech uttered by the user being related to a first command requiring the user identification and related to the user account
[Column 2, lines 56-62 "In some aspects, providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user includes providing a query that includes the authentication tokens and a flag that indicates the utterance was classified as spoken by the particular known user" where the authentication tokens are related to user logged in account];
[Column 3, lines 48-55 “In some implementations, the processor 112 of the user interface device 110 may perform the operations of the processor 122 of the server to authenticate the user based on the audio signal. In response to authenticating the user, the user interface device 110 provides the user access to one or more resources (computing, storage, network, applications, etc.) to which the user is entitled to access”]; [Column 1, lines 48-52 "Users of the speech-enabled device may ask questions that are related to the user's personal information. For example, John and Jane Doe may separately ask the speech enabled device what's scheduled for their respective day"]; [Column 2, lines 15-20 "A server may use the indication of the particular user, or lack of such an indication, to generate a response to the query. For example, the server may determine, based on receipt of the indication that the utterance was spoken by John Doe, that it has authority to provide appropriate personal information in response to the query”].
and generate the second response based on the speech uttered by the user being related to a second command different from the first command and requiring the user identification
[Column 7, lines 27-34 "For example, FIG. lA illustrates an utterance "OK Computer, what is on my calendar" spoken by a known user "Dad." The speech-enabled device 125 provides a query to the server 155, where the query includes a representation of the utterance and an indication that the utterance was classified as spoken by the known user "Dad." The server 155 generates a response "Dad, you have one meeting today " where this is the first command]; [Column 8, lines 8-17 "For example, a query that includes the authentication token for known user "Dad" 134 and indicates that "Dad" is classified as a speaker may enable access to information in the account of "Dad." The authentication tokens may indicate actions that may be performed in response to utterances. For example, an account of a known user "Dad" may have authorization to play music from "Provider X," so a query to play music from "Provider X" that includes the authorization token for "Dad" 134 may result in music being played from "Provider X." where play music in the utterance is the second command]; [Column 2, lines 15-20 "A server may use the indication of the particular user, or lack of such an indication, to generate a response to the query. For example, the server may determine, based on receipt of the indication that the utterance was spoken by John Doe, that it has authority to provide appropriate personal information in response to the query" where responses are provides to several commands and both questions require user identification];
[Column 1, lines 57-62 "To handle utterances from multiple different users, a speech-enabled device may attempt to identify a speaker of an utterance as a particular user, and if successful, provide a query to a server with an indication corresponding to the identified particular user, and if not successful, provide a query to a server without an indication corresponding to the identified particular user"].
Regarding claim 11, Oktem teaches a system comprising:
a server
[Column 1, lines 57-61 'A speech-enabled device may attempt to identify a speaker of an utterance as a particular user, and if successful, provide a query to a server with an indication corresponding to the identified particular use,…"].
and an electronic device configured to: receive speech uttered by a user
[Column 1, lines 46-47 "A speech-enabled device may be used by multiple different users”]; [Column 1, lines 49-50 "Users of the speech-enabled device may ask questions that are related to the user's personal information. For example, John and Jane Doe may separately ask the speech enabled device what's scheduled for their respective day"].
transmit data including a speech signal related to the speech uttered by the user to the server
[Column 1, lines 57-63 "To handle utterances from multiple different users, a speech-enabled device may attempt to identify a speaker of an utterance as a particular user, and if successful, provide a query to a server with an indication corresponding to the identified particular user, and if not successful, provide a query to a server without an indication corresponding to the identified particular user”] ; [Column 1, lines 63-67, column 2, lines 1-2 "For example, the speech-enabled device may recognize an utterance as spoken by "John Doe" based on recognizing the utterance as matching a preestablished speech pattern from "John Doe," and, in response, provide a query to the server that includes both an audio representation of the utterance and an indication that the utterance was spoken by "John Doe"].
receive information from the server indicating whether the speech uttered by the user is related to a user account logged in to the server
[Column 2: lines 15-20 "A server may use the indication of the particular user, or lack of such an indication, to generate a response to the query. For example, the server may determine, based on receipt of the indication that the utterance was spoken by John Doe, that it has authority to provide appropriate personal information in response to the query"].
and in response to the speech uttered by the user is related to the user account logged in to the server, display a second response on the display indicating the user who uttered the speech is currently logged in to the server,
[Column 14, lines 51-60 "If the process 500 determines that the query includes a known user indication, then the process 500 logs in or performs another credential action using the authentication token for the indicated known user (516), retrieves data (518), and serves to a voice-enabled device (520). For example, the server 155 logs into an account of "Dad" using the authentication token for "Dad," retrieves a calendar of "Dad," and provides a response to the speech-enabled device 125 with a summary of appointments for "Dad" based on retrieved calendar"]; [Column 8, lines 6-8 " The authentication token of a user may be a data structure that serves as a log-in for an account of the user in the system 100"].
and transmit a result of processing the speech signal including first user identification information mapped to the first identification information to the electronic device,
wherein the electronic device is further configured to:
[Column 14, lines 51-60 "If the process 500 determines that the query includes a known user indication, then the process 500 logs in or performs another credential action using the authentication token for the indicated known user (516), retrieves data (518), and serves to a voice-enabled device (520). For example, the server 155 logs into an account of "Dad" using the authentication token for "Dad," retrieves a calendar of "Dad," and provides a response to the speech-enabled device 125 with a summary of appointments for "Dad" based on retrieved calendar"]; [Column 8, lines 6-8 " The authentication token of a user may be a data structure that serves as a log-in for an account of the user in the system 100"].
However, Oktem does not teach in response to the speech uttered by the user being unrelated to the user account logged in to the server, display a first response on the display including an indicator indicating the user who has uttered the speech is not the user logged in to the user account,
wherein the server is configured to: generate identification information for the speech signal included in the data transmitted by the electronic device,
determine first identification information related to the identification information for the speech signal among identification information mapped to user identification information related to user accounts stored in a database of the server,
wherein the electronic device is further configured to: log out of the user account logged in to the server based on the speech uttered by the user being unrelated to the user account logged in to the server
But Chen teaches response to the speech uttered by the user being unrelated
to the user account logged in to the server, display a first response on the display including an indicator indicating the user who has uttered the speech is not the user logged in to the user account
[Column 5, lines 27-32 "Additionally, or alternatively, in response to the voice verification module 230 determining that the user-ID does not exist or the digital audio data does not match the voiceprint stored in the profile corresponding to the user-ID, 30 a rejection indication is transmitted to the device 110 in operation 428. Thus, the user is not authenticated" where the rejection indication is displayed to the device 110"];[Column 5, lines 62-65 "In some implementations, the functions of the STT module 210, the NLU module 220, and voice verification module 230 are integrated within the device 110, e.g., processor 112 of user interface device 110 of FIG. 1"];
[Column 5, lines 44-48 "In operation 430, the device 110 outputs an indication that NLU module 220 may determine the intent is enrolling the voiceprint is not verified, or authenticated. For example, the device 110 may output an audio signal via a speaker, or a visual signal via a display, indicating that the user's verification failed, and requests that the user try again"];
[Column 4, lines 67, column 5, lines 1-9 "For example, the voice verification module 230 compares the received user-ID data to a plurality identification codes corresponding to the plurality of user profiles stored in the database. In response to finding an identification code in the database that matches the received user-ID, the voice verification module 230 determines the user-ID exists. In response to not finding an identification code that matches the received user-ID, the voice verification module 230 determines the user-ID does not exist, or is not present in the database" where the user profiles are the user accounts."].
wherein the server is configured to: generate identification information for the speech signal included in the data transmitted by the electronic device
[Column 12, lines 23-30 "In one form of the method, the method further includes extracting the user identification code from the text derived from the voice audio, wherein comparing includes comparing the user identification code against the database, and wherein determining whether the user profile exists includes attempting to retrieve from the database the user profile that matches the user identification code stored in the database"].
determine first identification information related to the identification information for the speech signal among identification information mapped to user identification information related to user accounts stored in a database of the server - [Column 5, lines 27-32 "Additionally, or alternatively, in response to the voice verification module 230 determining that the user-ID does not exist or the digital audio data does not match the voiceprint stored in the profile corresponding to the user-ID, 30 a rejection indication is transmitted to the device 110 in operation 428. Thus, the user is not authenticated" where the rejection indication is displayed to the device 110"]; [Column 5, lines 62-65 "In some implementations, the functions of the STT module 210, the NLU module 220, and voice verification module 230 are integrated within the device 110, e.g., processor 112 of user interface device 110 of FIG. 1"];[Column 5, lines 44-48 "In operation 430, the device 110 outputs an indication that NLU module 220 may determine the intent is enrolling the voiceprint is not verified, or authenticated. For example, the device 110 may output an audio signal via a speaker, or a visual signal via a display, indicating that the user's verification failed, and requests that the user try again"]; [Column 4, lines 67, column 5, lines 1-9 "For example, the voice verification module 230 compares the received user-ID data to a plurality identification codes corresponding to the plurality of user profiles stored in the database. In response to finding an identification code in the database that matches the received user-ID, the voice verification module 230 determines the user-ID
exists. In response to not finding an identification code that matches the received user-ID, the voice verification module 230 determines the user-ID does not exist, or is not present
in the database" where the user profiles are the user accounts."].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem with the teaching of Chen because generating different responses when voice identification are incorporated, as taught by varying responses based on user recognition, would improve system robustness and user experience.
However, Oktem in view of Chen do not teach wherein the controller is further configured to: log out of the user account logged in to the server based on the speech uttered by the user being unrelated to the user account logged in to the server,
But Wang teaches log out of the user account logged in to the server based on the speech uttered by the user being unrelated to the user account logged in to the server [Page 2, 4th paragraph Summary of Invention “In this embodiment, the voiceprint matching is performed according to a certain frequency, and the current account is logged out when the voiceprint information does not match the preset voiceprint information, thereby improving the security of the account.“ [Page 3, Paragraph 4”When the first voiceprint information and the pre-stored preset voiceprint information associated with the at least two accounts do not match, log out of the at least two accounts.”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem in view of Chen in view Wang’s voice print based logout technique to log out of the current account when the received speech does not match that account’s associated voiceprints, and then use Oktem’s authentication token to log into the identified speaker’s account. This would protect the security of the user.
Regarding claim 12, Oktem teaches the system of claim 11, wherein the electronic device is configured to: store a user list including at least one piece of user identification information related to a user account with a history of logging in to the server - [Column 2, lines 47-55 "In some aspects, the subject matter described in this specification may be embodied in methods that may include the actions of storing authentication tokens corresponding to known users of a device, receiving an utterance from a speaker, classifying the utterance as spoken by a particular known user of the known users, and providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user" where storing authentication tokens are the user list in the memory (storing requires memory component) with the "known users of a device" (history of logged in users is kept) and the particular known user is the ID of the user].
and transmit the user list to the server along with the data including the speech signal - [Column 2, lines 8- 14 "Simultaneously, the system is configured to permit processing of queries by parties not otherwise recognized (e.g., a guest user) by providing to the server either simply the utterance, the utterance and the identification ("id") of one or more possible speakers but without confirmation that a speaker has been identified, or an utterance and some other indication such as a device Id" where the ID of one or more speakers is in the user list sent to the server]; [Column 3, lines 47- 53 "In certain aspects, providing a query that includes a representation of the utterance and that does not indicate that the utterance was classified as spoken by a particular known user of the known users includes providing a query that includes the authentication tokens and no flag that indicates the utterance was classified as spoken by a particular known user of the known users" where the query has both authentication tokens (user list) and representation of the utterance( data including speech)]; [Column 5, lines 58- 67, column 6, lines 1-5 "In another example, by having queries that include multiple or all authorization tokens for users stored on the speech-enabled device and indicating which authorization token corresponds to the speaker, or whether no authorization token corresponds to the speaker, the system may enable services to be shared between users of the shared speech-enabled device. For instance, if a server that processes queries receives a query with a pool of tokens that includes an authentication token for a particular user that is permitted to use a music streaming service, even if the query does not indicate that the particular user spoke the utterance, the server may still permit the music streaming service to be used in response to the query given the presence of the authentication token for the particular user" where the multiple authorization tokens is the user list sent to the server in the query].
and wherein the server is configured to: search the database for user identification information related to the user list - [Column 5, lines 58- 67, column 6, lines 1-5 "In another example, by having queries that include multiple or all authorization tokens for users stored on the speech-enabled device and indicating which authorization token corresponds to the speaker, or whether no authorization token corresponds to the speaker, the system may enable services to be shared between users of the shared speech-enabled device. For instance, if a server that processes queries receives a query with a pool of tokens that includes an authentication token for a particular user that is permitted to use a music streaming service, even if the query does not indicate that the particular user spoke the utterance, the server may still permit the music streaming service to be used in response to the query given the presence of the authentication token for the particular user" where the authorization tokens are received from database that is searched].
and determine whether identification information mapped to the user identification information related to the user list is related to the identification information for the speech signal- [Column 2, lines 56-62 "In some aspects, providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user includes providing a query that includes the authentication tokens and a flag that indicates the utterance was classified as spoken by the particular known user"].
Regarding claim 13, Oktem in view of Chen teaches the system of claim 11, wherein the electronic device is further configured to: determine whether the speech uttered by the user is related to the user account logged in to the server by comparing the first user identification information with second user identification information related to the user account logged in to the server.
Claim 13 is rejected for the same reasons as claim 3.
Regarding claim 14, the system of claim 11, wherein the indicator corresponds to indicating that the user account related to the user who has uttered the speech is currently logged in.
Claim 14 is rejected for the same reasons as claim 5.
Regarding claim 15, Oktem does teach the system of claim 11, wherein the server is further configured to: transmit a result of processing the speech signal, including a result of performing natural language processing on the speech signal, to the electronic device
and wherein the electronic device is further configured to: determine whether the speech uttered by the user requires user identification based on the result of performing natural language processing - [Column 1, lines 57- 59 "…A speech-enabled device may attempt to identify a speaker of an utterance as a particular user…"]; [Column 11, lines 64 -67 " The process 200 includes receiving an utterance from a speaker (210). For example, the speech-enabled device 125 may receive an utterance of "OK Computer, what is on my calendar" from a known user "Dad" through a microphone of the speech-enabled device 125. The process 200 includes classifying the utterance as spoken by a particular known user of the known users (215). For example, the speech-enabled device 125 may classify an utterance as spoken by the known user "Dad" where receiving sound waves from user and identifying the speaker is where Natural language processing begins. ].[Column 1, lines 16-17 "This specification generally relates to natural language processing"].
determine whether the speech uttered by the user is related to the user account logged in to the server based on the speech uttered by the user requiring identification of the user - [Column 10, lines 13-23 "The server 155 may receive a query from the speech enabled device 125 and provide a response to the query to the speech-enabled device 125. For example, the server 155 may receive a query "OK Computer, what is on my calendar" and in response, provide the response, "Dad, you have one meeting today at 1 PM with John." The server 155 may provide a response to a query based on determining whether the query includes an indication of a known user and whether the query includes a question related to personal information, as shown in FIGS. lA-lC and the accompanying Description"]; [Column 2, lines 56-62 "In some aspects, providing a query that includes a representation of the utterance and an indication of the particular known user as the speaker using the authentication token of the particular known user includes providing a query that includes the authentication tokens and a flag that indicates the utterance was classified as spoken by the particular known user" where the authentication tokens are related to user logged in account].
and generate the second response based on the speech uttered by the user not requiring identification of the user - [Column 14, lines 1-6 "For example, the server 155 may determine that as no particular known user is the speaker for a query, then the query is from a guest user. The process 400 includes in response to determining that the query is not from a known user, determining whether the query is non-personal (415)"]; [Column 14, lines 23-29 "The process 400 includes in response to determining that the query is non-personal, attempting to provide a response to the query ( 420). For example, in response to determining that the query "What time is it" is non-personal, the server 155 may attempt to determine the current time and provide a response of "It's 11 AM" to the speech-enabled device 125" where a response provided is the second response based on speech not requiring user identification].
But Chen teaches the system of claim 11, wherein the server is further configured to:
transmit a result of processing the speech signal, including a result of performing natural language processing on the speech signal, to the electronic device -[Column 5, lines 27-32 "Additionally, or alternatively, in response to the voice verification module 230 determining that the user-ID does
not exist or the digital audio data does not match the voiceprint stored in the profile corresponding to the user-ID, 30 a rejection indication is transmitted to the device 110 in operation 428. Thus, the user is not authenticated" where the rejection indication is displayed to the device 110" where transmitting the result (indication) is being done]; [Column 3, lines 5565 “FIG. 2 illustrates a voice authentication processor ("YAP") 200. The YAP 200 may be representative of processor 112 of the user interface device 110 and/or processor 122 of the server 120; however, embodiments are not limited thereto. The YAP 200 includes a speech-to-text ("STT") module 210, a natural language understanding ("NLU") module 220, a text-to-speech ("TTS") module 228, a voice verification module 230, and an application manager 240. The NLU module 220 includes a language parser 222, an entity recognizer 224, and an intent classifier 226” where Natural language processing is being performed as part of the whole process].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem with the teaching of Chen because transmitting results of processing helps improve user recognition, ensures secure operation, and clarity of the system.
Claims [7 ] is rejected under 35 U.S.C. 103 as being unpatentable over Oktem (US 11,238,848 B2) in view of Chen (US 12,008,091 B2) and in further view of Wang (WO2017059679A1) and in further view of Edara (US 9111294 B2).
Regarding claim 7, Oktem in view of Chen teaches the electronic device of claim 6, further comprising:
Oktem teaches
[Column 3, lines 48-55 “In some implementations, the processor 112 of the user interface device 110 may perform the operations of the processor 122 of the server to authenticate the user based on the audio signal. In response to authenticating the user, the user interface device 110 provides the user access to one or more resources (computing, storage, network, applications, etc.) to which the user is entitled to access];
[Column 1, lines 48-52 "Users of the speech-enabled device may ask questions that are related to the user's personal information. For example, John and Jane Doe may separately ask the speech enabled device what's scheduled for their respective day"]; [Column 2, lines 15-20 "A server may use the indication of the particular user, or lack of such an indication, to generate a response to the query. For example, the server may determine, based on receipt of the indication that the utterance was spoken by John Doe, that it has authority to provide appropriate personal information in response to the query.]; [Column 10, lines 13- "The server 155 may receive a query from the speech enabled device 125 and provide a response to the query to the speech-enabled device 125. For example, the server 155 may receive a query "OK Computer, what is on my calendar" and in response, provide the response, "Dad, you have one meeting today at 1 PM with John." The server 155 may provide a response to a query based on determining whether the query includes an indication of a known user and whether the query includes a question related to personal information, as shown in FIGS. lA- lC and the accompanying description"].
and determine the speech uttered by the user does not require the user identification based on the speech uttered by the user being unrelated to the command
[Column 3, lines 53-59 " In some implementations, providing a query that includes a representation of the utterance and that does not indicate that the utterance was classified as spoken by a particular known user of the known users includes providing a query that does not include an authentication token of any of the known users"]; [Column 2, lines 28-33 " In yet another example, the server may determine that even though a query corresponding to an utterance ( e.g., "What time is it") does not indicate a particular user, the query is not related to personal information and a reply may be provided (e.g., "It is now 11 AM")to the speech-enabled device" where the speech is irrelevant to a command stored].
However, Oktem in view of Chen and in view of Wang do not teach memory configured to store user histories of users using the electronic device, using the history stored in the memory, using the history stored in the memory.
But Edara teaches memory configured to store user histories of users using the electronic device - [0022 "In at least some embodiments, the keywords that are identified to be associated with a user are stored, at least temporarily, to a database in memory or storage 204 on the computing device"]; [Column 4, lines 3-7 "A computing device 200 could store, such as in memory 204 on the device, a set of positive trigger words (e.g., prefer, enjoy, bought, downloaded, etc.) and/or negative trigger words ( e.g., hate, dislike, or returned) to be used in identifying potential keywords in the audio data"]; [Column 5, lines 46-47 " In at least some embodiments additional data can be stored for identified keywords as well. Identifying information can be stored as well, such as may identify the speaker of the keyword, a person associated with the keyword, people nearby when the keyword was spoken, etc. In at least some embodiments priority information may be attached as well."]; [Column 6, lines 10-13 "In at least some embodiments, the keywords for a user might be transmitted to a content provider or other such party, whereby the provider is able to store the keywords in a data store for subsequent use with respect to the user" where subsequent user of keywords means there is history of the user stored].
using the history stored in the memory
[Column 5, lines 46-47 " In at least some embodiments additional data can be stored for identified keywords as well"];
[Column 3, lines 10-12 "Keywords associated with the user can be used for any appropriate purpose, such as for recommending advertising, product information, or other such content to the user. In this example, a recommendation engine 118 executing on one or more of the application servers 116 of the content provider 112 can receive a request to serve a particular type of content (e.g., advertising) to a user, and can determine keywords associated with that user using information stored in the user and/or keyword data stores 120, 122." where keywords stored could be a commands stored in the data stores];
[Column 2, lines 65-67 " The same or a different application server can compare the keywords in the request to the keywords associated with that user as stored in at least one keyword data store 122…" where keywords (command) from user is compared to data stored(history)];
[Column 9, lines 21-31 "While phone conversations are described in many of the examples herein, it should be understood that there can be various situations in which voice data can be obtained. For example, a user might talk to a friend about purchasing a mountain bike within an audio capture distance of that user's home computer. If the user has authorized the home computer to listen for, and analyze, voice content from the user, the computer can obtain keywords from the conversation and automatically provide recommendations during the conversation, such as by displaying one or more Web sites offering mountain bikes, performing a search for mountain bikes, etc" where user talking about purchasing a bike could include a command about where to find mountain bikes] ;
[Column 7, lines 57- 64 "In this example, the sniffer algorithms can pick up the trigger words "love" and "great" in the voice data from the user, and extract at least the corresponding portions of the voice data, shown in underline in the figure to include 60 the phrases "with Santa Barbara" and "wineries to visit." As discussed above, stop words can be removed and algorithms utilized to extract keywords such as "wineries" and Santa Barbara from the user's voice data" where "wineries to visit" is a command and keywords are stored from this command].
[Column 11, lines 30-33 "The server(s) also may be capable of executing programs or scripts in response requests from user devices, such as by executing programs or scripts in response requests from user devices…" where programs or scripts executed in response to finding a keyword match and thus if no keyword match, the corresponding action are not performed].
Claims [10 ] is rejected under 35 U.S.C. 103 as being unpatentable over Oktem (US 11,238,848 B2) in view of Chen (US 12,008,091 B2) and in further view of Wang (WO2017059679A1) and in further view of Abdel-Maguid (US 11,012,527 B2).
Regarding Claim 10, Oktem in view Chen in view of Wang teaches the electronic
device of claim 1, wherein the indicator device 110 outputs an indication that NLU module 220 may determine the intent is enrolling the voiceprint is not verified, or authenticated. For example, the device 110 may output an audio signal via a speaker, or a visual signal via a display, indicating that the user's verification failed, and requests that the user try again"].
However, Oktem in view of Chen in view of Wang do not teach the indicator includes at least one of a name, a nickname, an icon, or a photo.
But Abdel-Maguid teaches indicator includes at least one of a name, a nickname, an icon, or a photo - [Column2, lines 29-37 "The message is transmitted to a server computer which stores and forwards the message to the receiver users when the receiver users become available. The messages could be textual data, multimedia such as image, audio, and/or video. It is common for each user to have a profile containing identity information about the user. The profile may contain a name, short description, and image known as Avatar" where textual data or multimedia(image or photo) could be included with the indicator message].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings Oktem in view Chen in view of Wang with the teaching of Abdel-Maguid because displaying a user identity using a name, icon, or avatar in response to identification, would improve user recognition and interface clarity.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHEZA ABDUL AZIZ whose telephone number is (571)272-9610. The examiner can normally be reached Monday-Friday 7:30am-5pm Alternate Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHEZA ABDUL AZIZ/Examiner, Art Unit 2657
/DANIEL C WASHBURN/Supervisory Patent Examiner, Art Unit 2657
Prosecution Insights