Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed action
Claims 1-8, 10-13, 15 and 17-20 are pending and being considered.
Claims 1, 4, 13, 15, and 17 have been amended.
Response to 103
Applicant’s argument filed on 09/02/2025 have been fully considered and are not persuasive.
Arguments regarding claims 1 and 4:
In response to applicant’s argument on page 10 of remarks the applicant argues that Sharifi fails to teach “determine a device identifier associated with the device, determine the device corresponds to a device type and based on the device type, determine that sending of the first content to the device for presentation requires additional authorization from a user” the examiner respectfully disagrees because Sharifi teaches
In response to applicant's argument on page 10 that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., a system that determines the user device’s identifier, determine a device type corresponding to the determined device identifier and based on the device type determines that sending of the content to a device….) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The claim recites “determine a device identifier associated with the device, determine the device identifier corresponds to a device type and based on the device type, determine that sending of the first content to the device for presentation requires additional authorization from a user”
Arguments regarding claim 13:
In response to applicant’s argument on page 10 of remarks the applicant argues that Sharifi fails to teach “sending the device identifier to an access control system, determining, by the access control system, that the device identifier is associated with an indicator representing the user previously provided the additional authorization based on the device identifier being associated with the indicator, receiving, from the access control system, first data indicating the device is capable of outputting the first content” the examiner respectfully disagrees because Sharifi teaches on [0035] the profile service 170 may verify the identity of the user and verify that the user 10 consents to releasing the personal information 200 to the assistant service 300 and then release the requested personal information 200 to the assistants service 300 to fulfill the query 20 without requiring the user 10 to input the personal information in full (i.e., indication that user previously provided additional authorization). See on [0040-0042] the profile service receiving request for personal information, the personal information request 325 may identify the type of personal information the assistant service 300 needs as well as a user identifier identifying the user 10 so that the profile service 170 pulls the respective user profile 12 associated with the user 10 from the data storage 168 to inspect the personal information 200 and the user permissions 14. The personal information request 325 may also include a device identifier that uniquely identifies the user device 102 the user 10 is using to interact with the assistant service 300. Further teaches when the requested personal information 200 is stored in the centralized data store 168 and the set of user permissions 14 associated with the user 10 do not restrict the assistant service 300 from accessing the requested personal information, the profile service 170 performing the verification process through the assistant service 300.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4-9, 11-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sharifi et al (hereinafter Sharifi) (US 20230153410) in view of Zheng (US 20140011525).
Regarding claim 1 Sharifi teaches a computing system comprising: (Sharifi on [0057-0059] teaches a computing device that implements system and method described below);
at least one processor; and at least one memory comprising instructions that, when executed by the at least one processor, cause the computing system to: (Sharifi on [0057-0059] teaches a computing device that implements system includes a processor and memory storing instructions executed by the processor);
receive, from a device associated with profile data, first input audio data corresponding to a first spoken natural language user input requesting output of a first content (Sharifi on [0034] teaches the user 10 speaks a query 20 directed toward the assistant service 300 that is captured by the user device 110 (i.e., a device) in streaming audio indicating that the user 10 would like to purchase the book Fundamentals of Meteorology, the query 20 refers to a request to perform an action, operation, or task, and more specifically, a request for the assistant service 300 to perform an action, operation, or task related to purchasing the book for the user 10 i.e., input requesting output. See on [0053] teaches the speech recognizer 310 receives, as input, the audio data 120 characterizing the query 20 spoken by the user 10 that includes “That sounds perfect, I'd like to purchase this book” (i.e., input requesting output) shown in FIG. 1A and generates, as output, a speech recognition result 312 (e.g., transcription) of the query 20);
determine a device identifier associated with the device (Sharifi on [0036] teaches (user devices 110 include, but are not limited to, mobile devices (e.g., mobile phones, tablets, laptops, e-book readers, etc.) i.e., type of device. See on [0040] teaches the device identifier may indicate, or be used to obtain, a type of the user device);
determine the device identifier corresponds to a device type (Sharifi on [0036] teaches (user devices 110 include, but are not limited to, mobile devices (e.g., mobile phones, tablets, laptops, e-book readers, etc.) i.e., type of device. See on [0040] teaches the device identifier may indicate, or be used to obtain, a type of the user device);
based on the device type, determine that sending of the first content to the device for presentation requires additional authorization from a user (Sharifi on [0042] teaches an operating system the user device 110 runs may be ascertained from the device identifier contained in the personal information request 325 and be applied as an attribute for determining whether or not the user device is suitable for receiving OOB notifications. Further teaches the user device is not suitable for receiving OOB notifications when the user device does not include a display screen and/or application settings associated with the assistant service 300 indicate that the assistant service 300 is restricted from communicating OOB notifications to the user device 110 for gaining user consent (i.e., additional authorization) for releasing the personal information);
determine the additional authorization has yet to be granted (Sharifi on [0035] teaches in order to fulfill the query 20 (purchase and ship the book) spoken by the user 10, the assistant service 300 needs personal information 200 pertaining to the user's 10 payment details (e.g., credit card information) and home address. See on [0053-0054] teaches in order to identify an action 322 to perform in order to fulfil the query. Continuing with the example, the action 322 includes completing a transaction for the purchase of the book Fundamentals of Meteorology and shipment of the book to the user's address. Notably, the interpreter 320 learns that the personal information 200 pertaining to the user's payment details and address are needed for performing the action 322. As such, the interpreter 320 may generate the personal information request 325 that requests the profile service 170 to release the personal information 200 i.e., additional functionality access has yet to be granted);
receive, from an access control system, first data indicating the device is capable of outputting the first content (Sharifi on [0042-0043] teaches when the profile service 170 determines the user device 110 is suitable for receiving OOB notifications. FIG. 1C depicts the profile service 170 performing the verification process without the assistant service 300 by transmitting an OOB notification 250 to the user device 110 that causes the user device 110 to prompt the user 10 to provide consent to release the requested personal information to the assistant service. See on [0034-0035 and 0041] teaches the assistant service 300 prompts the user 10 by outputting synthesized speech 28 from the user device that requests the user to “Please input your shipping address and payment details in full so that I can ship you Fundamentals of Meteorology”. Subsequently, the user 10 inputs the personal information 200 (e.g., by spoken input or textual input) to the user device 110 for use by the assistant service 300 to fulfill the query 20. i.e., Note that OOB notifications is data that indicates that the device is capable of receiving content type based on releasing personal information);
based at least in part on receiving the first data, cause the device to present output audio data: indicating the additional authorization is required, and requesting the additional authorization (Sharifi on [0045-0046] teaches the instructions received by the assistant service 300 may cause the assistant service to output the synthesized speech 22 (i.e. output audio data) conveying the security statement 175 from the user device 110 tor the user 10 to answer via spoken input to provide consent tor releasing the personal information (i.e., additional access is required to output data). Further teaches the verification message output front the user device 110 as synthesized speech 22 states “So that I can access your address and payment details, please say ‘Bumblebee’ and speak your house number” i.e., requesting authorization. See on [0052] teaches releasing personal identification information after validating the token presented by the user. See also on [0044-0046] For instance, the token generator 172 may generate the arbitrary phrase “Bumblebee” as the unique token 174 prescribed to the user 10 to recite. As such, the instructions received by the assistant service 300 may cause the assistant service 300 to output a verification message as synthesized speech 22 from the user device 110 that prompts the user to speak each term of the one or more terms of the arbitrary phrase (e.g., Bumblebee) to provide consent for releasing the personal information 200. Additionally, the verification message output from the user device 110 as synthesized speech 22 may further indicate the type of personal information the assistant service is requesting the profile service to release);
after causing the device to present the output audio data, receive, from the device, second input audio data corresponding to a second spoken natural language user input including the additional authorization (Sharifi on [0046-0047] teaches the user device 110 captures the spoken utterance 24 and transmits audio data 121 characterizing the spoken utterance 24 to the profile service 170 for verification. Here, the profile service 170 executes an automatic speech recognition (ASR) system/model 176 for processing the received audio data 121 to generate a transcription 178 and executes a verifier 180 to determine whether the transcription 178 of the spoken utterance 24 recites each unique token 174 generated by the token generator 172. Continuing with the example, the verifier 180 is configured to verify the user consent (i.e., authorizing additional access) when the transcription 178 recites both the unique token 174 corresponding the arbitrary phrase “Bumblebee” and the unique token 174 corresponding to the fragments 202 associated with the house number “1-3-9” of the user's address (i.e., second input audio) as such, the profile service 170 may release the requested personal information 200 (e.g., the user's address and payment details) when the verifier determines the transcription 178 recites the unique tokens 174);
and based on the second spoken natural language user input, cause the first content to be sent to the device Sharifi on [0053-0054] teaches the interpreter 320 may generate the personal information request 325 that requests the profile service 170 to release the personal information 200. Using the techniques described above, the profile service 174 may release the requested personal information 200 pertaining to the payment details and address of the user to the executor 330. The executor 330 may be configured to perform the action 322 by purchasing the book from a merchant using the payment details of the user and brokering the shipment of the book to the user's address).
Although Sharifi teaches based on the second spoken natural language user input, cause the first content to be sent to the device, but fails to explicitly teach sending the first content to be displayed by the device, however Zheng from analogous art teaches
based on the second spoken natural language user input, cause the first content to be sent to the device for presentation (Zheng on [0044] teaches when receiving an aggregation display request from a user, an LBS system analyzes the user ID from the request, and verifies whether the user ID is valid. If the user is not a valid user of the LBS system, the LBS system denies the access of this user. If the user is a valid user of the LBS system, the LBS system obtains the user permission from the system database, and determines whether the requested content meets the permission, if not, indicates the user of unauthorized access information; if yes, sends the content requested by the user, and makes the content displayed on the aggregation display interface).
Thus, it would have been obvious to one ordinary skill in the art before the effective filing date to implement the teaching of Zheng into the teaching of Sharifi displaying the requested content on user device. One would be motivated to do so in order to manage and have access to requested data displayed on the user device (Zheng [0009-0019]).
Regarding claim 2 the combination of Sharifi and Zheng teaches all the limitations of claims 1 above, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: after determining the additional authorization has yet to be granted determine a number of previous spoken natural language user inputs received from the device and requiring the additional authorization; determine the number of previous spoken natural language user inputs satisfies a condition; (Sharifi on [0047-0048] teaches the profile service 170 may execute a speaker verification (SV) model 184 configured to receive the audio data 121 as input and generate, as output, an evaluation vector 186 representing voice characteristics of the spoken utterance 24. A scorer 188 may determine a SV confidence score indicating a probability/likelihood of the evaluation vector 186 matching a reference vector 205 for the user 10. The reference vector 205 may be obtained from the user profile 12 associated with the user and stored in the centralized data store 168. The SV model 184 may generate the reference vector 205 for the user 10 during a voice enrollment process (i.e., based on previous spoken natural language user input) where the reference vector 205 represents characteristics of the voice of the user. As such, the profile service 170 may verify the identity of the user 10 as the speaker of the spoken utterance 24 when the SV confidence score satisfies a confidence threshold (i.e., based on condition). In some scenarios, the TD-SV model 184 is generated dynamically using past instances of the user speaking a unique token as training data These past instances could pertain to previous interactions with the given assistant service 300 or other assistant services).
and based at least in part on determining the number of previous spoken natural language user inputs satisfies the condition, generate an access token corresponding to the profile data, wherein the access control system is configured to use the access token to receive the profile data, the access control system further configured to generate first data based at least in part on the profile data (Sharifi on [0032] teaches Thereafter, the profile service processes audio data of the spoken utterance captured by the user device to provide two-factor authentication where a transcription of the utterance must recite the correct house number (e.g., 5-1-0-5) and city name (e.g., Atlantic City) and a speaker-discriminative vector extracted from the audio data must match a reference speaker-discriminative vector associated with the user. As such, the profile service may perform speech recognition on the audio data to generate a transcription of the utterance to ascertain the house number and city name spoken by the user and determine whether it matches the corresponding house number and city name stored in the data store for the user. At the same time, the profile service may execute a speaker verification process by performing text-independent speaker identification (TI-SID) on the audio data to extract the speaker-discriminative vector and determine whether it matches the reference speaker-discriminative vector stored in the user profile for the user in the centralized data store. In addition to or in lieu of prompting the user to speak a fragment of the requested information, the profile service may generate a unique token (e.g., “Bumblebee”) and prompt the user to speak the unique token. In this scenario, the profile service may execute the speaker verification process by performing TI-SID on audio data characterizing the user uttering “Bumblebee” to extract the speaker-discriminative vector and determine whether it matches the reference speaker-discriminative vector for the user. Additionally or alternatively, the unique token may be a password known to the user and in which a text-dependent reference speaker-discriminative vector exists for the user where the user spoke the password during an enrollment process).
Regarding claim 4 Sharifi teaches a computing system comprising: (Sharifi on [0057-0059] teaches a computing device that implements system and method described below);
at least one processor; and at least one memory comprising instructions that, when executed by the at least one processor, cause the computing system to: (Sharifi on [0057-0059] teaches a computing device that implements system includes a processor and memory storing instructions executed by the processor);
receive, from a device, first input data corresponding to a first user input requesting output of a first content (Sharifi on [0034] teaches the user 10 speaks a query 20 directed toward the assistant service 300 that is captured by the user device 110 (i.e., first device) in streaming audio indicating that the user 10 would like to purchase the book Fundamentals of Meteorology, the query 20 refers to a request to perform an action, operation, or task, and more specifically, a request for the assistant service 300 to perform an action, operation, or task related to purchasing the book for the user 10. See on [0053] teaches the speech recognizer 310 receives, as input, the audio data 120 characterizing the query 20 spoken by the user 10 that includes “That sounds perfect, I'd like to purchase this book” (i.e., input requesting first content type) and generates output, a speech recognition result 312 (e.g., transcription) of the query 20);
determine a device identifier associated with the device (Sharifi on [0036] teaches (user devices 110 include, but are not limited to, mobile devices (e.g., mobile phones, tablets, laptops, e-book readers, etc.) i.e., type of device. See on [0040] teaches the device identifier may indicate, or be used to obtain, a type of the user device);
determine the device corresponds to a device type (Sharifi on [0036] teaches (user devices 110 include, but are not limited to, mobile devices (e.g., mobile phones, tablets, laptops, e-book readers, etc.) i.e., type of device. See on [0040] teaches the device identifier may indicate, or be used to obtain, a type of the user device);
based on the device type, determine that sending of the first content to the device for presentation requires additional authorization from a user (Sharifi on [0042] teaches an operating system the user device 110 runs may be ascertained from the device identifier contained in the personal information request 325 and be applied as an attribute for determining whether or not the user device is suitable for receiving OOB notifications. Further teaches the user device is not suitable for receiving OOB notifications when the user device does not include a display screen and/or application settings associated with the assistant service 300 indicate that the assistant service 300 is restricted from communicating OOB notifications to the user device 110 for gaining user consent (i.e., additional authorization) for releasing the personal information);
send, to an access control system, device profile data corresponding to the device (Sharifi on [0042-0043] teaches when the profile service 170 (i.e., access control system) determines the user device 110 is suitable for receiving OOB notifications (i.e., the user device 110 includes a display 116 and the application settings to not restrict OOB notifications) i.e., the profile service has access to profile data such as user device having display and application settings in order to determine that user device can receive OOB notification. See also on [0040] teaches in response to receiving the request 325 from the assistant service 300 for the personal information 200, FIG. 1A shows the profile service 170 determining whether the requested personal information 200 is stored in the centralized data store 168 and whether the set of user permissions 14 associated with the user 10 restrict the assistant service 300 from accessing the requested personal information 200. Here, the personal information request 325 may identify the type of personal information the assistant service 300 needs as well as a user identifier identifying the user 10 so that the profile service 170 pulls the respective user profile 12 associated with the user 10 from the data storage 168 to inspect the personal information 200 and the user permissions 14. The personal information request 325 may also include a device identifier that uniquely identifies the user device 102 (i.e., device profile data) the user 10 is using to interact with the assistant service 300. The device identifier may indicate, or be used to obtain, a type of the user device (e.g., phone, speaker, make/model, etc.), peripherals of the user device (e.g., display or no display?), an operating system running on the user device 110, or other characteristics associated with the user device 110.);
after sending the device profile data, receive, from the access control system, first data indicating the device is capable of outputting the first content (Sharifi on [0042-0043] teaches when the profile service 170 determines the user device 110 is suitable for receiving OOB notifications. FIG. 1C depicts the profile service 170 performing the verification process without the assistant service 300 by transmitting an OOB notification 250 to the user device 110 that causes the user device 110 to prompt the user 10 to provide consent to release the requested personal information (i.e., additional functionality access information) to the assistant service. See on [0034-0035 and 0041] teaches the assistant service 300 prompts the user 10 by outputting synthesized speech 28 from the user device that requests the user to “Please input your shipping address and payment details in full so that I can ship you Fundamentals of Meteorology”. Subsequently, the user 10 inputs the personal information 200 (e.g., by spoken input or textual input) to the user device 110 for use by the assistant service 300 to fulfill the query 20. i.e., Note that OOB notifications is data that indicates that the device is capable of receiving content type based on releasing personal information);
based at least in part on receiving the first data, cause the device to present output data requesting the additional authorization to send the first content to the device for presentation (Sharifi on [0043] teaches the user device 110 may display the prompt in a GUI presented on the display 116 and/or present the prompt via synthesized speech. In the example shown, the prompt is displayed as a textual message on the display 116 and provides selectable graphical elements that permit the user 10 to provide a user input indication indicating selection of the “Yes” graphical element to provide consent for releasing the personal information 200. See on [0045-0046] teaches the instructions received by the assistant service 300 may cause the assistant service to output the synthesized speech 22 (i.e., output audio data) conveying the security statement 175 from the user device 110 tor the user 10 to answer via spoken input to provide consent tor releasing the personal information (i.e., additional access is required to output data). Further teaches the verification message output front the user device 110 as synthesized speech 22 states “So that I can access your address and payment details, please say ‘Bumblebee’ and speak your house number” i.e., requesting authorization. See on [0052] teaches releasing personal identification information after validating the token presented by the user. See also on [0044-0046] For instance, the token generator 172 may generate the arbitrary phrase “Bumblebee” as the unique token 174 prescribed to the user 10 to recite. As such, the instructions received by the assistant service 300 may cause the assistant service 300 to output a verification message as synthesized speech 22 from the user device 110 that prompts the user to speak each term of the one or more terms of the arbitrary phrase (e.g., Bumblebee) to provide consent for releasing the personal information 200. Additionally, the verification message output from the user device 110 as synthesized speech 22 may further indicate the type of personal information the assistant service is requesting the profile service to release);
receive second input data corresponding to a second user input including the additional authorization; (Sharifi on [0046-0047] teaches the user device 110 captures the spoken utterance 24 and transmits audio data 121 characterizing the spoken utterance 24 to the profile service 170 for verification. Here, the profile service 170 executes automatic speech recognition (ASR) system/model 176 for processing the received audio data 121 to generate a transcription 178 and executes a verifier 180 to determine whether the transcription 178 of the spoken utterance 24 recites each unique token 174 generated by the token generator 172. Continuing with the example, the verifier 180 is configured to verify the user consent when the transcription 178 recites both the unique token 174 corresponding the arbitrary phrase “Bumblebee” and the unique token 174 corresponding to the fragments 202 associated with the house number “1-3-9” of the user's address (i.e., second user input) as such, the profile service 170 may release the requested personal information 200 (e.g., the user's address and payment details) when the verifier determines the transcription 178 recites the unique tokens 174. See also on [0044-0046] For instance, the token generator 172 may generate the arbitrary phrase “Bumblebee” as the unique token 174 prescribed to the user 10 to recite. As such, the instructions received by the assistant service 300 may cause the assistant service 300 to output a verification message as synthesized speech 22 from the user device 110 that prompts the user to speak each term of the one or more terms of the arbitrary phrase (e.g., Bumblebee) to provide consent for releasing the personal information 200. Additionally, the verification message output from the user device 110 as synthesized speech 22 may further indicate the type of personal information the assistant service is requesting the profile service to release).
and based on the second user input, cause the first content to be sent to the device
(Sharifi on [0053-0054] teaches the interpreter 320 may generate the personal information request 325 that requests the profile service 170 to release the personal information 200. Using the techniques described above, the profile service 174 may release the requested personal information 200 pertaining to the payment details and address of the user to the executor 330. The executor 330 may be configured to perform the action 322 by purchasing the book from a merchant using the payment details of the user and brokering the shipment of the book to the user's address).
Although Sharifi teaches based on the second spoken natural language user input, cause the first content to be sent to the device, but fails to explicitly teach sending the first content to be displayed by the device, however Zheng from analogous art teaches
based on the second user input, cause the first content to be sent to the device for presentation (Zheng on [0044] teaches when receiving an aggregation display request from a user, an LBS system analyzes the user ID from the request, and verifies whether the user ID is valid. If the user is not a valid user of the LBS system, the LBS system denies the access of this user. If the user is a valid user of the LBS system, the LBS system obtains the user permission from the system database, and determines whether the requested content meets the permission, if not, indicates the user of unauthorized access information; if yes, sends the content requested by the user, and makes the content displayed on the aggregation display interface).
Thus, it would have been obvious to one ordinary skill in the art before the effective filing date to implement the teaching of Zheng into the teaching of Sharifi displaying the requested content on user device. One would be motivated to do so in order to manage and have access to requested data displayed on the user device (Zheng [0009-0019]).
Regarding claim 13 Sharifi teaches a computer-implemented method comprising: (Sharifi on [0057-0059] teaches a computing device that implements system and method described below);
receiving, from a device, first input data corresponding to a first user input requesting output of a first content (Sharifi on [0034] teaches the user 10 speaks a query 20 directed toward the assistant service 300 that is captured by the user device 110 (i.e., first device) in streaming audio indicating that the user 10 would like to purchase the book Fundamentals of Meteorology, the query 20 refers to a request to perform an action, operation, or task, and more specifically, a request for the assistant service 300 to perform an action, operation, or task related to purchasing the book for the user 10. See on [0053] teaches the speech recognizer 310 receives, as input, the audio data 120 characterizing the query 20 spoken by the user 10 that includes “That sounds perfect, I'd like to purchase this book” (i.e., input requesting first content type) and generates output, a speech recognition result 312 (e.g., transcription) of the query 20);
determining a device identifier associated with the device (Sharifi on [0036] teaches (user devices 110 include, but are not limited to, mobile devices (e.g., mobile phones, tablets, laptops, e-book readers, etc.) i.e., type of device. See on [0040] teaches the device identifier may indicate, or be used to obtain, a type of the user device);
using the device identifier, determining that sending of the first content to the device for presentation requires additional authorization from a user (Sharifi on [0042] teaches an operating system the user device 110 runs may be ascertained from the device identifier contained in the personal information request 325 and be applied as an attribute for determining whether or not the user device is suitable for receiving OOB notifications. Further teaches the user device is not suitable for receiving OOB notifications when the user device does not include a display screen and/or application settings associated with the assistant service 300 indicate that the assistant service 300 is restricted from communicating OOB notifications to the user device 110 for gaining user consent (i.e., additional authorization) for releasing the personal information);
sending the device identifier to an access control system (Sharifi on [0040] teaches the profile service receiving request for personal information, the personal information request 325 may identify the type of personal information the assistant service 300 needs as well as a user identifier identifying the user 10 so that the profile service 170 pulls the respective user profile 12 associated with the user 10 from the data storage 168 to inspect the personal information 200 and the user permissions 14. The personal information request 325 may also include a device identifier that uniquely identifies the user device 102 the user 10 is using to interact with the assistant service 300. The device identifier may indicate, or be used to obtain, a type of the user device (e.g., phone, speaker, make/model, etc.))
determining, by the access control system, that the device identifier is associated with an indicator representing the user previously provided the additional authorization (Sharifi on [0030] teaches users/clients may explicitly grant permission to the profile service for storing their personal information and profiles in the centralized data store Additionally, these users/clients may also define application permissions indicating which services/applications are allowed to access the stored personal information and/or which services/applications are restricted from accessing the stored personal information. As such, the profile service may grant, with the consent of the user, these applications/services access to the personal information stored in the centralized data store when needed without having to burden the user by requiring these applications/services to request the user to input the same personal information repeatedly. See on [0035] teaches the profile service 170 may verify the identity of the user and verify that the user 10 consents to releasing the personal information 200 to the assistant service 300 and then release the requested personal information 200 to the assistants service 300 to fulfill the query 20 without requiring the user 10 to input the personal information in full. See on [0040-0042] the profile service receiving request for personal information, the personal information request 325 may identify the type of personal information the assistant service 300 needs as well as a user identifier identifying the user 10 so that the profile service 170 pulls the respective user profile 12 associated with the user 10 from the data storage 168 to inspect the personal information 200 and the user permissions 14. The personal information request 325 may also include a device identifier that uniquely identifies the user device 102 the user 10 is using to interact with the assistant service 300. Further teaches when the requested personal information 200 is stored in the centralized data store 168 and the set of user permissions 14 associated with the user 10 do not restrict the assistant service 300 from accessing the requested personal information, FIG. 1B shows the profile service 170 performing the verification process through the assistant service 300.)
(Sharifi on [0042-0043] teaches when the requested personal information 200 is stored in the centralized data store 168 and the set of user permissions 14 associated with the user 10 do not restrict the assistant service 300 from accessing the requested personal information, FIG. 1B shows the profile service 170 performing the verification process through the assistant service 300. Optionally, the profile service 170 may determine whether the user device 110 is suitable for receiving out-of-band (OOB) notifications directly from the profile service 170 and only perform the verification process through the assistant service 300, the user device is not suitable for receiving OOB notifications when the user device does not include a display screen and/or application settings associated with the assistant service 300 indicate that the assistant service 300 is restricted from communicating OOB notifications to the user device 110 for gaining user consent for releasing the personal information. Further teaches when the profile service 170 determines the user device 110 is suitable for receiving OOB notifications (i e., the user device 110 includes a display 116 and the application settings to not restrict OOB notifications), FIG. 1C depicts the profile service 170 performing the verification process without the assistant service 300 by transmitting an OOB notification 250 to the user device 110 that causes the user device 110 to prompt the user 10 to provide consent to release the requested personal information to the assistant service 300. Notably, the OOB notification 250 is communicated directly from the profile service 170 to the user device 110 via the network 130 without the need of the assistant service 300. The user device 110 may display the prompt in a GUI presented on the display 116 and/or present the prompt via synthesized speech);
and(Sharifi on [0053-0054] teaches the interpreter 320 may generate the personal information request 325 that requests the profile service 170 to release the personal information 200. Using the techniques described above, the profile service 174 may release the requested personal information 200 pertaining to the payment details and address of the user to the executor 330. The executor 330 may be configured to perform the action 322 by purchasing the book from a merchant using the payment details of the user and brokering the shipment of the book to the user's address).
Although Sharifi teaches based on the second spoken natural language user input, cause the first content to be sent to the device, but fails to explicitly teach sending the first content to be displayed by the device, however Zheng from analogous art teaches
cause the first content to be sent to the device for presentation (Zheng on [0044] teaches when receiving an aggregation display request from a user, an LBS system analyzes the user ID from the request, and verifies whether the user ID is valid. If the user is not a valid user of the LBS system, the LBS system denies the access of this user. If the user is a valid user of the LBS system, the LBS system obtains the user permission from the system database, and determines whether the requested content meets the permission, if not, indicates the user of unauthorized access information; if yes, sends the content requested by the user, and makes the content displayed on the aggregation display interface).
Thus, it would have been obvious to one ordinary skill in the art before the effective filing date to implement the teaching of Zheng into the teaching of Sharifi displaying the requested content on user device. One would be motivated to do so in order to manage and have access to requested data displayed on the user device (Zheng [0009-0019]).
Regarding claim 5 the combination of Sharifi and Zheng teaches all the limitations of claims 4 and above, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: determine a number of previous user inputs received from the device and requiring the additional authorization; determine the number of previous user inputs satisfies a condition; and generate the first output data further based at least in part on the number of previous user inputs satisfying the condition. (Sharifi on [0047-0048] teaches the profile service 170 may execute a speaker verification (SV) model 184 configured to receive the audio data 121 as input and generate, as output, an evaluation vector 186 representing voice characteristics of the spoken utterance 24. A scorer 188 may determine a SV confidence score indicating a probability/likelihood of the evaluation vector 186 matching a reference vector 205 for the user 10. The reference vector 205 may be obtained from the user profile 12 associated with the user and stored in the centralized data store 168. The SV model 184 may generate the reference vector 205 for the user 10 during a voice enrollment process (i.e., based on previous spoken natural language user input) where the reference vector 205 represents characteristics of the voice of the user. As such, the profile service 170 may verify the identity of the user 10 as the speaker of the spoken utterance 24 when the SV confidence score satisfies a confidence threshold (i.e., based on condition). In some scenarios, the TD-SV model 184 is generated dynamically using past instances of the user speaking a unique token as training data These past instances could pertain to previous interactions with the given assistant service 300 or other assistant services).
Regarding claim 6 and 15 the combination of Sharifi and Zheng teaches all the limitations of claims 4 and 13 respectively, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: generate an access token corresponding to the device profile data; send the access token to the access control system; and after sending the access token, receive second data from the access control system, wherein the second data indicates at least one requirement for sending the first content to the device (Sharifi on [0032 and 0042-0044] teaches Thereafter, the profile service processes audio data of the spoken utterance captured by the user device to provide two-factor authentication where a transcription of the utterance must recite the correct house number (e.g., 5-1-0-5) and city name (e.g., Atlantic City) and a speaker-discriminative vector extracted from the audio data must match a reference speaker-discriminative vector associated with the user. As such, the profile service may perform speech recognition on the audio data to generate a transcription of the utterance to ascertain the house number and city name spoken by the user and determine whether it matches the corresponding house number and city name stored in the data store for the user. At the same time, the profile service may execute a speaker verification process by performing text-independent speaker identification (TI-SID) on the audio data to extract the speaker-discriminative vector and determine whether it matches the reference speaker-discriminative vector stored in the user profile for the user in the centralized data store. In addition to or in lieu of prompting the user to speak a fragment of the requested information, the profile service may generate a unique token (e.g., “Bumblebee”) and prompt the user to speak the unique token. In this scenario, the profile service may execute the speaker verification process by performing TI-SID on audio data characterizing the user uttering “Bumblebee” to extract the speaker-discriminative vector and determine whether it matches the reference speaker-discriminative vector for the user. Additionally or alternatively, the unique token may be a password known to the user and in which a text-dependent reference speaker-discriminative vector exists for the user where the user spoke the password during an enrollment process).
Regarding claim 7 and 16 the combination of Sharifi and Zheng teaches all the limitations of claims 4 and 13 respectively, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: determine a user identifier associated with the second user input; determine the user identifier is authorized to be used in providing the additional authorization; and cause the first content to be sent to the device (Sharifi on [0040 and 0047] teaches the personal information request 325 may identify the type of personal information the assistant service 300 needs as well as a user identifier identifying the user 10 so that the profile service 170 pulls the respective user profile 12 associated with the user 10 from the data storage 168 to inspect the personal information 200 and the user permissions 14. See on [0050] teaches when the transcription 178 of the spoken utterance 24 recites (permitting some leeway) the unique tokens and the scorer 188 determines the utterance 24 was spoken by the user (e.g., the identity of the user is verified as the speaker of the utterance), the profile service verifies that the user 10 consents to the release of the requested personal information and thereby releases the requested personal information 200 stored on the centralized data store 168 to the assistants service 300).
and cause the first content to be sent to the device for presentation (Zheng on [0044] teaches when receiving an aggregation display request from a user, an LBS system analyzes the user ID from the request, and verifies whether the user ID is valid. If the user is not a valid user of the LBS system, the LBS system denies the access of this user. If the user is a valid user of the LBS system, the LBS system obtains the user permission from the system database, and determines whether the requested content meets the permission, if not, indicates the user of unauthorized access information; if yes, sends the content requested by the user, and makes the content displayed on the aggregation display interface).
Thus, it would have been obvious to one ordinary skill in the art before the effective filing date to implement the teaching of Zheng into the teaching of Sharifi displaying the requested content on user device. One would be motivated to do so in order to manage and have access to requested data displayed on the user device (Zheng [0009-0019]).
Regarding claim 8 and 17 the combination of Sharifi and Zheng teaches all the limitations of claims 4 and 13 respectively, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: determine state data corresponding to the device; determine the state data indicates the additional authorization has yet to be provided; and send the device profile data to the access control system based at least in part on determining the state data indicates the additional authorization has yet to be provided (Sharifi on [0035] teaches in order to fulfill the query 20 (purchase and ship the book) spoken by the user 10, the assistant service 300 needs personal information 200 pertaining to the user's 10 payment details (e.g., credit card information) and home address. See on [0053-0054] teaches in order to identify an action 322 to perform in order to fulfil the query. Continuing with the example, the action 322 includes completing a transaction for the purchase of the book Fundamentals of Meteorology and shipment of the book to the user's address. Notably, the interpreter 320 learns that the personal information 200 pertaining to the user's payment details and address are needed for performing the action 322. As such, the interpreter 320 may generate the personal information request 325 that requests the profile service 170 to release the personal information 200 i.e., additional functionality access has yet to be granted).
Regarding claim 9 and 18 the combination of Sharifi and Zheng teaches all the limitations of claims 4 and 13 respectively, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: determine the device corresponds to a device type (Sharifi on [0036] teaches (user devices 110 include, but are not limited to, mobile devices (e.g., mobile phones, tablets, laptops, e-book readers, etc.) i.e., type of device. See on [0040] teaches the device identifier may indicate, or be used to obtain, a type of the user device);
and determine the additional authorization is required based at least in part on the device corresponding to the device type (Sharifi on [0040-0044] teaches the device identifier may indicate, or be used to obtain, a type of the user device and releasing the requested information response to type of user device).
Regarding claim 11 and 20 the combination of Sharifi and Zheng teaches all the limitations of claims 4 and 13 respectively, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: determine an intent representing the first user input; and determine the additional authorization is required based at least in part on the intent (Sharifi on [0034] teaches the user 10 speaks a query 20 directed toward the assistant service 300 that is captured by the user device 110 (i.e., first device) in streaming audio indicating that the user 10 would like to purchase the book Fundamentals of Meteorology (i.e., intent of user), the query 20 refers to a request to perform an action, operation, or task, and more specifically, a request for the assistant service 300 to perform an action, operation, or task related to purchasing the book for the user 10 i.e., input requesting output. See on [0053] teaches the speech recognizer 310 receives, as input, the audio data 120 characterizing the query 20 spoken by the user 10 that includes “That sounds perfect, I'd like to purchase this book” (i.e., input requesting output) shown in FIG. 1A and generates, as output, a speech recognition result 312 (e.g., transcription) of the query 20);
Regarding claim 12 the combination of Sharifi and Zheng teaches all the limitations of claims 4 above, Sharifi further teaches wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: receive, from the access control system, second data indicating that the additional authorization has been provided (Sharifi on [0046-0047] teaches the user device 110 captures the spoken utterance 24 and transmits audio data 121 characterizing the spoken utterance 24 to the profile service 170 for verification. Here, the profile service 170 executes an automatic speech recognition (ASR) system/model 176 for processing the received audio data 121 to generate a transcription 178 and executes a verifier 180 to determine whether the transcription 178 of the spoken utterance 24 recites each unique token 174 generated by the token generator 172. Continuing with the example, the verifier 180 is configured to verify the user consent when the transcription 178 recites both the unique token 174 corresponding the arbitrary phrase “Bumblebee” and the unique token 174 corresponding to the fragments 202 associated with the house number “1-3-9” of the user's address (i.e., second input audio) as such, the profile service 170 may release the requested personal information 200 (e.g., the user's address and payment details) when the verifier determines the transcription 178 recites the unique tokens 174);
and after receiving the second data, send the first content to the device (Sharifi on [0053-0054] teaches the interpreter 320 may generate the personal information request 325 that requests the profile service 170 to release the personal information 200. Using the techniques described above, the profile service 174 may release the requested personal information 200 pertaining to the payment details and address of the user to the executor 330. The executor 330 may be configured to perform the action 322 by purchasing the book from a merchant using the payment details of the user and brokering the shipment of the book to the user's address).
Send the first content to the device for presentation (Zheng on [0044] teaches when receiving an aggregation display request from a user, an LBS system analyzes the user ID from the request, and verifies whether the user ID is valid. If the user is not a valid user of the LBS system, the LBS system denies the access of this user. If the user is a valid user of the LBS system, the LBS system obtains the user permission from the system database, and determines whether the requested content meets the permission, if not, indicates the user of unauthorized access information; if yes, sends the content requested by the user, and makes the content displayed on the aggregation display interface).
Thus, it would have been obvious to one ordinary skill in the art before the effective filing date to implement the teaching of Zheng into the teaching of Sharifi displaying the requested content on user device. One would be motivated to do so in order to manage and have access to requested data displayed on the user device (Zheng [0009-0019]).
Regarding claim 15 the combination of Sharifi and Zheng teaches all the limitations of claim 13 above, Sharifi further teaches generating an access token corresponding to the device profile data; sending the access token to the access control system; and after sending the access token, receiving the first data from the access control system (Sharifi on [0032 and 0042-0044] teaches Thereafter, the profile service processes audio data of the spoken utterance captured by the user device to provide two-factor authentication where a transcription of the utterance must recite the correct house number (e.g., 5-1-0-5) and city name (e.g., Atlantic City) and a speaker-discriminative vector extracted from the audio data must match a reference speaker-discriminative vector associated with the user. As such, the profile service may perform speech recognition on the audio data to generate a transcription of the utterance to ascertain the house number and city name spoken by the user and determine whether it matches the corresponding house number and city name stored in the data store for the user. At the same time, the profile service may execute a speaker verification process by performing text-independent speaker identification (TI-SID) on the audio data to extract the speaker-discriminative vector and determine whether it matches the reference speaker-discriminative vector stored in the user profile for the user in the centralized data store. In addition to or in lieu of prompting the user to speak a fragment of the requested information, the profile service may generate a unique token (e.g., “Bumblebee”) and prompt the user to speak the unique token. In this scenario, the profile service may execute the speaker verification process by performing TI-SID on audio data characterizing the user uttering “Bumblebee” to extract the speaker-discriminative vector and determine whether it matches the reference speaker-discriminative vector for the user. Additionally or alternatively, the unique token may be a password known to the user and in which a text-dependent reference speaker-discriminative vector exists for the user where the user spoke the password during an enrollment process).
Claims 3, 10 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sharifi et al (hereinafter Sharifi) (US 20230153410) in view of Zheng (US 20140011525) and further in view of Mahmood et al (hereinafter Mahmood) (US 20210090575).
Regarding claim 3, 10 and 19 the combination of Sharifi and Zheng teaches all the limitations of claims 1, 4 and 13 respectively, the combination fails to explicitly teach wherein the first content corresponds to long- form audio, and wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: determine the additional authorization is required based at least in part on the first content corresponding to the long-form audio, however Mahmood from analogous art teaches wherein the first content corresponds to long- form audio, and wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to: determine the additional authorization is required based at least in part on the first content corresponding to the long-form audio (Mahmood on [0025, 0031 and 0097-0100] teaches A natural language processing (NLP) system may cause skill systems to perform actions in response to natural language inputs (e.g., spoken inputs and/or typed inputs). For example, for the natural language input “play Adele music,” a music skill system may be invoked to output music sung by an artist named Adele (i.e., long form audio). See on [0214] teaches ASR feature vector data 1507 may include a different audio feature vector for each audio frame. Thus, for one 25 ms long audio frame, the ASR feature extraction component 1506 may output a single ASR feature vector. The ASR feature vectors 1507 output by the ASR feature extraction component 1506 may be output to the ASR component 250).
Thus, it would have been obvious to one ordinary skill in the art before the effective filing date to implement the teaching of Mahmood into the combined teaching of Sharifi and Zheng by granting access to first content type corresponding to long form audio. One would be motivated to do so in order to perform action based on long-form audio such as playing music (Mahmood [00025 and 0031]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOEEN KHAN whose telephone number is (571)272-3522. The examiner can normally be reached 7AM-5PM EST M-TH Alternate Fridays.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shewaye Gelagay can be reached on (571)272-4219. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOEEN KHAN/Primary Examiner, Art Unit 2436