DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicants’ amendment filed on 11/04/25 has been entered. Claims 1-2, 4, 6-9, 11-16, 20-22 has been amended. Claims 3 and 23 have been canceled. No new claims have been added. Claims 1-2, 4-16, 20-22 are still pending in this application, with claims 1, 20-21 being independent.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-16, 20-22 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication No. U.S. Patent Application Publication No. 20190354594 to Foster et al. (“Foster”) in view of U.S. Patent Application Publication No. 20230118412 to Gao et al. (“Gao”).
As to claim 1 and 21, Foster discloses a method and a system, the method implemented by one or more processors [paragraph 0055], the method comprising: as part of an ongoing dialog between a user of a client device and an automated assistant that is accessible at the client device [paragraph 0028: "conversational agent" , "mimic communication of a real person .... based on human conversation", "dialog"; paragraph 0045: "receive natural language messages from users and seek to return human-like natural language responses to the messages"]: receiving spoken natural language (NL) based input from the user of the client device during a given dialog turn of the ongoing spoken dialog between the user of the client device and the automated assistant [paragraph 0038, 0045]; determining one or more style signals that are specific to the given dialog turn of the ongoing dialog between the user of the client device and the automated assistant [paragraphs 0017, 0024 and 0037]; processing, using a large language model (LLM) behavior controller, the one or more style signals to determine a given NL based response style, from among a plurality of disparate NL based response styles, that is not specified by the spoken NL based input but is to be utilized in responding to the NL based input [As per Foster, the persona is also NOT selected by the NL based input, but by "configuration input" (paragraph 0017)]; processing, using a LLM [paragraph 0018], the spoken NL based input and a given NL based response style tag that is associated with the given NL based response style to generate LLM output [paragraphs 0018, 0042]; determining, based on the LLM output, an audible NL based response that is in the given NL based response style and that is responsive to the NL based input [paragraph 0038: “a language generation model is invoked. Here, the invoked model can be one of the persona-based language generation models 110, which can automatically generate a response to the query in language that corresponds to a persona (e.g., friendly, professional, funny, cool ... )"]; and causing the audible NL based response to be rendered at the client device [paragraph 0028: “"output generator for rendering a response”].
Foster does not expressly disclose persona being determining based on prosodic properties of the spoken NL based input.
In the same or similar field of invention, Gao discloses persona being determining based on prosodic properties of the spoken NL based input [Gao paragraph 0008, 0088, 0093].
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Foster to have feature of persona being determining based on prosodic properties of the spoken NL based input as taught by Gao. The suggestion/motivation would have been to provide feature of stylizing that machine-generated speech may enable users to experience a more natural interaction with an assistant system, thus reducing fatigue and building trust [Gao paragraph 0008].
Claim 20 has similar limitations of to claim 1 and 21 [see rejection of claims 1 and 21]. In addition, Claim 20 further claims “wherein the LLM was previously fine-tuned with respect to the plurality of disparate NL based response styles prior to the ongoing spoken dialog between the user of a client device and the automated assistant that is accessible at the client device”. Foster discloses wherein the LLM was previously fine-tuned with respect to the plurality of disparate NL based response styles prior to the ongoing spoken dialog between the user of a client device and the automated assistant that is accessible at the client device [paragraph 0027: “language generation model tuned for the selected domain can then be utilized” and paragraph 0043: “There are many different ways to customize a model for persona. As described above, one way is by using a persona identifier. More specifically, the closest speaker identifier from training data can be set as the persona identifier. In this case, a model does not have to be retrained. However, retraining can also be utilized to customize a model for persona. For example, a decoder of a model can be retrained to make it consistent with a desired persona “].
As to claims 2 and 22, Foster discloses wherein the spoken NL based input received from the user of the client device is a spoken utterance, and wherein the one or more style signals are determined based on processing the spoken utterance [paragraph 0028]. Foster includes speech-to-text input & text to-speech output (see [0028]) and therefore covers the case where the dialog is spoken and the NL based input is a spoken utterance. The various "style signals" described are well known features of human speech and no technical details of how the determined are disclosed, so that they must be assumed straightforward: in any case the result would be, at best, producing more human dialog output
As to claim 4, Foster discloses wherein processing the one or more style signals to determine the given NL based response style using the LLM behavior controller comprises: accessing a previously learned mapping that maps the one or more style signals obtained for the given dialog turn of the ongoing spoken dialog between the user of the client device and the automated assistant to the given NL based response style [paragraphs 0015, 0018, 0043, 0045].
As to claim 5, Foster discloses wherein the plurality of disparate response styles comprise one or more of: a dominant response style, a submissive response style, an inquisitive response style, a proactive response style, an engaging response style, a terse response style, a polite response style, or a direct response style [Foster discloses a number of “personas”, paragraphs 0017, 0037, 0040-41, 0043, 0045].
As to claim 6, Foster discloses prior to processing the spoken NL based input and the given NL based response style tag that is associated with the given NL based response style to generate the LLM output using the LLM: obtaining the given NL based response style tag that is associated with the given NL based response style; and pre-pending the given NL based response style tag to the spoken NL based input [paragraphs 0028, 0037, 0040, 0041, 0043, 0045].
As to claim 7, Foster discloses prior to processing the spoken NL based input and the given NL based response style tag that is associated with the given NL based response style to generate the LLM output using the LLM: obtaining the given NL based response style tag that is associated with the given NL based response style; and post-pending the given NL based response style tag to the spoken NL based input [paragraphs 0028, 0037, 0040, 0041, 0043, 0045].
As to claim 8, Foster discloses prior to processing the NL based input and the given NL based response style tag that is associated with the given NL based response style to generate the LLM output using the LLM: obtaining the given NL based response style tag that is associated with the given NL based response style; pre-pending the given NL based response style tag to the spoken NL based input; and post-pending the given NL based response style tag to the spoken NL based input [paragraphs 0028, 0037, 0040, 0041, 0043, 0045].
As to claim 9, Foster discloses obtaining one or more contextual signals associated with one or more of: the ongoing spoken dialog between the user of the user client device and the automated assistant, the user of the client device, or the client device; determining, based on the one or more contextual signals, a current context; and processing, using the LLM, and along with the spoken NL based input and the given NL based response style tag that is associated with the given NL based response style, the current context to generate the LLM output [paragraphs 0026, 0028, 0030, 0037, 0040, 0041, 0043, 0045]
As to claim 10, Foster discloses wherein the one or more contextual signals are distinct from the one or more style signals [paragraphs 0026, 0030, 0045].
As to claim 11, Foster discloses obtaining one or more contextual signals associated with one or more of: the user of the client device, or the client device; and processing, using the LLM, and along with the spoken NL based input and the given NL based response style tag that is associated with the given NL based response style, the one or more contextual signals to generate the LLM output [paragraphs 0026, 0030, 0045].
As to claim 12, Foster discloses as part of the ongoing spoken dialog between the user of the client device and the automated assistant that is accessible at the client device: receiving additional spoken NL based input from the user of the client device during a given additional dialog turn of the ongoing spoken dialog between the user of the client device and the automated assistant; determining one or more additional style signals that are specific to the additional given dialog turn of the ongoing spoken dialog between the user of the client device and the automated assistant; processing, using the LLM behavior controller, the one or more additional style signals to determine a given additional NL based response style, from among the plurality of disparate NL based response styles, that is not specified by the additional spoken NL based input but is to be utilized in responding to the additional spoken NL based input; processing, using the LLM, the additional spoken NL based input and a given additional response style tag that is associated with the given additional response style to generate additional LLM output; determining, based on the additional LLM output, an additional audible NL based response that is in the given additional response style and that is responsive to the additional spoken NL based input; and causing the additional audible NL based response to be rendered at the client device [See rejection of claim 1, what can be done for one dialog turn (as in claim 1) can be done for additional dialog turn]. Gao discloses persona being determining based on prosodic properties of the spoken NL based input [Gao paragraph 0008, 0088, 0093]. In addition, the same motivation is used as the rejection of claim 1.
As to claim 13, Foster discloses wherein causing the audible NL based response to be rendered at the client device comprises causing the audible NL based response to be audibly rendered at the client device via one or more speakers of the client device [paragraph 0028: “output generator for rendering a response (e.g. text-to-speech …), paragraph 0066 teaches output to display, speakers…].
As to claim 14, Foster discloses wherein the LLM output comprises a probability distribution over a sequence of words or phrases, and wherein determining the NL based response that is in the given audible NL based response style and that is responsive to the spoken NL based input based on the LLM output comprises: biasing, based on the given NL based response style, selection of one or more words or phrases for inclusion in the audible NL based response [paragraphs 0028, 0040, 0041, 0043].
As to claim 15, Foster discloses wherein biasing selection of the one or more words or phrases for inclusion in the audible NL based response based on the given NL based response style comprises: selecting, for inclusion in the audible NL based response, the one or more words or phrases that semantically reflect the given NL based response style [paragraph 0041, 0043: “words can be removed, or new words added by retraining a model. By way of example, vocabulary can be substantially limited for a child-oriented persona”].
As to claim 16, Foster discloses wherein causing the audible NL based response to be rendered at the client device comprises causing the audible NL based response to be audibly rendered at the client device via one or more speakers of the client device, and wherein causing the audible NL based response to be audibly rendered at the client device via one or more speakers of the client device comprises: processing, using a text-to-speech model, and based on one or more given audible NL based response style prosodic properties that verbally reflect the given NL based response style, the one or more words or phrases selected for inclusion in the audible NL based response to generate synthesized speech audio data that captures synthesized speech including the one or more words or phrases selected for inclusion in the audible NL based response [paragraph 0028: “output generator for rendering a response (e.g. text-to-speech …), paragraph 0066 teaches output to display, speakers…].
Response to Arguments
Applicant’s arguments with respect to claims 1-2, 4-16, 20-22 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTIM G SHAH whose telephone number is (571)270-5214. The examiner can normally be reached Mon-Fri 7:30am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached at 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTIM G SHAH/Primary Examiner, Art Unit 2693