DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
Acknowledgment is made of the information disclosure statements filed on 10/10/2024, 01/17/2025, 8/12/2025, 1/20/2026, 2/4/2026, 2/18/2026 and 5/15/2026. It is noted by the Examiner that all of the references were considered.
Priority
The instant application has provisional application 17/532,794, date 11/22/2021 and another provisional 63/241,232 filed on 09/07/2021 and PCT/US21/61214, Filing Date November/30/2021.
Drawings
The drawings submitted on 10/10/2024 have been considered and accepted.
Claim objections
Claim 17 and 19 are claim dependency errors. These claims should be dependent on system claim 11. and Claim 17 should be system claim. Appropriate corrections are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1- 20 are rejected under 35 U.S.C. § 101 because the claims are directed to a judicial exception (an abstract idea) and do not recite additional elements that amount to significantly more.
Summary of the statutory framework and guidance applied
The Office applies the two-step framework for subject matter eligibility consistent with current USPTO guidance.
Step 1: Determine whether the claim recites a statutory category (process, machine, manufacture, or composition of matter).
Step 2A, Prong One: If the claim recites a statutory category, determine whether the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon).
Step 2A, Prong Two: If a judicial exception is recited, determine whether the claim integrates the exception into a practical application.
Step 2B: If the claim does not integrate the exception into a practical application, determine whether the claim recites additional elements that amount to significantly more than the judicial exception.
Step 1 (statutory category)
Claims 1-10 are directed to a method (process).
Claims 11-19 are directed to a system (machine).
Claim 20 is directed to a non-transitory computer-readable medium (manufacture). Each of these falls within a statutory category.
Step 2A, Prong One (judicial exception)
The claims recite Collecting/receiving utterance/question from a user, and answer the question which is mental process organizing human activity. The claim recites nothing more than the generic sequence data in, generic model processing and data out. These elements constitute a judicial exception in the form of abstract mental processing and data processing.
Step 2A, Prong Two (integration into a practical application)
The claims do not integrate the judicial exception into a practical application. The additional elements recited—generic computing components (processors, client device, microphone, LLM),—are described at a high level and perform conventional functions of receiving, generic model processing and outputting data. The claims do not describe a specific improvement to the functioning of the computer or another technological improvement. They instead use generic computer implementation to carry out the abstract mental activity.
Step 2B (significantly more)
The claims recite no additional elements or combination of elements that amount to significantly more than the abstract idea. The claimed processors, client device (spec, [0032]) , microphone, LLM, are well-understood, routine, and conventional components in the field of machine learning and natural language processing. {Confirmed by the specification [0032] & [0042] itself which describe the LLM as any of several known off-the-shelf architectures.}, Client device is also generic component according to spec [0032] . Microphone is pre-solution activity. Also, in US 20200402507 A1, Neelakantan teaches transformer model. (“[0097] In some implementations, a method is provided that includes generating, using an automatic speech recognition component, recognized text from a spoken utterance of a user during a dialog session. The dialog session is between the user and an automated assistant, and the spoken utterance is captured in audio data detected by one or more microphones of a client device. The method further includes adding the spoken utterance to a dialog history of the dialog session. The method further includes generating an encoded dialog representation based on processing the recognized text, and any other text of the dialog history of the dialog session, using a transformer encoder. … … using a transformer decoder attended to both the encoded dialog representation and the encoded resource representation.”) The claims do not recite a non-conventional arrangement of components or otherwise specify how the claimed elements effect a technological improvement.
Limitation-by-limitation analysis for independent claims
Claim 1 (method) – Limitation analysis
“receiving a stream of audio data that captures a spoken utterance of the user, the stream of audio data being generated by one or more microphones of the client device, and the spoken utterance including an assistant query;” — This is data gathering/input identification and is a conventional, pre-solution activity.
“determining, based on processing the stream of audio data, a set of assistant outputs, each assistant output in the set of assistant outputs being responsive to the assistant query included in the spoken utterance;” —This recites identifying query can be utterance. If query is about restaurant, response can be near by restaurant, hours of operation, menu etc. Human can response to the querry and understand what could be the response.
“processing, using a large language model (LLM), the set of assistant outputs and context of the dialog session to generate a set of modified assistant outputs that each reflect a first personality from among a plurality of disparate personalities; and.” — Human can understand context in the sentence or dialog query, and provide answer based on context, and response can be modified based on contextual meaning of the sentence. The additional elements recited in the claim—generic computing components ( LLM),—are described at a high level and perform conventional functions of receiving, generic model processing and outputting data can be edited/modified according to various context.
“causing a given modified assistant output, from among the set of modified assistant outputs, to be provided for presentation to the user” — recites output given to the user. The claim recites post solution activity.
Conclusion for claim 1: The claim is directed to abstract mental activity of human.
conventional ML elements; it does not integrate the judicial exception into a practical application and does not include additional elements that amount to significantly more.
Claim 11 (system) – Limitation analysis
The claim recites generic hardware components (“one or more processors,” “memory”, spec. [0136] )configured to execute instructions that perform the same abstract data recited in claim 1. Implementing the abstract method on generic hardware does not provide a practical application or significantly more.
Conclusion for claim 11: The system claim is ineligible for the same reasons as claim 1.
Claim 20 (computer-readable medium) – Limitation analysis
The claim recites instructions stored on a non-transitory medium that cause a processor to perform the same abstract steps recited in claim 1. The storage of instructions to perform an abstract method on a computer-readable medium does not render the subject matter eligible.
Conclusion for claim 20: The claim is ineligible for the same reasons as claim 1.
Dependent claims:
(claims 2/12)- The dependent claims “wherein processing the set of assistant outputs and the context of the dialog session to generate the set of modified assistant outputs that each reflect the first personality and using the LLM causes the LLM to adapt the set of assistant output to a first vocabulary, that is associated with the first personality, to generate the set of modified assistant outputs.” Human can understand query meaning, and personal traits like age, gender, and location. The output can be personalized based on what is the utterance/query, human can understand conversation/dialog session between male, female or/and regarding male and female, or/and location, gender or age. Human can understand query is being new or augmented, based on that human can response answer by modifying previous answer or provide dynamically produce new answer. Human can understand domain specific conversation, or male and female, location based conversation because word choice in the sentence will be specific to that domain. For example, younger people conversation and older people conversation will vary. Same with male female same with different language, and different people conversation. Which recites modify output to a vocabulary goes with personality. Human can understand some word choices are from female and male in the same topic or same conversation. Also, butler and chef will use different vocabularies while they are talking. They do not integrate the abstract idea into a practical application, nor do they supply additional elements or an inventive concept sufficient to render the claims eligible.
Claim(3/13)- “wherein the first personality is disparate from a second personality, and wherein the first vocabulary, that is associated with the first personality and that is used to generate the set of modified assistant outputs, is disparate from a second vocabulary, that is associated with the second personality.”- Human can respond to the same query based on different personalities. Human can respond by modifying previous answer or dynamically produce new answer. Human can understand domain specific conversation, or male and female, or location-based conversation because word choice in the sentence will be specific to that domain. For example, younger people conversation and older people conversation will vary. The claim recites vocabulary are completely different from one another. Human know if vocabulary are same or different in a sentence.
Claim(4/14)- “wherein causing the given modified assistant output to be provided for presentation to the user comprises: generating, using a first set of prosodic properties that is associated with the first personality, synthesized speech audio data that captures the given modified assistant output; and causing the synthesized speech audio data that captures the given modified assistant output to be audibly rendered via one or more speakers of the client device.”-Human can think and generate speech. Human can respond to the same query based on different personalities. Human can respond by modifying previous answer or dynamically produce new answer. Human can understand domain specific conversation, or male and female, or location-based conversation because word choice in the sentence will be specific to that domain. For example, younger people conversation and older people conversation will vary. The additional elements recited—generic components, Speaker and client device (spec.31],—are described at a high level and perform conventional functions of receiving, generic model processing and outputting data. Also, Outputting audio data is post processing. The claim recites no specific technical mechanism for how prosody is generated or mapped.
Claim 5/15 -“wherein the first personality is disparate from a second personality, and wherein the first set of prosodic properties, that is associated with the first personality and that is used to generate the synthesized speech audio data, is disparate from a second set of prosodic properties, that is associated with the second personality.”- the claim recites distinguished prosodic property sets for two different personalities. No new additional element used that can overcome claim eligible.
Claim 6/16 -“selecting the given modified assistant output, from among the set of modified assistant outputs, based on a probability distribution over a sequence of words or phrases that is generated using the LLM.” – Human can choose one response from plurality of responses based on confidence. the claim recites mathematical operation as selection mechanism. This claim recites mathematical algorithm combine with mental process. The claim recites LLM as an additional element, but Zheng describe token selection like ordinary operation. (“Note that although the bias brought by vp seems to be context independent (i.e., it may select words independently at each decoding step), …” page 5, column 1, section 3.4.2) by Zheng
Claim 7/17- “wherein the LLM is a first LLM, from among a plurality of disparate LLMs, that is specific to the first personality.” Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim(s) is/are directed to an abstract idea. The claim recites LLM is one of disparate personality LLM. The additional elements recited—generic components, LLM,—are described at a high level and perform conventional functions of receiving, generic model processing and outputting data.
Claim 8/18-“wherein the first personality is defined by the user in settings of a software application.” Humen can configure setting that reflects user preferences. The claim recites user preference.
Claim 9/19- “as part of a subsequent dialog session between the user and the automated assistant implemented by the client device: receiving a subsequent stream of audio data that captures a subsequent spoken utterance of the user, the subsequent stream of audio data being generated by one or more of the microphones of the client device, and the subsequent spoken utterance including a subsequent assistant query; determining, based on processing the subsequent stream of audio data, a subsequent set of assistant outputs, each assistant output in the subsequent set of assistant outputs being responsive to the subsequent assistant query included in the subsequent spoken utterance; processing, using the LLM or an additional LLM, the subsequent set of assistant outputs and subsequent context of the subsequent dialog session to generate a subsequent set of modified assistant outputs that each reflect a second personality that differs from the first personality; and causing a given subsequent modified assistant output, from among the subsequent set of modified assistant outputs, to be provided for presentation to the user.” The claim recites second dialog session and same process repeated as claim 1. The new conversation starts. The additional element recited as claim 1. The claim is ineligible because of human mental process tied with generic addition components.
Claim 10-“wherein the subsequent set of assistant outputs and the subsequent context of the subsequent dialog session are processed using the additional LLM to generate the subsequent set of modified assistant outputs that each reflect the second personality that differs from the first personality, wherein the LLM is associated with the first personality, and wherein the additional LLM is associated with the first personality.”- Human can respond to the same query based on different personalities. Human can respond by modifying previous answer or dynamically produce new answer. Human can understand domain specific conversation, or male and female, or location-based conversation because word choice in the sentence will be specific to that domain. For example, younger people conversation and older people conversation will vary. The claim recites vocabulary are completely different from one another. Human know if vocabulary are same or different in a sentence. The claim recites additional LLM. The claim is ineligible because of human mental process tied with generic addition components.
Overall conclusion
Claims 1- 20 are rejected under 35 U.S.C. § 101 as being directed to an abstract idea (mental process + mathematical concepts) and failing to recite additional elements or combinations that amount to significantly more than the judicial exception. The claims are therefore not directed to patent-eligible subject matter.
REJECTION
Claims 1- 20 are rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter (an abstract idea) and lacking additional limitations that transform the abstract idea into patent-eligible subject matter.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1, 2, 3, 9-10, 12, 13, and 19-20 are rejected on the ground of non-statutory double patenting as being unpatentable over claims 1, 3-7, 17, 18, 21-29 of U.S. Patent No. US12148421 B2 Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the issued patent are narrower in scope than claims of the instant application. Therefore, the claims of the issued patent anticipate the claims of the instant application. Please see the below claim mapping.
Each of the other claims map as the following, where (I) represents the instant application and (P) represents the issued patent: Claim 1 (I), Claim 4 (P); Claim 2 (I), Claim 5+6 (P); Claim 3 (I), Claim 5+6 (P); Claim 9 (I), Claim 4 (P); Claim 10 (I), Claim 7 (P);
Instant Application 18/912,175 { 20250037711}
EFD: 09/07/2021
US12148421 B2
1. A method implemented by one or more processors, the method comprising: as part of a dialog session between a user of a client device and an automated assistant implemented by the client device:
receiving a stream of audio data that captures a spoken utterance of the user, the stream of audio data being generated by one or more microphones of the client device, and
the spoken utterance including an assistant query;
determining, based on processing the stream of audio data, a set of assistant outputs, each assistant output in the set of assistant outputs being responsive to the assistant query included in the spoken utterance;
causing a given modified assistant output, from among the set of modified assistant outputs, to be provided for presentation to the user.
processing, using a large language model (LLM), the set of assistant outputs and context of the dialog session to generate a set of modified assistant outputs that each reflect a first personality from among a plurality of disparate personalities; and
1. (Previously Presented) A method implemented by one or more processors, the method comprising: as part of a dialog session between a user of a client device and an automated assistant implemented by the client device:
receiving a stream of audio data that captures a spoken utterance of the user, the stream of audio data being generated by one or more microphones of the client device, and
the spoken utterance including an assistant query;
determining, based on processing the stream of audio data, a set of assistant outputs, each assistant output in the set of assistant outputs being responsive to the assistant query included in the spoken utterance, wherein determining the set of assistant outputs comprises:
processing, using an automatic speech recognition model, the stream of audio data to generate ASR output;
processing, using a natural language understanding (NLU) model, the ASR output to generate NLU output; and causing the set of assistant outputs to be determined based on the NLU output;
processing the set of assistant outputs and context of the dialog session to: generate a set of modified assistant outputs using one or more large language model (LLM) outputs generated using an LLM, each of the one or more LLM outputs being determined by processing, using the LLM, at least part of the context of the dialog session and one or more of the assistant outputs included in the set of assistant outputs, and
generate an additional assistant query that is related to the spoken utterance based on at least part of the context of the dialog session and at least part of the assistant query, wherein generating the additional assistant query that is related to the spoken utterance comprises:
determining, based on the NLU output, an intent associated with the assistant query that is included in the spoken utterance; identifying, based on the intent associated with the assistant query at least one related intent that is related to the intent associated with the assistant query that is included in the spoken utterance; and
generating the additional assistant query that is related to the spoken utterance based on the at least one related intent; determining, based on the additional assistant query, additional assistant output that is responsive to the additional assistant query; processing, based on the additional assistant output that is responsive to the additional assistant query, the set of modified assistant outputs to generate a set of additional modified assistant outputs; and
causing a given additional modified assistant output, from the set of additional modified assistant outputs, to be provided for presentation to the user.
3. (Previously Presented) The method of claim 1, wherein processing the set of assistant outputs and the context of the dialog session to generate the set of modified assistant outputs comprises: processing, using the LLM, the set of assistant outputs and the context of the dialog session to generate one or more of the LLM outputs; and determining the set of modified assistant outputs based on one or more of the LLM outputs.
4. (Previously Presented) The method of claim 3, wherein processing the set of assistant outputs and the context of the dialog session to generate one or more of the LLM outputs using the LLM comprises: processing the set of assistant outputs and the context of the dialog session using a first set of LLM parameters, of a plurality of disparate sets of LLM parameters, to determine one or more of the LLM outputs having a first personality, of a plurality of disparate personalities, wherein the set of modified assistant outputs include one or more first personality assistant outputs that reflect the first personality.
.
2. The method of claim 1, wherein processing the set of assistant outputs and the context of the dialog session to generate the set of modified assistant outputs that each reflect the first personality and using the LLM causes the LLM to adapt the set of assistant output to a first vocabulary, that is associated with the first personality, to generate the set of modified assistant outputs.
3. The method of claim 2, wherein the first personality is disparate from a second personality, and wherein the first vocabulary, that is associated with the first personality and that is used to generate the set of modified assistant outputs, is disparate from a second vocabulary, that is associated with the second personality.
5. (Previously Presented) The method of claim 4, wherein processing the set of assistant outputs and the context of the dialog session to generate one or more of the LLM outputs using the LLM comprises: processing the set of assistant outputs and the context of the dialog session using a second set of LLM parameters, of a plurality of disparate sets of LLM parameters, to determine one or more of the LLM outputs having a second personality, of a plurality of disparate personalities, wherein the set of modified assistant outputs include one or more second personality assistant outputs that reflect the second personality, and wherein the second personality is unique from the first personality.
6. (Original) The method of claim 5, wherein the one or more first personality assistant outputs that are included in the set of modified assistant outputs and that reflect the first personality are determined using a first vocabulary associated with the first personality, wherein the one or more second personality assistant outputs that are included in the set of modified assistant outputs and that reflect the second personality are determined using a second vocabulary associated with the second personality, and wherein the second personality is unique from the first personality based on the second vocabulary being unique from the first vocabulary.
9. The method of claim 1, further comprising: as part of a subsequent dialog session between the user and the automated assistant implemented by the client device:
receiving a subsequent stream of audio data that captures a subsequent spoken utterance of the user, the subsequent stream of audio data being generated by one or more of the microphones of the client device, and
the subsequent spoken utterance including a subsequent assistant query;
determining, based on processing the subsequent stream of audio data, a subsequent set of assistant outputs, each assistant output in the subsequent set of assistant outputs being responsive to the subsequent assistant query included in the subsequent spoken utterance;
processing, using the LLM or an additional LLM, the subsequent set of assistant outputs and subsequent context of the subsequent dialog session to generate a subsequent set of modified assistant outputs that each reflect a second personality that differs from the first personality; and
causing a given subsequent modified assistant output, from among the subsequent set of modified assistant outputs, to be provided for presentation to the user.
10. The method of claim 9, wherein the subsequent set of assistant outputs and the subsequent context of the subsequent dialog session are processed using the additional LLM to generate the subsequent set of modified assistant outputs that each reflect the second personality that differs from the first personality, wherein the LLM is associated with the first personality, and
wherein the additional LLM is associated with the first personality.
4, For subsequent input , see claim 1 last portion.
5. (Previously Presented) The method of claim 4, wherein processing the set of assistant outputs and the context of the dialog session to generate one or more of the LLM outputs using the LLM comprises: processing the set of assistant outputs and the context of the dialog session using a second set of LLM parameters, of a plurality of disparate sets of LLM parameters, to determine one or more of the LLM outputs having a second personality, of a plurality of disparate personalities, wherein the set of modified assistant outputs include one or more second personality assistant outputs that reflect the second personality, and wherein the second personality is unique from the first personality.
6. (Original) The method of claim 5, wherein the one or more first personality assistant outputs that are included in the set of modified assistant outputs and that reflect the first personality are determined using a first vocabulary associated with the first personality, wherein the one or more second personality assistant outputs that are included in the set of modified assistant outputs and that reflect the second personality are determined using a second vocabulary associated with the second personality, and wherein the second personality is unique from the first personality based on the second vocabulary being unique from the first vocabulary.
7. (Previously Presented) The method of claim 5, wherein the one or more first personality assistant outputs that are included in the set of modified assistant outputs and that reflect the first personality are associated with a first set of prosodic properties utilized in providing the given additional modified assistant output for audible presentation to the user, wherein the one or more second personality assistant outputs that are included in the set of modified assistant outputs and that reflect the second personality are associated with a second set of prosodic properties utilized in providing the given additional modified assistant output for audible presentation to the user, and wherein the second personality is unique from the first personality based on the second set of prosodic properties being unique from the first set of prosodic properties.
Claim 20 (I) is a non-transitory computer-readable storage medium claim with a limitation similar to the limitation of US12148421 B2 method Claim 4 and is rejected under similar rationale.
Claim 12 (I) is a system claim with limitations similar to the limitations of US12148421 B2 method Claim 6 and is rejected under similar rationale.
Claim 13 (I) is a system claim with limitations similar to the limitations of US12148421 B2 method Claim 6 and is rejected under similar rationale.
Claim 19 (I) is a system claim with limitations similar to the limitations of US12148421 B2 method Claim 4 and is rejected under similar rationale.
Claim 4, 5, and 14, 15, is rejected on the ground of non-statutory double patenting as being unpatentable over claims 7 of US12148421 B2 in view of Aher et al. US 20210319780 A1.
4. The method of claim 1, wherein causing the given modified assistant output to be provided for presentation to the user comprises: generating, using a first set of prosodic properties that is associated with the first personality, synthesized speech audio data that captures the given modified assistant output; and causing the synthesized speech audio data that captures the given modified assistant output to be audibly rendered via one or more speakers of the client device.
7. (Previously Presented) The method of claim 5, wherein the one or more first personality assistant outputs that are included in the set of modified assistant outputs and that reflect the first personality are associated with a first set of prosodic properties utilized in providing the given additional modified assistant output for audible presentation to the user, wherein the one or more second personality assistant outputs that are included in the set of modified assistant outputs and that reflect the second personality are associated with a second set of prosodic properties utilized in providing the given additional modified assistant output for audible presentation to the user, and wherein the second personality is unique from the first personality based on the second set of prosodic properties being unique from the first set of prosodic properties.
5. The method of claim 4, wherein the first personality is disparate from a second personality, and wherein the first set of prosodic properties, that is associated with the first personality and that is used to generate the synthesized speech audio data, is disparate from a second set of prosodic properties, that is associated with the second personality.
7. (Previously Presented) The method of claim 5, wherein the one or more first personality assistant outputs that are included in the set of modified assistant outputs and that reflect the first personality are associated with a first set of prosodic properties utilized in providing the given additional modified assistant output for audible presentation to the user, wherein the one or more second personality assistant outputs that are included in the set of modified assistant outputs and that reflect the second personality are associated with a second set of prosodic properties utilized in providing the given additional modified assistant output for audible presentation to the user, and wherein the second personality is unique from the first personality based on the second set of prosodic properties being unique from the first set of prosodic properties.
US12148421 B2 does not teach synthesize speech. Aher teaches synthesized speech response.
As to claim 4, the issued patent teaches all of the limitations as in claim 7. Aher teaches (“[0081] In an embodiment, at step 608, the voice application determines predicted prosodic characteristics of the response using a model and modifies the predicted prosodic characteristics to generate the prosodic characteristics of the synthesized speech response. The model may, for example, include the results of a training model (e.g., training model 370 of FIG. 3), which may include correlations, probabilities, confidences, and other values indicative of the model output. In an embodiment, the voice application may select from among a plurality of versions of a word, each having a particular prosodic character, for the version that most matches the desired or predicted prosodic character. For example, the voice application may access a database that stores a plurality of audio files, each corresponding to a word, phrase, or grouping thereof, and may select the audio file having associated prosodic metrics that are most similar to the predicted metrics.”) by Aher et al. US 20210319780 A1
Aher is considered to the claimed invention because it relates to systems for managing responses to voice inputs, and, more particularly, systems for generating more natural speech responses to voice inputs based on prosody.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify instant application to incorporate the teachings of Aher et al. US 20210319780 A1in order to synthesize response.
One could have been motivated to do so because system will have prosody improvement. (“[0005] … To illustrate, the default synthesized speech response may lack prosody indicative of emotion, sentiment, or context, and the system's modification of the response's prosody may provide an improvement. In an embodiment, the system interpolates, normalizes, or otherwise modifies the prosodic character of the response to improve prosody.”). by Aher et al. US 20210319780 A1
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 3, 11, 12, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over VAN OS et al. US 20150382047 A1 in view of Jonnalagadda et al. US 20210201144 A1.
Regarding claims 1, VAN OS teaches:
1. A method implemented by one or more processors, the method comprising: as part of a dialog session between a user of a client device and an automated assistant implemented by the client device: FIG. 10 -16, and FIG. 19-22, [0057], [0060] [0061], [0063]. Tv assistant disclose a virtual assistant that interacts with a user of client device (i.e remote/companion device) during a dialogue session to control content (i.e. Automated assistant implemented by the client device engaging the user in a dialogue). (“[0019] FIG. 10 illustrates an exemplary process for controlling television interactions using a virtual assistant and displaying associated information using different interfaces.”) (“[0086] In some examples, virtual assistant client module 364 can selectively provide information (e.g., user data 366) stored on television set-top box 104 in response to requests from virtual assistant server 114 (or it can be used on television set-top box 104 itself in executing speech recognition and/or virtual assistant functions). Virtual assistant client module 364 can also elicit additional input from the user via a natural language dialogue or other user interfaces upon request by virtual assistant server 114. Virtual assistant client module 364 can pass the additional input to virtual assistant server 114 to help virtual assistant server 114 in intent inference and/or fulfillment of the user's intent expressed in the user request.”) (“[0126] FIG. 12 illustrates exemplary television control using a virtual assistant. FIG. 12 illustrates interface 1254, which can include a virtual assistant interface formatted as a conversational dialog between the assistant and the user. For example, interface 1254 can include assistant greeting 1256 prompting the user to make a request. Subsequently-received user speech can then be transcribed, such as transcribed user speech 1258, showing the back and forth conversation. In some examples, interface 1254 can appear on user device 102 in response to a trigger to initiate receipt of speech input (triggers such as button presses, key phrases, or the like).”) by VAN OS et al. US 20150382047 A1
receiving a stream of audio data that captures a spoken utterance of the user, the stream of audio data being generated by one or more microphones of the client device, and the spoken utterance including an assistant query; VAN OS teaches In some examples, users can interact with television set-top box 104 through user device 102, remote control 106, or interface elements integrated within television set-top box 104 (e.g., buttons, a microphone, a camera, a joystick, etc.). For example, speech input including media-related queries or commands for the virtual assistant can be received at user device 102 and/or remote control 106, and the speech input can be used to cause media-related tasks to be executed on television set-top box 104. (“[0048] This relates to systems and processes for controlling television user interactions using a virtual assistant. In one example, a virtual assistant can be used to interact with a media control device, such as a television set-top box controlling content shown on a television display. A mobile user device or a remote control with a microphone can be used to receive speech input for the virtual assistant. …”) (“[0059] … In some examples, users can interact with television set-top box 104 through user device 102, remote control 106, or interface elements integrated within television set-top box 104 (e.g., buttons, a microphone, a camera, a joystick, etc.). For example, speech input including media-related queries or commands for the virtual assistant can be received at user device 102 and/or remote control 106, and the speech input can be used to cause media-related tasks to be executed on television set-top box 104. Likewise, tactile commands for controlling media on television set-top box 104 can be received at user device 102 and/or remote control 106 (as well as from other devices not shown). The various functions of television set-top box 104 can thus be controlled in a variety of ways, giving users multiple options for controlling media content from multiple devices.”) by VAN OS et al. US 20150382047 A1
determining, based on processing the stream of audio data, a set of assistant outputs, each assistant output in the set of assistant outputs being responsive to the assistant query included in the spoken utterance; [0140]-[0142] VAN OS teaches Tv assistant discloses determining a user intent of the query and result of the query is displayed based on intent determination. (i.e.) (“[0054] FIG. 1 illustrates exemplary system 100 for controlling television user interaction using a virtual assistant. It should be understood that controlling television user interaction as discussed herein is merely one example of controlling media on one type of display technology and is used for reference, and the concepts discussed herein can be used for controlling any media content interactions generally, including on any of a variety of devices and associated displays (e.g., monitors, laptop displays, desktop computer displays, mobile user device displays, projector displays, etc.). The term “television” can thus refer to any type of display associated with any of a variety of devices. Moreover, the terms “virtual assistant,” “digital assistant,” “intelligent automated assistant,” or “automatic digital assistant” can refer to any information processing system that can interpret natural language input in spoken and/or textual form to infer user intent, and perform actions based on the inferred user intent. For example, to act on an inferred user intent, the system can perform one or more of the following: identifying a task flow with steps and parameters designed to accomplish the inferred user intent; inputting specific requirements from the inferred user intent into the task flow; executing the task flow by invoking programs, methods, services, APIs, or the like; and generating output responses to the user in an audible (e.g., spoken) and/or visual form.”) (“[0140] In some examples, a determination can be made as to whether results of a virtual assistant query should be displayed on user device 102 directly or on display 112 associated with television set-top box 104. In one example, in response to determining that the user intent of a query includes a request for information, an informational response can be displayed on user device 102. In another example, in response to determining that the user intent of a query includes a request to play media content, media content responsive to the query can be played via television set-top box 104.”) (“[0141] FIG. 15 illustrates virtual assistant interface 1254 with a conversational dialog example between a virtual assistant and a user. Assistant greeting 1256 can prompt the user to make a request. In the first query, transcribed user speech 1574 (which can also be typed or entered in other ways) includes a request for an informational answer associated with displayed media content. In particular, transcribed user speech 1574 inquires who is playing in a soccer game that may be, for example, shown on an interface on user device 102 (e.g., listed in interface 1150 of FIG. 11) or on display 112 (e.g., listed in interface 510 of FIG. 5 or playing as video 726 on display 112 of FIG. 7B). The user intent of transcribed user speech 1574 can be determined based on displayed media content. For example, the particular soccer game in question can be identified based on content shown on user device 102 or on display 112. The user intent of transcribed user speech 1574 can include obtaining an informational answer detailing the teams playing in the soccer game identified based on the displayed content. In response to determining that the user intent includes a request for an informational answer, the system can determine to display the response within interface 1254 in FIG. 15 (as opposed to on display 112). The response to the query can, in some examples, be determined based on metadata associated with the displayed content (e.g., based on a description of the soccer game in a television listing). As shown, assistant response 1576 can thus be displayed on touchscreen 246 of user device 102 in interface 1254, identifying teams Alpha and Zeta as playing in the game. Accordingly, in some examples, an informational response can be displayed within interface 1254 on user device 102 based on determining that a query includes an informational request.”) by VAN OS et al. US 20150382047 A1
causing a given modified assistant output, from among the set of modified assistant outputs, to be provided for presentation to the user. [0054], [0140]-[0142] VAN OS teaches (“[0051] Television display content can also be used as contextual input for determining user intent from speech input. For example, speech input can be received from a user, including a query associated with content shown on a television display. The user intent of the query can be determined based on the content shown on the television display as well as a viewing history of media content on the television display (e.g., disambiguating the query based on characters in a playing TV show). The results of the query can then be displayed based on the determined user intent.”) (“[0214] In addition, in any of the various examples discussed herein, various aspects can be personalized for a particular user. User data, including contacts, preferences, location, favorite media, and the like, can be used to interpret voice commands and facilitate user interaction with the various devices discussed herein. The various processes discussed herein can also be modified in various other ways according to user preferences, contacts, text, usage history, profile data, demographics, or the like. In addition, such preferences and settings can be updated over time based on user interactions (e.g., frequently uttered commands, frequently selected applications, etc.). Gathering and use of user data that is available from various sources can be used to improve the delivery to users of invitational content or any other content that may be of interest to them. The present disclosure contemplates that in some instances, this gathered data can include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data can include demographic data, location-based data, telephone numbers, email addresses, home addresses, or any other identifying information.”) by VAN OS et al. US 20150382047 A1
VAN OS does not explicitly teach LLM based personality conditioned modification of candidate output.
Jonnalagadda teaches:
processing, using a large language model (LLM), the set of assistant outputs and context of the dialog session to generate a set of modified assistant outputs that each reflect a first personality from among a plurality of disparate personalities; and FIG. 16, STEP 1630. Jonnalagadda teaches Sequence to sequence networks and transformer networks may be employed to augment the phrases in a dynamically generated message.( i.e. generate a set of modified assistant outputs ). Also, ). Each assistant includes different preferred classification and action response models, personality weights (i.e. each reflect a first personality from among a plurality of disparate personalities.) (“[0136] After language preference is determined, the system may determine what assistant, among a hierarchy of possible automated assistants, the contact is engaging with (at 1420). Each assistant includes different preferred classification and action response models, personality weights, and access permissions. …”) (“[0144] The message templates and phrase selections would likewise be modified based upon conversation lifecycle (at 1720). …”) (“[0145] Returning to FIG. 16, the template is then populated with phrase selections (at 1630). Sequence to sequence networks and transformer networks may be employed to augment the phrases in a dynamically generated message. …”) (“[0146] Specific phrases may be selected based upon weighted outcomes (success ranks). The system calculates phrase relevance scores to determine the most relevant phrases given a lead state, sending template, and message component. Some (not all) of the attributes used to describe lead state are: the client, the conversation, the objective (primary versus secondary objective), series in the conversation, and attempt number in the series, insights, target language and target variables. For each message component, the builder filters (potentially thousands of) phrases to obtain a set of maximum-relevance candidates. In some embodiments, within this set of maximum-relevance candidates, a single phrase is randomly selected to satisfy a message component. …”) (“[0147] In addition to performance-based selection, as discussed above (but not illustrated here), phrase selection may be influenced by the “personality” of the system for the given conversation. Personality of an AI assistant may not just be set, as discussed previously, but may likewise be learned using machine learning techniques that determines what personality traits are desirable …”) (“[0148] Message phrase packages are constructed to be tone, cadence, and timbre consistent throughout, and are tagged with descriptions of these traits (professional, firm, casual, friendly, etc.), using standard methods from cognitive psychology. Additionally, in some embodiments, each phrase may include a matrix of metadata that quantifies the degree a particular phrase applies to each of the traits. The system will then map these traits to the correct set of descriptions of the phrase packages and enable the correct packages. This will allow customers or consultants to more easily get exactly the right Assistant personality (or conversation personality) for their company, particular target, and conversation. This may then be compared to the identity personality profile, and the phrases which are most similar to the personality may be preferentially chosen, in combination with the phrase performance metrics. …”) by Jonnalagadda et al. US 20210201144 A1
Jonnalagadda is considered to be analogous to the claimed invention because it relates to Systems and methods for generating custom client intents in an AI driven conversation system are provided.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os to incorporate the teachings of Jonnalagadda in order to include generative adversarial networks.
One could have been motivated to do so because system can have dynamic messaging system. (“Abstract … systems and methods for model deployment in a dynamic messaging system are provided. In yet additional embodiments, systems and methods for improved functioning of a dynamic messaging system are provided. …”) by Jonnalagadda et al. US 20210201144 A1
Claim 11 is a system claim with a limitation similar to the limitation of method Claim 1 and is rejected under similar rationale. Additionally,
Regarding claims 11, VAN OS teaches:
11. A system comprising: at least one processor; and memory storing instructions that, when executed, cause the at least one processor to be operable to: VAN OS teaches (“[0064] FIG. 2 illustrates a block diagram of exemplary user device 102 according to various examples. As shown, user device 102 can include a memory interface 202, one or more processors 204, and a peripherals interface 206. …”) by VAN OS et al. US 20150382047 A1
Claim 20 is a non-transitory computer-readable storage medium claim with a limitation similar to the limitation of method Claim 1 and is rejected under similar rationale. Additionally,
Regarding claims 20, VAN OS teaches:
20. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to be operable to perform operations, the operations comprising: as part of a dialog session between a user of a client device and an automated assistant implemented by the client device: VAN OS teaches (“[0068] … . In other examples, the instructions (e.g., for performing portions or all of the various processes described herein) can be stored on a non-transitory computer-readable storage medium of server system 110, or can be divided between the non-transitory computer-readable storage medium of memory 250 and the non-transitory computer-readable storage medium of server system 110. In the context of this document, a “non-transitory computer-readable storage medium” can be any medium that can contain or store the program for use by or in connection with the instruction execution system, apparatus, or device.”) by VAN OS et al. US 20150382047 A1
Regarding Claim 2, the combination teaches the method claim 1 as identified above.
Jonnalagadda teaches:
2. The method of claim 1, wherein processing the set of assistant outputs and the context of the dialog session to generate the set of modified assistant outputs that each reflect the first personality and Jonnalagadda teaches augment the phrases, and The message templates and phrase selections would likewise be modified based upon conversation lifecycle (at 1720). the system will then map these traits to the correct set of descriptions of the phrase packages and enable the correct packages. (“[0144] The message templates and phrase selections would likewise be modified based upon conversation lifecycle (at 1720). …”) (“[0145] Returning to FIG. 16, the template is then populated with phrase selections (at 1630). Sequence to sequence networks and transformer networks may be employed to augment the phrases in a dynamically generated message. …”) (“[0148] Message phrase packages are constructed to be tone, cadence, and timbre consistent throughout, and are tagged with descriptions of these traits (professional, firm, casual, friendly, etc.), using standard methods from cognitive psychology. Additionally, in some embodiments, each phrase may include a matrix of metadata that quantifies the degree a particular phrase applies to each of the traits. The system will then map these traits to the correct set of descriptions of the phrase packages and enable the correct packages. This will allow customers or consultants to more easily get exactly the right Assistant personality (or conversation personality) for their company, particular target, and conversation. This may then be compared to the identity personality profile, and the phrases which are most similar to the personality may be preferentially chosen, in combination with the phrase performance metrics. …”) by Jonnalagadda et al. US 20210201144 A1
using the LLM causes the LLM to adapt the set of assistant output to a first vocabulary, that is associated with the first personality, to generate the set of modified assistant outputs. Jonnalagadda teaches augment the phrases, and The message templates and phrase selections would likewise be modified based upon conversation lifecycle (at 1720). (“[0144] The message templates and phrase selections would likewise be modified based upon conversation lifecycle (at 1720). …”) (“[0145] … This results in the insertion of customer/industry specific values at specific place in the outgoing messages, as well as employing different lexica or jargon for different industries or clients. …”) by Jonnalagadda et al. US 20210201144 A1
Jonnalagadda is considered to be analogous to the claimed invention because it relates to Systems and methods for generating custom client intents in an AI driven conversation system are provided.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os to incorporate the teachings of Jonnalagadda in order to include generative adversarial networks.
One could have been motivated to do so because system can have dynamic messaging system. (“Abstract … systems and methods for model deployment in a dynamic messaging system are provided. In yet additional embodiments, systems and methods for improved functioning of a dynamic messaging system are provided. …”) by Jonnalagadda et al. US 20210201144 A1
Claim 12 is a system claim with a limitation similar to the limitation of method Claim 2 and is rejected under similar rationale.
Regarding Claim 3, the combination teaches method claim 2 as identified above.
Jonnalagadda further teaches:
3. The method of claim 2, wherein the first personality is disparate from a second personality, and wherein the first vocabulary, that is associated with the first personality and that is used to generate the set of modified assistant outputs, is disparate from a second vocabulary, that is associated with the second personality. Jonnalagadda teaches the phrases which are most similar to the personality may be preferentially chosen (“[0136] After language preference is determined, the system may determine what assistant, among a hierarchy of possible automated assistants, the contact is engaging with (at 1420). Each assistant includes different preferred classification and action response models, personality weights, and access permissions. Thus, a response received by one assistant may be treated significantly differently than that of a second assistant.”) (“[0148] … This will allow customers or consultants to more easily get exactly the right Assistant personality (or conversation personality) for their company, particular target, and conversation. This may then be compared to the identity personality profile, and the phrases which are most similar to the personality may be preferentially chosen, …”) by Jonnalagadda et al. US 20210201144 A1
Jonnalagadda is considered to be analogous to the claimed invention because it relates to Systems and methods for generating custom client intents in an AI driven conversation system are provided.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os to incorporate the teachings of Jonnalagadda in order to include generative adversarial networks.
One could have been motivated to do so because system can have dynamic messaging system. (“Abstract … systems and methods for model deployment in a dynamic messaging system are provided. In yet additional embodiments, systems and methods for improved functioning of a dynamic messaging system are provided. …”) by Jonnalagadda et al. US 20210201144 A1
Claim 13 is a system claim with a limitation similar to the limitation of method Claim 3 and is rejected under similar rationale.
Claims 4, 5, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over VAN OS and Jonnalagadda in view of Brittan et al. US 6708153 B2.
Regarding Claim 4, the combination teaches method claim 1 as identified above.
Jonnalagadda teaches be tone, cadence, and timbre consistent throughout (“[0129] The populated message is output to the communication channel appropriate messaging platform (at 1130), which as previously discussed typically includes an email service, but may also include SMS services, instant messages, social networks, audio networks using telephony or speakers and microphone, or video communication devices or networks or the like. …”) (“[0148] Message phrase packages are constructed to be tone, cadence, and timbre consistent throughout, and are tagged with descriptions of these traits (professional, firm, casual, friendly, etc.), using standard methods from cognitive psychology. …”) by Jonnalagadda et al. US 20210201144 A1
The combination does not explicitly teach the way claim 4 generate synthesized speech audio data.
Brittan teaches:
4. The method of claim 1, wherein causing the given modified assistant output to be provided for presentation to the user comprises: generating, using a first set of prosodic properties that is associated with the first personality, synthesized speech audio data that captures the given modified assistant output; and causing the synthesized speech audio data that captures the given modified assistant output to be audibly rendered via one or more speakers of the client device. Brittan teaches (“(13) The way the synthesized voice sounds (which is referred to herein as the `voice personality` since the synthesized voice can be made to imitate a particular person) depends on an implicit or explicit set of characterisers 30 of the process used for voice synthesis. Thus, if the voice output is synthesised using a voie tract model, then the characterisers setting the voice personality are the parameters of the model; alternatively, if the voice output is synthesised by the concatenation of digitized voice elements such as phonemes, allophones, diphones or the like, then these elements (which may have been derived from a particular celebrity) constitute the characterisers of the voice personality.” Column 2, lines 51-62) by Brittan et al. US 6708153 B2
Brittan is considered to be analogous to the claimed invention because it relates to the setting of the voice personality of a voice service site.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os Tv assistant micropone , speaker equipped client device and Zheng’s LLM generated persona tagged textual output to incorporate the teachings of Brittan in order to synthesizing the LLM’s persona-tagged text into speech using a persona specific parameter like prosody set, audibly rendered via device speaker.
One could have been motivated to do so because voice-specific system can be achieved. (12) The foregoing arrangement offers the user the advantages of a familiar voice to guide and recommend products or services, and the resultant reassurance is of benefit both the the user and the store.” Col. 6, lines 30-33 ) by Brittan et al. US 6708153 B2
Claim 14 is a system claim with a limitation similar to the limitation of method Claim 4 and is rejected under similar rationale.
Regarding Claim 5, the combination teaches method claim 4 as identified above.
Brittan further teaches:
5. The method of claim 4, wherein the first personality is disparate from a second personality, and wherein the first set of prosodic properties, that is associated with the first personality and that is used to generate the synthesized speech audio data, is disparate from a second set of prosodic properties, that is associated with the second personality. Brittan teaches multiple distinct voice personality characteristics (VPC ) set in simultaneous use and each imparting different voice and prosody, directly disclosing disparate prosodic property sets tied to disparate persona. (“(4) The voice browser 3 includes a selector 33 for selecting between available sets of voice personality characteristics (abbreviated to `VPC` below) 30, the selected set being supplied to the text-to-speech converter 6 (FIG. 2) of the browser to condition the voice output of the page currently being processed by the browser. The selected VPC set imparts, for example, a celebrity voice personality to the page output. FIG. 6 shows four VPC sets, namely: a default VPC set 30D held by the browser and used when no other VPC set is available or desired (the default VPC set can be one specified by the user 2); a respective VPC set 30X and 30Y associated with voice page servers 4A and 4B; and an independent VPC set 30Z hosted on a third party server 31.”) COL. 5, Lines 11-24) (“(6) Thus, with reference to the FIG. 6 example, user 2 browses to voice page 40 which is served by server 4A to the voice browser 3. The head portion of page 40 includes a link line 41 identifying VPC set 30X as the one to be used (this link includes the URL of the set). VPC selector 33 recognises this reference and selects the corresponding VPC set 30X over the default VPC set 30D (see decision circle 50 in FIG. 6). VPC set 30X is then retrieved from server 4A and saved to the store 34 from where it is used by TTS converter 6 in preference to the default VPC set 30D.” col. 5, lines 33-42) (“FIG. 6 is a diagram of an embodiment of the invention in which the same voice personality is used by two different voice services.”) (“(20) Voice-based services are highly attractive because of their ease of use. However, the reaction of users to different voice output personalities can vary widely--one voice personality can encourage a certain class of users whilst alienating a different class.” Col. 3, lines 48-52) (“(24) Typically, the voice service sites operate by serving, from voice page servers, voice pages in the form of text with embedded voice markup tags to a voice browser, the voice browser interpreting these pages and carrying out speech recognition of user voice input, text to speech conversion to generate voice output, and dialog management; the voice browser being disposed between voice page servers and the user. In this case, it is the browser that manages which set of voice personality characterisers is used in rendering a particular voice page.” Col. 3, lines 65-67 & col. 4, lines 1-7) (“(27) According to a further aspect of the present invention, there is provided a server site holding sets of voice personality characterisers for imparting particular voice personalities to the presentation of the output of voice service sites, said sets having associated usage permissions regarding use of the sets in presenting voice sites.” Col. 4, lines 29-35) by Brittan et al. US 6708153 B2
Brittan is considered to be analogous to the claimed invention because it relates to the setting of the voice personality of a voice service site.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os and Zheng to incorporate the teachings of Brittan in order to differentiate personalities based on prosodic feature.
One could have been motivated to do so because voice-specific system can be achieved. (12) The foregoing arrangement offers the user the advantages of a familiar voice to guide and recommend products or services, and the resultant reassurance is of benefit both the the user and the store.” Col. 6, lines 30-33 ) by Brittan et al. US 6708153 B2
Claim 15 is a system claim with a limitation similar to the limitation of method Claim 5 and is rejected under similar rationale.
Claims 6, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over VAN OS and Jonnalagadda in view of Hou et al. US 20220398379 A1 and further in view of Perneti; US 20220391113 A1
Regarding Claim 6, the combination teaches method claim 1 as identified above.
The combination does not explicitly teach selecting the given modified assistant output..
Hou teaches:
6. The method of claim 1, further comprising: selecting the given modified assistant output, from among the set of modified assistant outputs, based on a Hou teaches the method selects one or more outputs of the first machine learning model that include one or more modified versions of the textual data that comply with a style (i.e. personality) corresponding to one or more descriptors of the target audience. (“[0017] In an embodiment, the method utilizes a second machine learning model (i.e., author characteristic classification model) and a generated text corpus, as discussed above, to generate a set of training data for a first machine learning model (i.e., style transfer model). The method can utilize the second machine learning model to identify sentences of the generated corpus with various audience attributes (i.e., named entity recognition (NER) task) of the sentences. …”) (“[0021] In an embodiment, the method identifies a target audience of textual data. The method receives a selection of one or more descriptors of a GUI by a user that indicate attributes associated with an audience of the user. In the embodiment, the method utilizes attributes of the audience to determine a style associated with the selected audience. In various embodiments of the present disclosure a style is a set of linguistic variants with specific social meanings, which can include group membership, personal attributes, or beliefs. For example, style is the particular variety of language used by different individuals and/or in different situations or settings. In this example, the vernacular, or everyday language may be used among casual friends, whereas more formal language, with respect to grammar, pronunciation or accent, and lexicon or choice of words, is often used in a cover letter and résumé and while speaking during a job interview.”) (“[0023] … The method selects one or more outputs of the first machine learning model that include one or more modified versions of the textual data that comply with a style corresponding to one or more descriptors of the target audience. For example, the method selects one or more modified sentences output by a style transfer model (e.g., first machine learning model). In this example, the one or more modified sentences can include the input textual data modified to include jargon, grammar, and phrases corresponding to a style or pattern of the target audience. Additionally, the method transmits the one or more modified versions of the textual data to a computing device of the user.”) (“[0044] At block 308, the method of program 175, generates text modification recommendations correspond to the user input. In an embodiment, program 175 inputs textual data a user provides to client device 104 into a first machine learning model, which transforms the textual data using one or more styles associated with a target audience of one or more descriptors of a set of data inputs. In this embodiment, program 175 selects one or more outputs of the first machine learning model as text modification recommendations and transmits the selected outputs to client device 104.”) by Hou et al. US 20220398379 A1
Hou is considered to be analogous to the claimed invention because it relates to generally to natural language generation particularly to tailoring textual content to a target audience.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os and Jonnalagadda to incorporate the teachings of Hou in order to include multiple machine learning model.
One could have been motivated to do so because system can with tailoring textual content to a target audience based on audience style. (“[0016] In an embodiment, a system executing the tailoring textual content to a target audience method collects textual data to generate a text corpus to train a first machine learning model (i.e., style transfer model) based on an encoder-decoder framework (e.g., Bidirectional and Auto-Regressive Transformers (BART)).., …”) by Hou et al. US 20220398379 A1
The combination does not explicitly teach a probability distribution over a sequence of words or phrases that is generated using the LLM.probability distribution over a sequence of words or phrases Perneti teaches (“[0046] In an embodiment, a language representation model, such as the Bidirectional Encoder Representations from Transformers (BERT) process is used to create a probability distribution of all the sentence embeddings …”) by Perneti; US 20220391113 A1
Perneti is considered to be analogous to the claimed invention because it discloses sentence segmentation and embedding.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os, Jonnalagadda, and Singh to incorporate the teachings of Perneti in order to include probability distribution.
One could have been motivated to do so because system can improve on text comparison. (“[0077] Embodiments thus provide a method of performing configuration drift handling that greatly improves on existing approaches of manual text comparison between two configurations. ...” page 9, Col. 1) by Perneti; US 20220391113 A1
Claim 16 is a system claim with a limitation similar to the limitation of method Claim 6 and is rejected under similar rationale.
Claims 7, 9-10, and 17, 19 are rejected under 35 U.S.C. 103 as being unpatentable over VAN OS and Jonnalagadda in view of Hou et al. US 20220398379 A1
Regarding Claim 7, the combination teaches method claim 1 as identified above.
The combination does not explicitly teach wherein the LLM is a first LLM, from among a plurality of disparate LLMs, that is specific to the first personality.
Hou teaches:
7. The method of claim 1, wherein the LLM is a first LLM, from among a plurality of disparate LLMs, that is specific to the first personality. Hou teaches multiple machine learning models and first machine learning model (i.e.first LLM) is associated with style. (“[0019] In an embodiment, the method utilizes outputs of the second and third machine learning models to train a first machine learning model (e.g., style transfer model) on a generated a set of training data. …”) (“[0017] In an embodiment, the method utilizes a second machine learning model (i.e., author characteristic classification model) and a generated text corpus, as discussed above, to generate a set of training data for a first machine learning model (i.e., style transfer model). The method can utilize the second machine learning model to identify sentences of the generated corpus with various audience attributes (i.e., named entity recognition (NER) task) of the sentences. Additionally, the method can determine a sentiment classification of the sentences of the generated corpus and/or comments of feedback data. The audience attributes can include features (e.g., descriptors) such as age demographics, occupation/educational backgrounds, etc. In this embodiment, the method can train the second machine learning model based on a sequence classification framework (e.g., Bidirectional Encoder Representations from Transformers (BERT)). For example, the method can create a training set from the 45 and annotate (e.g., label) sentences of the training set with audience attributes for target audiences for the second machine learning model. As a result, the second machine learning model outputs a set of sentences with the corresponding labels (e.g., audience attributes, sentiments, etc.) and the method utilizes the outputs of the remainder of the generated text corpus to create a labelled sentence corpus.”) (“[0044] At block 308, the method of program 175, generates text modification recommendations correspond to the user input. In an embodiment, program 175 inputs textual data a user provides to client device 104 into a first machine learning model, which transforms the textual data using one or more styles associated with a target audience of one or more descriptors of a set of data inputs. In this embodiment, program 175 selects one or more outputs of the first machine learning model as text modification recommendations and transmits the selected outputs to client device 104.”) by Hou et al. US 20220398379 A1
Hou is considered to be analogous to the claimed invention because it relates to generally to natural language generation particularly to tailoring textual content to a target audience.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os and Jonnalagadda to incorporate the teachings of Hou in order to include multiple machine learning model.
One could have been motivated to do so because system can with tailoring textual content to a target audience based on audience style. (“[0016] In an embodiment, a system executing the tailoring textual content to a target audience method collects textual data to generate a text corpus to train a first machine learning model (i.e., style transfer model) based on an encoder-decoder framework (e.g., Bidirectional and Auto-Regressive Transformers (BART)).., …”) by Hou et al. US 20220398379 A1
Claim 17 is a system claim with a limitation similar to the limitation of method Claim 7 and is rejected under similar rationale.
Regarding Claim 9, the combination teaches method claim 1 as identified above.
9. The method of claim 1, further comprising: as part of a subsequent dialog session between the user and the automated assistant implemented by the client device: VAN OS teaches (“[0086] In some examples, virtual assistant client module 364 can selectively provide information (e.g., user data 366) stored on television set-top box 104 in response to requests from virtual assistant server 114 (or it can be used on television set-top box 104 itself in executing speech recognition and/or virtual assistant functions). Virtual assistant client module 364 can also elicit additional input from the user via a natural language dialogue or other user interfaces upon request by virtual assistant server 114. Virtual assistant client module 364 can pass the additional input to virtual assistant server 114 to help virtual assistant server 114 in intent inference and/or fulfillment of the user's intent expressed in the user request.”) by VAN OS et al. US 20150382047 A1
receiving a subsequent stream of audio data that captures a subsequent spoken utterance of the user, the subsequent stream of audio data being generated by one or more of the microphones of the client device, and the subsequent spoken utterance including a subsequent assistant query; FIG. 1,10 -16, and FIG. 19-22, [0057], [0060] [0061], [0063]. VAN OS teaches a virtual assistant that interacts with a user of client device (i.e remote/companion device) during a dialogue session to control content (i.e. Automated assistant implemented by the client device engaging the user in a dialogue). (“[0008] In another example, speech input can be received from a user, and the speech input can include a query associated with content shown on a television display. A user intent of the query can be determined based on the content shown on the television display and/or a viewing history of media content. A result of the query can be displayed based on the determined user intent.”) (“[0019] … Remote control 106 can include microphone 372 for capturing audio input (e.g., speech input from a user), button(s) 374 for capturing tactile input, and transceiver 376 for facilitating communication with television set-top box 104 via remote controller 342. ...”) (“[0126] FIG. 12 illustrates exemplary television control using a virtual assistant. FIG. 12 illustrates interface 1254, which can include a virtual assistant interface formatted as a conversational dialog between the assistant and the user. For example, interface 1254 can include assistant greeting 1256 prompting the user to make a request. Subsequently-received user speech can then be transcribed, such as transcribed user speech 1258, showing the back and forth conversation. In some examples, interface 1254 can appear on user device 102 in response to a trigger to initiate receipt of speech input (triggers such as button presses, key phrases, or the like).”) (“[0249] In some examples, the one or more virtual assistant queries comprise a query about a person appearing in the media content. In other examples, the one or more virtual assistant queries comprise a query about a character appearing in the media content. In another example, the one or more virtual assistant queries comprise a query for media content associated with a person appearing in the media content. In some examples, the media content or the viewing history of media content comprise an episode of a television show, and the one or more virtual assistant queries comprise a query about another episode of the television show. In another example, the media content or the viewing history of media content comprise an episode of a television show, and the one or more virtual assistant queries comprise a request to set a reminder to watch or record a subsequent episode of the media content. In still another example, the one or more virtual assistant queries comprise a query for descriptive details of the media content. In one example, the descriptive details comprise one or more of a show title, a character list, an actor list, an episode description, a team roster, a team ranking, or a show synopsis.”) by VAN OS et al. US 20150382047 A1
determining, based on processing the subsequent stream of audio data, a subsequent set of assistant outputs, each assistant output in the subsequent set of assistant outputs being responsive to the subsequent assistant query included in the subsequent spoken utterance; FIG. 5, VAN OS teaches (“[0101] As illustrated, media content interface 510 can include (as a result of a virtual assistant query) selectable video links 512, selectable text links 514, and additional content link 513. In some examples, links can be selected by navigating focus, a cursor, or the like to a particular element and selecting it using a remote control (e.g., remote control 106). In other examples, links can be selected using voice commands to the virtual assistant (e.g., watch that soccer game, show details about the basketball game, etc.). Selectable video links 512 can include still or moving images and can be selectable to cause playback of the associated video. In one example, selectable video link 512 can include a playing video of the associated video content. In another example, selectable video link 512 can include a live feed of a television channel. For example, selectable video link 512 can include a live feed of a soccer game on a sports channel as a result of a virtual assistant query about sporting events currently on television. Selectable video link 512 can also include any other video, animation, image, or the like (e.g., a triangular play symbol). Moreover, link 512 can link to any type of media content, such as a movie, television show, sporting event, music, or the like.”) (“[0126] FIG. 12 illustrates exemplary television control using a virtual assistant. FIG. 12 illustrates interface 1254, which can include a virtual assistant interface formatted as a conversational dialog between the assistant and the user. For example, interface 1254 can include assistant greeting 1256 prompting the user to make a request. Subsequently-received user speech can then be transcribed, such as transcribed user speech 1258, showing the back and forth conversation. In some examples, interface 1254 can appear on user device 102 in response to a trigger to initiate receipt of speech input (triggers such as button presses, key phrases, or the like).”) (“[0142] The second query in interface 1254, however, includes a media request. In particular, transcribed user speech 1578 requests changing displayed media content to “the game.” The user intent of transcribed user speech 1578 can be determined based on displayed content (e.g., to identify which game the user desires), such as a game listed in interface 510 of FIG. 5, a game listed in interface 1150 of FIG. 11, a game referenced in previous queries (e.g., in transcribed user speech 1574), or the like. The user intent of transcribed user speech 1578 can thus include changing displayed content to a particular game—here, the soccer game with teams Alpha and Zeta. In one example, the game can be displayed on user device 102. In other examples, however, based on the query including a request to play media content, the game can be shown via television set-top box 104. In particular, in response to determining that the user intent includes a request to play media content, the system can determine to display the media content result via television set-top box 104 on display 112 (as opposed to within interface 1254 in FIG. 15). In some examples, a response or paraphrase confirming the virtual assistant's intended action can be shown in interface 1254 or on display 112 (e.g., “Changing to the soccer game.”) (“[0146] In the second query, however, transcribed user speech 1682 includes a request to show pictures of players of a team (e.g., pictures of “Team Alpha”). As in the examples discussed above, the user intent of transcribed user speech 1682 can be determined. The user intent of transcribed user speech 1682 can include performing a search (e.g., a web search) for pictures associated with “Team Alpha,” and displaying the resulting pictures. In response to determining that the user intent includes a request to display media that may be presented in thumbnail format, or media associated with a web search, or other non-specific media without a particular format, the system can automatically determine to display the desired media result on touchscreen 246 in interface 1254 of user device 102 (as opposed to displaying the resulting pictures on display 112 via television set-top box 104). For example, as shown, thumbnail photos 1684 can be displayed within interface 1254 on user device 102 in response to the user's query. The virtual assistant system can thus cause media of a certain format, or media that might be presented in a certain format (e.g., in a group of thumbnails), to be displayed on user device 102 by default.”) by VAN OS et al. US 20150382047 A1
The combination does not explicitly teach subsequent context to generate subsequent output.
Hou teaches:processing, using the LLM or an additional LLM, the subsequent set of assistant outputs and subsequent context of the subsequent dialog session to generate a subsequent set of modified assistant outputs that each reflect a second personality that differs from the first personality; and Hou teaches the method selects one or more outputs of the first machine learning model that include one or more modified versions of the textual data that comply with a style (i.e. personality) corresponding to one or more descriptors of the target audience. (“[0017] In an embodiment, the method utilizes a second machine learning model (i.e., author characteristic classification model) and a generated text corpus, as discussed above, to generate a set of training data for a first machine learning model (i.e., style transfer model). The method can utilize the second machine learning model to identify sentences of the generated corpus with various audience attributes (i.e., named entity recognition (NER) task) of the sentences. …”) (“[0021] In an embodiment, the method identifies a target audience of textual data. The method receives a selection of one or more descriptors of a GUI by a user that indicate attributes associated with an audience of the user. In the embodiment, the method utilizes attributes of the audience to determine a style associated with the selected audience. In various embodiments of the present disclosure a style is a set of linguistic variants with specific social meanings, which can include group membership, personal attributes, or beliefs. For example, style is the particular variety of language used by different individuals and/or in different situations or settings. In this example, the vernacular, or everyday language may be used among casual friends, whereas more formal language, with respect to grammar, pronunciation or accent, and lexicon or choice of words, is often used in a cover letter and résumé and while speaking during a job interview.”) (“[0023] … The method selects one or more outputs of the first machine learning model that include one or more modified versions of the textual data that comply with a style corresponding to one or more descriptors of the target audience. For example, the method selects one or more modified sentences output by a style transfer model (e.g., first machine learning model). In this example, the one or more modified sentences can include the input textual data modified to include jargon, grammar, and phrases corresponding to a style or pattern of the target audience. Additionally, the method transmits the one or more modified versions of the textual data to a computing device of the user.”) (“[0044] At block 308, the method of program 175, generates text modification recommendations correspond to the user input. In an embodiment, program 175 inputs textual data a user provides to client device 104 into a first machine learning model, which transforms the textual data using one or more styles associated with a target audience of one or more descriptors of a set of data inputs. In this embodiment, program 175 selects one or more outputs of the first machine learning model as text modification recommendations and transmits the selected outputs to client device 104.”) by Hou et al. US 20220398379 A1
Hou is considered to be analogous to the claimed invention because it relates to generally to natural language generation particularly to tailoring textual content to a target audience.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os and Jonnalagadda to incorporate the teachings of Hou in order to include multiple machine learning model.
One could have been motivated to do so because system can with tailoring textual content to a target audience based on audience style. (“[0016] In an embodiment, a system executing the tailoring textual content to a target audience method collects textual data to generate a text corpus to train a first machine learning model (i.e., style transfer model) based on an encoder-decoder framework (e.g., Bidirectional and Auto-Regressive Transformers (BART)).., …”) by Hou et al. US 20220398379 A1
Regarding Claim 10, the combination teaches method claim 9 as identified above.
Hou further teaches:
10. The method of claim 9, wherein the subsequent set of assistant outputs and the subsequent context of the subsequent dialog session are processed using the additional LLM to generate the subsequent set of modified assistant outputs that each reflect the second personality that differs from the first personality, wherein the LLM is associated with the first personality, and Hou teaches the method selects one or more outputs of the first machine learning model that include one or more modified versions of the textual data that comply with a style (i.e. personality) corresponding to one or more descriptors of the target audience. (“[0017] In an embodiment, the method utilizes a second machine learning model (i.e., author characteristic classification model) and a generated text corpus, as discussed above, to generate a set of training data for a first machine learning model (i.e., style transfer model). The method can utilize the second machine learning model to identify sentences of the generated corpus with various audience attributes (i.e., named entity recognition (NER) task) of the sentences. …”) (“[0021] In an embodiment, the method identifies a target audience of textual data. The method receives a selection of one or more descriptors of a GUI by a user that indicate attributes associated with an audience of the user. In the embodiment, the method utilizes attributes of the audience to determine a style associated with the selected audience. In various embodiments of the present disclosure a style is a set of linguistic variants with specific social meanings, which can include group membership, personal attributes, or beliefs. For example, style is the particular variety of language used by different individuals and/or in different situations or settings. In this example, the vernacular, or everyday language may be used among casual friends, whereas more formal language, with respect to grammar, pronunciation or accent, and lexicon or choice of words, is often used in a cover letter and résumé and while speaking during a job interview.”) (“[0023] … The method selects one or more outputs of the first machine learning model that include one or more modified versions of the textual data that comply with a style corresponding to one or more descriptors of the target audience. For example, the method selects one or more modified sentences output by a style transfer model (e.g., first machine learning model). In this example, the one or more modified sentences can include the input textual data modified to include jargon, grammar, and phrases corresponding to a style or pattern of the target audience. Additionally, the method transmits the one or more modified versions of the textual data to a computing device of the user.”) (“[0044] At block 308, the method of program 175, generates text modification recommendations correspond to the user input. In an embodiment, program 175 inputs textual data a user provides to client device 104 into a first machine learning model, which transforms the textual data using one or more styles associated with a target audience of one or more descriptors of a set of data inputs. In this embodiment, program 175 selects one or more outputs of the first machine learning model as text modification recommendations and transmits the selected outputs to client device 104.”) by Hou et al. US 20220398379 A1
Hou is considered to be analogous to the claimed invention because it relates to generally to natural language generation particularly to tailoring textual content to a target audience.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os and Jonnalagadda to incorporate the teachings of Hou in order to include multiple machine learning model.
One could have been motivated to do so because system can with tailoring textual content to a target audience based on audience style. (“[0016] In an embodiment, a system executing the tailoring textual content to a target audience method collects textual data to generate a text corpus to train a first machine learning model (i.e., style transfer model) based on an encoder-decoder framework (e.g., Bidirectional and Auto-Regressive Transformers (BART)).., …”) by Hou et al. US 20220398379 A1
Claim 19 is a system claim with a limitation similar to the limitation of method Claim 9 and is rejected under similar rationale.
Claims 8, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over VAN OS and Jonnalagadda in view of Singh et al. WO 2020226663 A1
Regarding Claim 8, the combination teaches method claim 1 as identified above.
Singh teaches:
8. The method of claim 1, wherein the first personality is defined by the user in settings of a software application. Singh teaches a user can initiate a content agent relating to setting up router, where the user has not specified the model. One or more content parameters relating to the specific router can include inherent uncertainty due to the system not knowing the router model. The content agent can initially provide responsive content relating to setting up routers in general. If the user provides a request for content that requires knowledge of the router model (such as a request for setting up a model specific feature), dialog module 112 can initiate a dialog session to prompt the user for the router model (“[0001] Humans may engage in human-to-computer dialog with interactive software applications referred to herein as "automated assistants" (also referred to as "digital agents", "chatbots", "interactive personal assistants", "intelligent personal assistants", "assistant applications", "conversational agents", etc.). For example, humans (which when they interact with automated assistants may be referred to as "users") may provide commands and/or requests to an automated assistant using spoken natural language input (i.e., utterances), which may in some cases be converted into text and then processed, and/or by providing textual (e.g., typed) natural language input. …”) (“[0032] Dialog manager 112 can additionally and/or alternatively initiate a dialog to refine the corpus of content at the request of the user. A user can subsequently initiate an additional dialog using dialog module 112 to further refine the corpus of content 116. For example, a user can initiate an additional dialog session in response to the content agent being used in rendering undesirable content. ... … For example, a user can initiate a content agent relating to setting up router, where the user has not specified the model. One or more content parameters relating to the specific router can include inherent uncertainty due to the system not knowing the router model. The content agent can initially provide responsive content relating to setting up routers in general. If the user provides a request for content that requires knowledge of the router model (such as a request for setting up a model specific feature), dialog module 112 can initiate a dialog session to prompt the user for the router model.”) by Singh et al. WO 2020226663 A1
Singh is considered to be analogous to the claimed invention because it relates to a user can interact with an automated assistant and/or other application(s) to request a variety of digital content including: video, audio, images, web pages, news articles, videos, recipes, blog posts, electronic books, memes, social media posts, and/or additional types of content.
Therefore, it would have been obvious for someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Os and Jonnalagadda to incorporate the teachings of Singh in order to include subsequent context to generate subsequent output.
One could have been motivated to do so because system can have the efficiency of user interaction with and control of the client device can therefore be improved. (“… The efficiency of user interaction with and control of the client device can therefore be improved, reducing both friction between the user and the client device responses and reducing computational overheads.”) by Singh et al. WO 2020226663 A1
Claim 18 is a system claim with a limitation similar to the limitation of method Claim 8 and is rejected under similar rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FOUZIA HYE SOLAIMAN whose telephone number is (571)270-5656. The examiner can normally be reached M-F (8-5)AM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D. Shah can be reached at (571) 270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/F.H.S./Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
10/09/2026
