DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to the submission filed February 20, 2025. Claims 1-20 are pending,
Information Disclosure Statement
The information disclosure statement (IDS) submitted on February 20, 2025; February 21, 2025 is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claims 1, 11, and 19 are directed to an electronic device, a method, and a non-transitory medium for generating emotion based synthetic speech based on received text, The claims recite limitations for:
based on receiving a reference speech, identifying an emotion corresponding to the reference speech among a plurality of emotions can be achieved by a person listening to the speech and detecting the emotion of the speaker;
obtaining a token set corresponding to the identified emotion among token sets corresponding to each of the plurality of emotions can be achieved by the person reviewing a database of features that are associated with the emotions;
inputting information on the reference speech and the obtained token set into a style encoder and obtaining style information for outputting a synthesized speech of the identified emotion can be achieved by applying natural language processing mathematical rules and principles to generate a contour or vector of the parameters needed to generate speech;
based on a text being input, inputting the text into a decoder obtained on the basis of the style information and obtaining a synthesized speech corresponding to the text can be achieved by the person, reviewing the text and creating a template/outline of how each word should be generated; and
outputting the synthesized speech corresponding to the text, utilizing the template, speaking the words in the desired emotion and/or style.
The recited limitations are directed a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of the generic device, memory. processor, and generic computer components. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
This judicial exception is not integrated into a practical application because the recited generic device, memory. processor, generic computer components and various modules amounts to no more than mere instructions to apply the exception using generic computer components. Accordingly, the elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea. The claims are not patent eligible.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as indicated with respect to integration of the abstract idea into a practical application, the additional elements of the generic device, memory. processor, and generic computer components to perform the various steps amounts to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. The claims are not patent eligible.
Dependent claims 2-9, 12-17 and 20 do not integrate the judicial exception into a practical application and do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The limitations of the dependent claims are directed to steps of organizing or manipulating emotion/style information, performing mathematical calculations, updating synthesis rules and natural language models, and generating speech reflective of emotions/styles.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Jang et al (KR 2020-0138993 – English Translation), hereinafter Jang.
Jang discloses method and apparatus for synthesizing emotional speech using emotion tokens. Regarding claim 1, Jang teaches an electronic device [para 0017 – emotion speech synthesis device]; comprising: memory storing one or more computer programs and a plurality of token sets corresponding to each of a plurality of emotions [para 0017; 0050]; and one or more processors one or more processors [para 0017; 0050] communicatively coupled to the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors [voice data; emotion database -- para 0017; 0031; 0033; 0050] individually or collectively, cause the electronic device to: based on receiving a reference speech, identify an emotion corresponding to the reference speech among the plurality of emotions [para 0017; 0019; 0031; 0033; 0050], obtain a token set corresponding to the identified emotion among the plurality of token sets stored in the memory [emotion tokens -- para 0017; 0031; 0033; 0050], input information on the reference speech and the obtained token set into a style encoder and obtain style information for outputting a synthesized speech of the identified emotion [prosody encoder (121); vocalized style --- para 0017; 0019; 0022; 0031; 0033], based on a text being input, input the text into a decoder obtained on the basis of the style information and obtain a synthesized speech corresponding to the text [prosody encoder (121); vocalized style; emotion voice decoder --- para 0017; 0019; 0022; 0031; 0033; 0036], and output the synthesized speech corresponding to the text [prosody encoder (121); vocalized style --- para 0017; 0019; 0022; 0031; 0033; 0041-0044; 0046-0048].
Regarding claim 2, Jang teaches the electronic device of claim 1, wherein the information on the reference speech includes reference embedding [embedding vectors --para 0017; 0019; 0022; 0031; 0033; 0035; 0041-0044; 0046-0048]; and wherein the style encoder is configured to: based on similarity between at least one style token included in the obtained token set and the reference embedding, output the style information including style embedding indicating a weighted sum of the at least one style token [para 0035; 0043]..
Regarding claim 3, Jang teaches the electronic device of claim 2, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: input a mel-spectrogram corresponding to the reference speech into a reference encoder and obtain the reference embedding [spectrogram -- para 0017; 0019; 0022; 0031; 0033; 0041-0044; 0046-0048], and input phonemes corresponding to the text into a text encoder and obtain text embedding.[ para 0017; 0019; 0022; 0031; 0033; 0041-0044; 0046-0048].
Regarding claim 4, Jang teaches the electronic device of claim 1, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on receiving an emotion identifier (ID), identify the emotion corresponding to the emotion ID among the plurality of emotions [emotion tokens -- para 0017; 0019; 0022; 0031; 0033; 0041-0044; 0046-0048].
Regarding claim 5, Jang teaches the electronic device of claim 1, wherein the style encoder is an unsupervised learning model that learned similarity between sample reference embedding corresponding to at least one sample reference speech among a plurality of sample reference speeches and at least one style token included in a token set corresponding to the emotion of the at least one sample reference speech [embedding vectors/similarity --para 0017; 0019; 0022; 0031; 0033; 0035; 0041-0044; 0046-0048].
Regarding claim 6, Jang teaches the electronic device of claim 5, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: obtain a language token, a speaker token, and a residual token corresponding to the at least one sample reference speech [prosody encoder (121); vocalized style --- para 0017; 0019; 0022; 0031-0039], and wherein the style encoder is an unsupervised learning model that learned similarity between the sample reference embedding corresponding to the at least one sample reference speech and the at least one style token, the language token, the speaker token, and the residual token included in the token set corresponding to the emotion of the at least one sample reference speech [embedding vectors/similarity/learning unit/attention mechanism --para 0017; 0019; 0022; 0031-0039; 0041-0044; 0046-0048].
Regarding claim 7, Jang teaches the electronic device of claim 6, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on a language of the reference speech corresponding to the language token of the at least one sample reference speech, input the at least one style token and the language token included in the token set corresponding to the emotion of the reference speech into the style encoder and obtain the style information [prosody encoder (121); vocalized style --- para 0017; 0019; 0022; 0031; 0033; 0035; 0041-0044; 0046-0048].
Regarding claim 8, Jang teaches the electronic device of claim 6, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: based on a speaker to be synthesized corresponding to the speaker token of the at least one sample reference speech, input the at least one style token and the speaker token included in the token set corresponding to the emotion of the reference speech into the encoder and obtain the style information [prosody encoder (121); vocalized style --- para 0017; 0019; 0022; 0031; 0033; 0035; 0041-0044; 0046-0048].
Regarding claim 9, Jang teaches the electronic device of claim 1, wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to: receive an uttered speech of a user; and based on the received uttered speech, fine-tune the decoder such that the synthesized speech corresponding to the text output by the decoder includes an utterance feature of the user [training phase – 0033-0039].
Regarding claim 10, Jang teaches the electronic device of claim 2, wherein the at least one style token included in the obtained token set corresponds to at least one of prosodic features of a speech [emotion tokens -- para 0017; 0019; 0022; 0031-0039; 0041-0044; 0046-0048].
Claims 11-18 and 19-20 are rejected under similar rationale as claims 1-10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Byun et al (US Patent Application Publication No. 2022/0230623) discloses generating synthetic speech having a target emotion and target rate of speech.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANGELA A ARMSTRONG whose telephone number is (571)272-7598. The examiner can normally be reached M,T,TH,F 11:30-8:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ANGELA A. ARMSTRONG
Primary Examiner
Art Unit 2659
/ANGELA A ARMSTRONG/Primary Examiner, Art Unit 2659
Prosecution Insights