DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-12 are pending. Claims 1 and 7 are independent.
Apparent priority 05/28/2024.
This action is Non-Final.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/26/2024 was filed. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3 and 7-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
The Supreme Court has long held that “[l]aws of nature, natural phenomena, and abstract ideas are not patentable.” Alice Corp. Pty. Ltd. v. CLS Bank Int’l, 134 S. Ct. 2347, 2354 (2014) (quoting Assoc. for Molecular Pathology v. Myriad Genetics, Inc., 133 S. Ct. 2107, 2116 (2013) (internal quotation marks omitted)). The “abstract ideas” category embodies the longstanding rule that an idea, by itself, is not patentable. Alice Corp., 134S. Ct. at 2355 (quoting Gottschalk v. Benson, 409 U.S. 63, 67 (1972).
In Alice, the Supreme Court sets forth an analytical “framework for distinguishing patents that claim laws of nature, natural phenomena, and abstract ideas [or mental processes ] from those that claim patent-eligible applications of those concepts.” Id. at 2355 (citing Mayo Collaborative Servs. v. Prometheus Labs., Inc., 132 S. Ct. 1289, 1296–97 (2012)). The first step in the analysis is to “determine whether the claims at issue are directed to one of those patent-ineligible concepts.” Id. If the claims are directed to a patent-ineligible concept, the second step in the analysis is to consider the elements of the claims “individually and ‘as an ordered combination’” to determine whether there are additional elements that “‘transform the nature of the claim’ into a patent-eligible application.” Id. (quoting Mayo, 132 S. Ct. at 1298, 1297). In other words, the second step is to “search for an ‘inventive concept’—i.e., an element or combination of elements that is ‘sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the [ineligible concept] itself’”. Id. (brackets in original) (quoting Mayo, 132 S. Ct. at 1294). The prohibition against patenting an abstract idea “‘cannot be circumvented by attempting to limit the use of the formula to a particular technological environment’ or adding ‘insignificant post-solution activity.’” Bilski v. Kappos, 561 U.S. 593, 610–11 (2010) (citation omitted).
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP 2106.03. Independent Claim 1 recites a speech synthesis apparatus comprising a memory configured to store language information set by a user and audio samples of a speaker selected by the user; and a processor configured to generate audio signals corresponding to input text. An apparatus is a Statutory category of invention. Independent Claim 7 recites a speech synthesis method comprising steps similar to Claim 1, and thus is a process (a series of steps or acts). A process is a statutory category of invention. Dependent claims 2-3 and 8-9 are dependent on claims 1 and 7, respectively, and therefore recite their respective statutory classes.
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim. In applying the framework set out in Alice, examiner found Applicant’s claims 1 and 5 are directed to a patent-ineligible abstract concept of producing audio signals from received a received text input based on applying a model that uses stored audio samples and language information. The steps of Applicant’s claims 1-3 and 7-9 are an abstract concept that would fall under the judicial exception of mental processes. Specifically, the claims recite the step of “receiving a speech synthesis request for input text, wherein the speech synthesis request includes language information and speaker information set by a user.” The receiving the request may involve data transferring, as characterized, for example, by a human making a request to another human, such as a voice actor, to utter speech corresponding to a text. Therefore, this step is directed to a mental process. Furthermore, the step of “generating audio signals corresponding to the input text by applying a speech synthesis model to the input text, the language information, and audio samples of the speaker information, wherein the language information is different from a language related to the audio samples.” recites steps that are directed to mental processes. Under the broadest reasonable interpretation, generating audio signals may correspond to the voice actor generating a voice corresponding to a text. The language information and the audio samples of the speaker information may simply correspond to material for the voice actor to modulate their voice and attempt to mimick the audio samples. In this way, the characterization of the voice actor attempting to utter a voice to mimick the audio samples corresponds to the applying of the “speech synthesis model”. Therefore, the recited language is directed to a mental process. The claims recite limitations that taken in combination, recite at least a series of mental processes.
Dependent claims 2-3 and 8-9 are directed to a mathematical concept by the recitation of normalizing training latent variables.
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d). Furthermore, independent Claim 1 further recites “a memory” and “a processor” as an additional element beyond the judicial exception. However, these additional elements do not amount to significantly more than the abstract idea because the additional elements constitute a generic computer environment. Alice, 134 S. Ct. at 2357. The Claims need meaningful limitations that go beyond generally linking the use of an abstract idea to a particular technological environment. Therefore, the steps are all abstract and the Claim as a whole is abstract. “[S]imply appending generic computer functionality to lend speed or efficiency to the performance of an otherwise abstract concept does not meaningfully limit claim scope for purposes of patent eligibility.” CLS Bank, 2013 U.S. App. LEXIS 9493, at *29 (citing Bancorp, 687 F.3d at 1278, and Dealertrack, Inc. v. Huber, 674 F.3d 1315, 1333-34 (Fed. Cir. 2012) (finding that the claimed computer-aided clearinghouse process is a patent-ineligible abstract idea)); SiRF Tech., Inc. v. Int'l Trade Comm'n, 601 F.3d 1319, 1333 (Fed. Cir. 2010) (“In order for the addition of a machine to impose a meaningful limit on the scope of a claim, it must play a significant part in permitting the claimed method to be performed, rather than function solely as an obvious mechanism for permitting a solution to be achieved more quickly, i.e., through the utilization of a computer for performing calculations.”). Additionally, dependent claims 2-3 and 8-9 do not provide any additional elements that integrate the judicial exception into a practical application. The claims simply describe that language information of training text is removed. The claims 3 and 9 further provide that a normalization of training latent variables is performed, which was determined to recite a mathematical concept, however, such mathematical concept is recited at such a high-level of generality that this limitation is merely a post-solution step. Therefore, this step is an insignificant extra-solution activity and does not integrate the judicial exception into a practical application. See MPEP 2106.05(g).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
At step 2A, prong two, the additional elements of the “memory” and the “processor” were found to be a generic computer environment. At Step 2B, the re-evaluation of the insignificant extra-solution activity consideration takes into account whether or not the extra-solution activity is well understood, routine, and conventional in the field. See MPEP 2106.05(g). Here, the memory and process perform purely conventional activities such that they are well understood, routine and conventional. Even when considered in combination, these additional elements represent mere instructions to apply an exception, and therefore do not provide an inventive concept. Additionally, dependent claims 2-3 and 8-9 do not add an inventive concept.
In conclusion, Examiner notes that none of recited steps in Applicant's claims 1-3 and 7-9 refer to a specific machine by reciting structural limitations of any apparatus or to any specific operations that would cause a machine to be the mechanism to perform these steps. Although the claims may be processed by a computing system having a processor, the computing system is merely a general purpose computing system. Therefore, all of the claims 1-3 and 7-9 are abstract.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3 and 7-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yang (US PG Pub 20220246136).
As per claims 1 and 7, Yang discloses:
A speech synthesis apparatus and method comprising: a memory configured to store (Yang; Fig. 17, item 1720; p. 0129) language information set by a user (Yang; p. 0101 - An indication of a reference language 1406 may be obtained. For example, the user may indicate in various ways that he wants to use the accent of the reference language 1406 in the speech to be generated. The reference language 1406 may comprise one or more reference languages, and may be the same as or different from the target language; see also p. 0051 - The language embedding selector 510 may attempt to retrieve a language embedding vector corresponding to the reference language ID 502 from a language embedding vector database 512. The language embedding vector database 512 may also be referred to as a language latent space information database, and may comprise representations of different languages' characteristics in any other form than embedding vectors. The language embedding vector database 512 may comprise multiple language embedding vectors corresponding to multiple languages respectively. The language embedding vector database 512 may be established through collecting language embedding vectors of those languages in a multilingual corpus (stored language information) during the training of the multilingual neural TTS system) and audio samples of a speaker selected by the user (Yang; p. 0100 - An indication of a target speaker 1404 may be obtained. For example, a user may indicate in various ways that he wants to use the voice of the target speaker 1404 in the speech to be generated; see also p. 0052 - the language encoder 500 may be implemented by a language embedding generator 520. The language embedding generator 520 may generate a language embedding vector corresponding to a reference language based on a corpus 504 of the reference language. For example, the corpus 504 of the reference language may be obtained, which comprises multiple speech waveforms in the reference language (stored audio samples of a speaker). Acoustic features may be extracted from speech waveforms in the corpus 504 through various conventional techniques, and provided to the language embedding generator 520); and a processor configured to (Yang; Fig. 17, item 1710; p. 0129) generate audio signals corresponding to input text by applying a speech synthesis model to the input text, the language information, and the audio samples in response to a speech synthesis request of the user (Yang; Fig. 9 & Fig. 10; p. 0033 - The neural TTS system 100 may be configured for receiving a text input 102, and generating a speech waveform 106 corresponding to the text input 102…; see also p. 0129; see also p. 0040; The cited figures and paragraphs describe how a speech waveform is generated in a target language from a received text input in a source language, using the stored language information in the language embedding vector database and the audio samples in the speaker embedding vector database and applying a speech synthesis model), wherein the language information is different from a language related to the audio samples (Yang; p. 0026-0027 - …even if a speaker's voice only appeared in one language during training (audio samples), the system may generate speech in other languages with the speaker's voice (language information). Since speakers in the multilingual corpus may cover different ages, genders, languages, etc., it is easy to register a new voice with limited data, e.g., generating high-quality speech for a new speaker with limited registration data and in any language covered in the training… language related to the audio samples different from the language information).
As per claims 2 and 8, Yang discloses: The speech synthesis apparatus and method of claims 1 and 11, wherein the speech synthesis model is trained to generate audio signals from which language information of training text is removed, wherein the audio signals include features of the training text and features of training audio signals (Yang; p. 0049 - The speaker embedding vector 404 may be formed through performing L2 normalization to the mapping's output. The speaker embedding generator 400 may be trained with a corpus set of multiple speakers, and is designed for speaker recognition that is independent of text or content. Therefore, the speaker embedding generator 400 may provide, independently from content, better estimation of the speaker embedding vector).
As per claims 3 and 9, Yang discloses: The speech synthesis apparatus and method of claims 2 and 8, wherein the speech synthesis model is configured to remove language information of the training text by normalizing training latent variables, that include the features of the training text and the features of the training audio signals, using language information of the training text (Yang; p. 0049 - The speaker embedding vector 404 may be formed through performing L2 normalization to the mapping's output. The speaker embedding generator 400 may be trained with a corpus set of multiple speakers, and is designed for speaker recognition that is independent of text or content. Therefore, the speaker embedding generator 400 may provide, independently from content, better estimation of the speaker embedding vector).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-6 and 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Casanova, E., Weber, J., Shulby, C. D., Junior, A. C., Gölge, E., & Ponti, M. A. (2022, June). Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone. In International conference on machine learning (pp. 2709-2720). PMLR. (hereinafter “Casanova”).
As per claims 4 and 10, Yang discloses: The speech synthesis apparatus and method of claims 1 and 7, wherein the speech synthesis model comprises: a language embedding module configured to transform the language information into a language embedding (Yang; Fig. 6, item 600; p. 0053 - the language embedding generator 600 may be based on a neural network for generating a language embedding vector 604 based on an acoustic feature 602); a character embedding module configured to transform the input text into character embeddings (Yang; p. 0033 - the text input 102 may be first divided into a sequence of elements, e.g., a phoneme sequence, a grapheme sequence, a character sequence, etc…; see also p. 0068 - A text embedding unit 912 in the encoder 910 may convert the text input 902 into a text embedding vector…); an encoder configured to encode the character embeddings into text feature vectors (Yang; p. 0035 - The encoder 112 may convert information contained in the text input 102 into a space that is more robust and more suitable for learning alignments with acoustic features, e.g., converting the information in the text input 102 to a text feature in the space; see also p. 0068 - The encoder 910 may output a text feature corresponding to the text input 902); a speaker encoder configured to encode the audio samples and output a speaker embedding (Yang; p. 0056 - If it is desired to generate a speech waveform in a certain target speaker's voice, the multilingual neural TTS system may obtain information 732 of the target speaker and provide a speaker embedding vector 734 corresponding to the target speaker through the speaker encoder 730. In one case, the target speaker information 732 may be a target speaker ID of the target speaker, and thus the speaker encoder 730 may employ the speaker embedding selector 310 in FIG. 3 to retrieve the speaker embedding vector 734 corresponding to the target speaker); an inverted decoder configured to output a latent variable transformed based on the latent variable, the speaker embedding, and the language embedding (Yang; Fig. 9, item 930 and Fig. 10, item 1010; p. 0074 - An acoustic feature 1002, which may correspond to the acoustic feature 908 in FIG. 9, may be input to a neural network 1010. The neural network 1010 may be based on various structures, e.g., a quasi-regressive neural network (QRNN) 1012 together with a 1×1 convolution layer 1014. Through the neural network 1010, a transformed acoustic feature (transformed latent variable) may be obtained; p. 0061 - the neural vocoder 820 may generate a speech waveform 806 based on at least one of the acoustic feature 704, the speaker embedding vector 734 and the language embedding vector 744… Fig 10 shows the speaker embedding vector and language embedding vector being combined with the transformed acoustic feature to yield a further transformed acoustic feature that is input to the neural vocoder); and an audio generator configured to generate the audio signals from the transformed latent variable (Yang; Fig. 10, item 1000; p. 0077 - The neural vocoder 1000… may generate a speech waveform 1008 based on a combination of the transformed acoustic feature, the transformed speaker embedding vector, and the transformed language embedding vector (the combination corresponds to the recited “transformed latent variable”)).
Yang however, fails to disclose a stochastic duration predictor configured to predict phoneme duration data including duration of each phoneme of the input text based on the text feature vectors and the speaker embedding; a projection module configured to generate a distribution of the text feature vectors; an alignment module configured to generate a latent variable based on the distribution of the text feature vectors and the phoneme duration data. Casanova does teach a stochastic duration predictor configured to predict phoneme duration data including duration of each phoneme of the input text based on the text feature vectors and the speaker embedding (Casanova; Pg. 3, Section 2. YourTTS Model - …
P
Z
p
distribution is predicted by the text encoder and the duration is sampled from random noise through the inverse transformation of the stochastic duration predictor and then, converted to integer. In this way, a latent variable
z
p
is sampled from the distribution
P
Z
p
…); a projection module configured to generate a distribution of the text feature vectors (Casanova; Pg. 3, Section 2. YourTTS Model - …
P
Z
p
distribution is predicted by the text encoder and the duration is sampled from random noise through the inverse transformation of the stochastic duration predictor and then, converted to integer. In this way, a latent variable
z
p
is sampled from the distribution
P
Z
p
…); an alignment module configured to generate a latent variable based on the distribution of the text feature vectors and the phoneme duration data (Casanova; See Figure 1 (b) where it is shown that the output of the stochastic duration predictor is sent to the Alignment Generation element, during the inference procedure. A further example is shown in Figure 1 (a) where a Monotonic Alignment Search is used during the training procedure; see also Pg. 3, Section 2. YourTTS Model). Therefore it would have been obvious to one of ordinary skill in the art to modify the apparatus and method of Yang to include a stochastic duration predictor configured to predict phoneme duration data including duration of each phoneme of the input text based on the text feature vectors and the speaker embedding; a projection module configured to generate a distribution of the text feature vectors; an alignment module configured to generate a latent variable based on the distribution of the text feature vectors and the phoneme duration data, as taught by Casanova, because this approach achieves promising results in a target language with a single-speaker dataset, opening possibilities for zero-shot multi speaker TTS and zero-shot voice conversion systems in low-resource languages. Further… it is possible to fine-tune the YourTTS model with less than 1 minute of speech and achieve state-of-the art results in voice similarity and with reasonable quality. This is important to allow synthesis for speakers with a very different voice or recording characteristics from those seen during training (Casanova; Abstract).
As per claims 5 and 11, Yang in view of Casanova discloses: The speech synthesis apparatus and method of claims 4 and 10, wherein the inverted decoder is configured to: condition the latent variable on the speaker embedding and the language embedding; and output the transformed latent variable based on the conditioned latent variable (Yang; p. 0061 - the speaker embedding vector 734 of the target speaker and the language embedding vector 744 of the reference language are used as global conditions of the acoustic feature predictor 710, and may also be optionally used as global conditions of the neural vocoder 820).
As per claims 6 and 12, Yang in view of Casanova discloses: The speech synthesis apparatus and method of claims 4 and 10, wherein the audio generator is configured to: condition the transformed latent variable on the speaker embedding; and generate the audio signals from the conditioned latent variable (Yang; p. 0061 - the speaker embedding vector 734 of the target speaker and the language embedding vector 744 of the reference language are used as global conditions of the acoustic feature predictor 710, and may also be optionally used as global conditions of the neural vocoder 820).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The prior art made of record and not relied upon includes:
Li (US PG Pub 20250349282) discloses machine learning models configured as zero-shot personalized text-to-speech models which comprise a feature extractor, a speaker encoder, and a text-to-speech module. The feature extractor is configured to extract acoustic features and prosodic features from new target reference speech associated with the new target speaker. The speaker encoder is configured to generate a speaker embedding corresponding to the new target speaker based on the acoustic features extracted from the new target reference speech. The text-to-speech module is configured to generate the personalized voice corresponding for the new target speaker based on the speaker embedding and the prosodic features extracted from the new target reference speech without applying the text-to-speech module on new labeled training data associated with the new target speaker (Li; Abstract).
Zhang (US PG Pub 20200380952) discloses receiving an input text sequence to be synthesized into speech in a first language and obtaining a speaker embedding, the speaker embedding specifying specific voice characteristics of a target speaker for synthesizing the input text sequence into speech that clones a voice of the target speaker. The target speaker includes a native speaker of a second language different than the first language. The method also includes generating, using a text-to-speech (TTS) model, an output audio feature representation of the input text by processing the input text sequence and the speaker embedding. The output audio feature representation includes the voice characteristics of the target speaker specified by the speaker embedding (Zhang; Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Rodrigo A Chavez whose telephone number is (571)270-0139. The examiner can normally be reached Monday - Friday 9-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 5712727602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RODRIGO A CHAVEZ/Examiner, Art Unit 2658
/RICHEMOND DORVIL/Supervisory Patent Examiner, Art Unit 2658
Prosecution Insights