DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 6/10/26 have been fully considered but they are not persuasive.
Regarding the 35 U.S.C. 103 rejection of the claims, Applicant argues that even if Meyer’s describe sentence and word end markers, the markers are not used to train a speech synthesis module to synthesize speech i.e. limitation “wherein the speech synthesis module is trained to synthesize speech of a predetermined speaker according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts, so that speech characteristics of the predetermined speaker according to a context of the predetermined speaker can be reflected in a speech synthesis.” (Arguments, pg. 6 - pg. 8, third para.). Examiner respectfully disagrees as Applicant is arguing that a reference (i.e., Meyer) should be teaching a limitation another reference (i.e. Kim) is applied to teach. Reference Kim is clearly applied to teach the argued limitation (see Office Action, 3/26/26, pg. 7-8).
Kim explicitly describes training its speech synthesizer 230 to produce speech of a target speaker by reflecting the articulatory feature, the emotion feature, and/or the prosody feature of the same target speaker (para. [0066]-[0067]), where the speech synthesizer 230 receives text, extract position information from the text and generates the speech using at least the position information extracted by an attention block of the synthesizer (fig. 3; para. [0066]), and where the position information is information regarding which location of the input text is being converted into a speech (para. [0096]), and where the attention extracts position information from the text that includes sentences (para. [0087]-[0088]), corresponding to limitation “wherein the speech synthesis module is trained to synthesize speech of a predetermined speaker according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts, so that speech characteristics of the predetermined speaker according to a context of the predetermined speaker can be reflected in a speech synthesis.”.
Applicant also argues that the Office Action’s reliance on Meyer’s to describe “a relationship between sentences” is not supported according to the way the language is described in the specification, and as such, argues that the rejection does not address limitation “wherein the relationship between sentences is regarding what kind of relationship each unit text has with a content of a previous sentence” (Arguments, pg. 8, fourth para. – pg. 9, first para.). Examiner respectfully disagrees as Meyer discloses its Markers as indicating the locations of the beginnings ([begin sentence] marker) and endings ([end sentence] marker) of sentences in received input text (para. [0049]-[0050]; para. [0067]) that includes adjacent sentences (Fig. 3A; para. [0061]), corresponding to limitation “wherein the relationship between sentences is regarding what kind of relationship each unit text has with a content of a previous sentence” since [begin sentence] and [end sentence] markers correspond to where a sentence begins or ends in relation to other sentences, consistent with Applicant’s original specification disclosure (para. [0047])
Regarding reference Kim, Applicant argues that the “position information” of Kim is not a pre-processing classification at least one of a position or a relationship, and as such argues that Kim fails to address limitation “a pre-processing module configured to classify at least one of a position within a sentence and a relationship between sentences for each of unit texts inputs” (Arguments, pg. 9, second para. – pg. 10, third para.). Examiner respectfully disagrees as Applicant is once again arguing that a reference (i.e., Kim) should be teaching a limitation another reference (i.e., Meyer) is applied to teach. Meyer is clearly applied to teach the argued limitation (see Office Action, 3/26/26, pg. 6-7).
Meyer explicitly discloses the use of markers indicating the locations lexical, syntactic and/or prosodic boundaries in text input including indicating the locations of boundaries between words, the locations of words and/or word sequences of particular text normalization types, such as dates, times, currency, addresses, natural numbers, digit sequences (para. [0049]) as well as sentence beginning and end markers (para. [0067]), corresponding to limitation “a pre-processing module configured to classify at least one of a position within a sentence and a relationship between sentences for each of unit texts inputs”.
Applicant further argues that combining the references would not teach or suggest missing limitation “a pre-processing module that classifies, for each unit text, the type of discourse relationship that unit text has with the content of a prior sentence, or a speech synthesis module trained to synthesize speech of a predetermined speaker according to such classification so that speaker-context-specific speech characteristics are reflected in synthesis” (Arguments, pg. 10, fourth para. – pg. 11, second para.).
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “a pre-processing module that classifies, for each unit text, the type of discourse relationship that unit text has with the content of a prior sentence, or a speech synthesis module trained to synthesize speech of a predetermined speaker according to such classification so that speaker-context-specific speech characteristics are reflected in synthesis”) are not recited in the rejected claims. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). As shown above, the combination of the references disclose limitations “a pre-processing module configured to classify at least one of a position within a sentence and a relationship between sentences for each of unit texts inputs” and “wherein the speech synthesis module is trained to synthesize speech of a predetermined speaker according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts, so that speech characteristics of the predetermined speaker according to a context of the predetermined speaker can be reflected in a speech synthesis.”.
Furthermore, in response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, given Meyer’s lack of a teaching of training its synthesis system, it would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Kim with the method of Meyer in arriving at the missing features of Meyer, because such combination would have resulted in generating output speech data that simulates a speech of a target speaker by reflecting the articulatory feature, the emotion feature, and/or the prosody feature of the target speaker (Kim, para. [0067]).
Regarding the dependent claims 5, 6 and 9-11 rejected with additional references Arik and Stanton, Applicant argues that Arik and Stanton fail to disclose the above argued limitations recited in independent claims 1 and 12 and as such argues that the references fail to disclose language recited in the dependent claims.
Examiner respectfully disagrees as the combination of Meyer and Kim disclose the language recited in the independent claims as provided above, and absent any specific argument as to why the cited portions of Arik and Stanton fail to disclose or suggest limitations recited in the dependent claims, Examiner maintains that the rejection of the dependent clams are also appropriate, where Applicant’ arguments regarding Arik and Stanton fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Patent
(US 12,080,270 B2)
Instant Application
(18/788,311)
1. An apparatus for synthesizing speech, which is a computing apparatus that includes one or more processors and a memory storing one or more programs executed by the one or more processors, the apparatus comprising:
a pre-processing module configured to mark a preset classification symbol on each of unit texts input; and wherein the pre-processing module includes: a classification unit configured to classify a position within a sentence and a relationship between sentences for each of unit texts input; and a classification marking unit configured to mark the classification symbol on each unit text according to the position within the sentence and the relationship between the sentences of each of the unit texts,
a speech synthesis module configured to receive each unit text marked with the classification symbol and synthesize speech uttering the unit text based on the input unit text,
wherein the relationship between sentences is regarding what kind of relationship each unit text has with a content of a previous sentence, wherein the speech synthesis module is trained to synthesize speech of a predetermined speaker according to the position within the sentence and the relationship between the sentences of each of the unit texts, so that speech characteristics of the predetermined speaker according to a context of the predetermined speaker can be reflected in a speech synthesis.
1. An apparatus for synthesizing speech, which is a computing apparatus that includes one or more processors and a memory storing one or more programs executed by the one or more processors, the apparatus comprising:
a pre-processing module configured to classify at least one of a position within a sentence and a relationship between sentences for each of unit texts input; and
a speech synthesis module configured to synthesize speech uttering the unit text based on the input unit text and at least one of the position within the sentence and the relationship between the sentences of each of the unit texts,
wherein the relationship between sentences is regarding what kind of relationship each unit text has with a content of a previous sentence, wherein the speech synthesis module is trained to synthesize speech of a predetermined speaker according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts, so that speech characteristics of the predetermined speaker according to a context of the predetermined speaker can be reflected in a speech synthesis.
Claims 1-12 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 of U.S. Patent No. US 12,080,270 B2.
Claims 1-12 of the instant application is anticipated by patent Claims 1- 12 in that claims 1-12 of the Patent contains all the limitations of Claims 1-12 of the instant application. Claims 1-12 of the instant application therefore are not patently distinct from the patent claims and as such are unpatentable for provisional obvious-type double patenting. The Instant application claims 1 and 11 are broader in every aspect than the patent claim 1, 11 and 12 and are therefore an obvious variant thereof.
Although the conflicting claims are not identical, they are not patentably distinct from each other because removing inherent and/or unnecessary limitations/step and rearranging the claims would be within the level of one of ordinary skill in the art. It is well settled that the omission of an element, e.g. “interference affected channel”, and its function is an obvious expedient if the remaining elements perform the same function as before. In re Karlson, 136 USPQ 184 (CCPA 1963). Also note Ex parte Rainu, 168 USPQ 375 (Bd. App. 1969). Omission of a reference element or step whose function is not needed would be obvious to one of ordinary skill in the art.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 1-4, 7, 8 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Meyer et al US 2011/0202344 A1 (“Meyer”) in view of Kim et al US 2020/0342852 A1 (“Kim”)
Per claim 1, Meyer discloses an apparatus for synthesizing speech, which is a computing apparatus that includes one or more processors and a memory storing one or more programs executed by the one or more processors, the apparatus comprising:
a pre-processing module configured to classify at least one of a position within a sentence and a relationship between sentences for each of unit texts input (fig. 2; fig. 3A; The example text input 150 is a plain text transcription of desired speech output 110, submitted to the TTS engine as, "Arriving at 221 Baker St. Please enjoy your visit." …, para. [0012]; Markers 254 may indicate in any suitable way the locations of various lexical, syntactic and/or prosodic boundaries and/or events that may be inferred from text input 240. For example, markers 254 may indicate the locations of boundaries between words … Markers 254 may also indicate the locations of words and/or word sequences of particular text normalization types, such as dates, times, currency, addresses, natural numbers, digit sequences and the like …, para. [0049]; For example, markers 330 include [begin sentence] and [end sentence] markers …, para. [0067]; para. [0077]-[0079]); and
a speech synthesis module configured to synthesize speech uttering the unit text based on the input unit text and at least one of the position within the sentence and the relationship between the sentences of each of the unit texts, wherein the relationship between sentences is regarding what kind of relationship each unit text has with a content of a previous sentence (Markers 254 may also indicate the locations of the beginnings and endings of sentences …, para. [0049]; Markers 254 generated from text input 240 by front-end 250 may be used by synthesis system 200 in further processing to select appropriate audio recordings 232 for rendering text input 240 as speech…., para. [0050]; For example, markers 330 include [begin sentence] and [end sentence] markers that may be derived from certain capitalizations and punctuation marks in text input 310 …, para. [0067]),
Meyer does not explicitly disclose wherein the speech synthesis module is trained to synthesize speech of a predetermined speaker according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts, so that speech characteristics of the predetermined speaker according to a context of the predetermined speaker can be reflected in a speech synthesis
However, this feature is taught by Kim (fig. 3; the attention 320 generates position information of an input text from which a speech is to be synthesized …, para. [0066]; the encoder 310 and the decoder 330 included in the speech synthesizer 230 may receive the articulatory feature, the emotion feature, and the prosody feature of the speaker. Herein, the articulatory feature, the emotion feature, and the prosody feature may be a speaker embedding vector … the encoder 310, the attention 320, and the decoder 330 included in the speech synthesizer 230 may configure a single artificial neural network text-to-speech synthesis model that simulates a speech of a target speaker by reflecting the articulatory feature, the emotion feature, and/or the prosody feature of the target speaker…. the single artificial neural network text-to-speech synthesis model configured by the speech synthesizer 230 may be trained using a sequence-to-sequence model (seq2seq) …, para. [0067]; the attention 724 included in the decoder 720 of FIG. 7 may correspond to the attention 320 of FIG. 3…., para. [0087]; the input text may include a sentence …, para. [0088]; the position information may be information regarding which location of the input text is being converted into a speech …, para. [0096])
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Kim with the method of Meyer in arriving at the missing features of Meyer, because such combination would have resulted in generating output speech data that simulates a speech of a target speaker by reflecting the articulatory feature, the emotion feature, and/or the prosody feature of the target speaker (Kim, para. [0067])
Per claim 2, Meyer in view of Kim discloses the apparatus according to claim 1,
Meyer discloses wherein the preprocessing module comprises: a classification unit configured to classify the at least one of the position within the sentence and the relationship between the sentences for each of the unit texts input (para. [0077]-[0078]); and
a classification marking unit configured to mark the classification symbol on each unit text according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts (para. [0077]-[0078]).
Per claim 3, Meyer in view of Kim discloses the apparatus according to claim 2,
Meyer discloses wherein the classification unit is configured to classify the position within the sentence with respect to a beginning part and an end part of the unit text (para. [0077]-[0078]); and
the classification marking unit is configured to mark the classification symbol at the beginning part and the end part of the unit text (para. [0077]-[0078]).
Per claim 4, Meyer in view of Kim discloses the apparatus according to claim 3,
Meyer discloses wherein the unit text is obtained by dividing a sentence into several parts (Front-end 250 may process and/or analyze text input 240 to determine the sequence of words and/or sounds represented by the text …, para. [0047]); and
the classification marking unit is configured to omit the classification symbol according to the position within the sentence of the unit text (fig. 3A; fig. 3B).
Per claim 7, Meyer in view of Kim discloses the apparatus according to claim 1,
Meyer discloses: wherein the pre- processing module includes: a classification unit configured to classify the at least one of the position within the sentence and the relationship between the sentences for each of the unit texts input (fig. 2; fig. 3A; para. [0012]; para. [0067]; para. [0077]-[0079];
and a classification marking unit configured to generate the classification mark information according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts (para. [0012]; para. [0049]; para. [0067]; para. [0077]-[0079]).
Per claim 8, Meyer in view of Kim discloses the apparatus according to claim 7, wherein the classification unit is configured to classify the position within the sentence with respect to a beginning part and an end part of the unit text (para. [0077]-[0078]); and
the classification marking unit is configured to generate the classification mark information with respect to the start part and the end part of the unit text (para. [0077]-[0078]).
Per claim 12, Meyer in view of Kim discloses a method for synthesizing speech performed on a computing apparatus that include one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising:
classifying at least one of a position within a sentence and a relationship between sentences for each of unit texts input (fig. 2; fig. 3A; The example text input 150 is a plain text transcription of desired speech output 110, submitted to the TTS engine as, "Arriving at 221 Baker St. Please enjoy your visit." …, para. [0012]; Markers 254 may indicate in any suitable way the locations of various lexical, syntactic and/or prosodic boundaries and/or events that may be inferred from text input 240. For example, markers 254 may indicate the locations of boundaries between words … Markers 254 may also indicate the locations of words and/or word sequences of particular text normalization types, such as dates, times, currency, addresses, natural numbers, digit sequences and the like …, para. [0049]; For example, markers 330 include [begin sentence] and [end sentence] markers …, para. [0067]; para. [0077]-[0079]); and
synthesizing, using a speech synthesis module, speech uttering the unit text based on the input unit text and at least one of the position within the sentence and the relationship between sentences of each of the unit texts, wherein the relationship between sentences is regarding what kind of relationship each unit text has with a content of a previous sentence (Markers 254 may also indicate the locations of the beginnings and endings of sentences …, para. [0049]; Markers 254 generated from text input 240 by front-end 250 may be used by synthesis system 200 in further processing to select appropriate audio recordings 232 for rendering text input 240 as speech…., para. [0050]; For example, markers 330 include [begin sentence] and [end sentence] markers that may be derived from certain capitalizations and punctuation marks in text input 310 …, para. [0067]),
Meyer does not explicitly disclose wherein the speech synthesis module is trained to synthesize speech of a predetermined speaker according to the at least one of the position within the sentence and the relationship between the sentences of each of the unit texts, so that speech characteristics of the predetermined speaker according to a context of the predetermined speaker can be reflected in a speech synthesis
However, this feature is taught by Kim (fig. 3; the attention 320 generates position information of an input text from which a speech is to be synthesized …, para. [0066]; the encoder 310 and the decoder 330 included in the speech synthesizer 230 may receive the articulatory feature, the emotion feature, and the prosody feature of the speaker. Herein, the articulatory feature, the emotion feature, and the prosody feature may be a speaker embedding vector … the encoder 310, the attention 320, and the decoder 330 included in the speech synthesizer 230 may configure a single artificial neural network text-to-speech synthesis model that simulates a speech of a target speaker by reflecting the articulatory feature, the emotion feature, and/or the prosody feature of the target speaker…. the single artificial neural network text-to-speech synthesis model configured by the speech synthesizer 230 may be trained using a sequence-to-sequence model (seq2seq) …, para. [0067]; the attention 724 included in the decoder 720 of FIG. 7 may correspond to the attention 320 of FIG. 3…., para. [0087]; the input text may include a sentence …, para. [0088]; the position information may be information regarding which location of the input text is being converted into a speech …, para. [0096])
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Kim with the method of Meyer in arriving at the missing features of Meyer, because such combination would have resulted in generating output speech data that simulates a speech of a target speaker by reflecting the articulatory feature, the emotion feature, and/or the prosody feature of the target speaker (Kim, para. [0067])
2. Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Meyer in view of Kim as applied to claim 2 above, and further in view of Arik et al US 2019/0122651 A1 (“Arik”)
Per claim 5, Meyer in view of Kim discloses the apparatus according to claim 2,
Kim discloses wherein the speech synthesis module includes: an encoder configured to receive each of the unit texts and generate an embedding vector for each of the unit texts received (para. [0088]-[0089])
Meyer in view of Kim does not explicitly disclose wherein the speech synthesis module includes: an encoder configured to receive each of unit texts marked with the classification symbol and generate an embedding vector for each of the unit texts received or a decoder configured to synthesize speech for each unit text based on the embedding vector for each of the unit texts
However, these features are taught by Arik:
wherein the speech synthesis module includes: an encoder configured to receive each of unit texts marked with the classification symbol and generate an embedding vector for each of the unit texts received (para. [0040]; an input text in converted (205) into trainable embedding representations using an embedding model, such as text embedding model 110 …, para. [0043]; Text Preprocessing, para. [0045]; Replace spaces between words with special separator characters which indicate the duration of pauses …, para. [0050]; a phoneme-only model requires a preprocessing step to convert words to their phoneme representations …, para. [0054]; the text embedding model 110 may comprise a phoneme-only model …, para. [0055]) and
a decoder configured to synthesize speech for each unit text based on the embedding vector for each of the unit texts (a Deep Voice 3 architecture 100 uses residual convolutional layers in an encoder 105 to encode text into per-timestep key and value vectors 120 for an attention-based decoder 130. In one or more embodiments, the decoder 130 uses these to predict the mel-scale log magnitude spectrograms 142 that correspond to the output audio …, para. [0040])
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Arik with the method of Meyer in arriving at the missing features of Meyer, because such combination would have resulted in providing speech with better quality and naturalness (Arik, Abstract; para. [0063]).
Per claim 6, Meyer in view of Kim and Arik discloses the apparatus according to claim 5,
Arik discloses: an attention unit configured to receive an embedding vector for each unit text from the encoder and generate an attention map by determining an attention weight for each unit text based on the embedding vector (para. [0043]; para. [0061]),
wherein the decoder is configured to receive an embedding vector and an attention map for each unit text from the attention unit, and synthesize speech for each unit text based on the embedding vector and the attention map (The attention value vectors may be computed from attention key vectors and text embeddings. to jointly consider the local information in he and the long-term context information in hk. The key vectors hk are used by each attention block to compute attention weights …, para. [0061]).
3. Claims 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Meyer in view of Kim as applied to claim 2 above, and further in view of Stanton et al US 2021/0035551 A1 (“Stanton”)
Per claim 9, Meyer in view of Kim discloses the apparatus according to claim 7,
Kim discloses wherein the speech synthesis module includes: a first encoder configured to receive each of the unit texts and generate a first embedding vector for each of the unit texts (para. [0088]-[0089]);
Meyer in view of Kim does not explicitly disclose a second encoder configured to receive classification mark information for each of the unit texts and generate a second embedding vector for the classification mark information, a combining unit configured to generate a combination vector by combining the first embedding vector and the second embedding vector or a decoder configured to synthesize speech for each of the unit texts based on the combination vector.
However, these features are taught by Stanton:
a second encoder configured to receive classification mark information for each of the unit texts and generate a second embedding vector for the classification mark information (The context features may include … or speech tags (e.g., noun, verb, adjective, etc.) …, para. [0037]; Context Features, Style Embedding, fig. 6B; para. [0072]; para. [0080]);
a combining unit configured to generate a combination vector by combining the first embedding vector and the second embedding vector (para. [0037]; Input Text Sequence, Text Encoder, Context Features, Style Embedding, Concatenator, fig. 6B; para. [0072]; para. [0080]); and
a decoder configured to synthesize speech for each of the unit texts based on the combination vector (para. [0037]; Input Text Sequence, Text Encoder, Context Features, Style Embedding, Concatenator, fig. 6B; para. [0072]; para. [0080]); and a decoder configured to synthesize speech for each of the unit texts based on the combination vector (Decoder, fig. 6B; provides the concatenation to the decoder 658 of the TTS model 650 for conversion into synthesized speech …, para. [0072])
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Stanton with the method of Meyer in view of Kim in arriving at the missing features of Meyer in view of Kim, because such combination would have resulted in synthesizing expressive speech having a natural sounding prosody/style based on the input text and the context features (Stanton, para. [0088]).
Per claim 10, Meyer in view of Kim and Stanton discloses the apparatus according to claim 9,
Meyer discloses wherein the classification mark information is indexed according to the at least one of the position within the sentence and the relationship between the sentences (para. [0012]; para. [0049]; para. [0077]-[0078]); and
Stanton suggests the second encoder is configured to generate a one-hot encoding vector for the classification mark information of each of the unit texts, and generate the second embedding vector by embedding the one-hot encoding vector (Each character in the sequence of characters can be represented as a one-hot vector and embedded into a continuous vector. That is, the subsystem 102 can represent each character in the sequence as a one-hot vector and then generate an embedding, i.e., a vector or other ordered collection of numeric values, of the character before providing the sequence as input to the seq2seq network 106, para. [0041]; the encoder 652, the attention module 656, and the decoder 658 may collectively correspond to the seq2seq recurrent neural network 106 …, para. [0080], context embedding 612 provided to seq2seq network 106/model 650 as suggesting limitation).
Per claim 11, Meyer in view of Kim and Stanton discloses the apparatus according to claim 9,
Stanton suggests: an attention unit configured to receive the combination vector for each unit text from the combining unit, and generate an attention map by determining an attention weight for each unit text received (fig. 6B; element 656),
wherein the decoder is configured to receive a combination vector and an attention map for each unit text from the attention unit, and synthesize speech for each unit text based on the combination vector and the attention map (fig. 6B, elements 656,658; the attention module 512 outputs a set of combination weights 516, 516a-n that represent the contribution of each style token 514 to the encoded prosody embedding PE 450. The attention module 512 may determine the combination weights 516 by normalizing the style tokens 514 via a softmax activation …, para. [0068]; the attention module 656 is configured to convert the concatenation 655 to a fixed-length context vector 657 for each output step of the decoder 658 to produce the output audio signal 670 …, para. [0080], attention modules as calculating weights/map for input, attention module 656 as providing output to decoder 658 as suggesting limitation).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO 892 form.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUJIMI A ADESANYA whose telephone number is (571)270-3307. The examiner can normally be reached Monday-Friday 8:30-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUJIMI A ADESANYA/Primary Examiner, Art Unit 2658
Prosecution Insights