DETAILED ACTION
This communication is in response to the Amendments and Arguments filed on 02/06/2026.
Claims 1-20 are pending and have been examined.
All previous objections/rejections not mentioned in this Office Action have been withdrawn by the examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 02/06/2026 regarding the 101 rejection have been fully considered but they are not persuasive.
Applicant asserts on pgs 13-14 that the claims not organizing human activity, as the category is primarily directed towards economic or commercial practices, as well as managing human relationships. The Examiner respectfully notes that the claims are rejected under the –Mental Processes—grouping of abstract ideas – not organizing human activity. Therefore, the arguments presented are moot. However, in the interest of responding to the general assertion that the claims are patent eligible, the claims, as recited, can be interpreted as a human reading text aloud and determining how they said the words, including prosody, timing, and emotional context, looking at text including alternate phrasing, determining how to say the new text aloud based on how they spoke the previous text, and speaking the entirety of the text including the original text and the alternate phrasing using the identified prosody, timing, and emotional context. The recitation of a processor reads to a generic computer component, which is not sufficient to amount to significantly more than the judicial exception.
Hence, Applicant’s arguments are not persuasive.
Applicant’s arguments with respect to claim(s) 1, 14, and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Please see the updated mappings citing the new art of Ingel to teach the amended claim limitations.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim(s) 1, 14, and 20, the limitation(s) of obtaining, obtaining, obtaining, and generating, as drafted, are processes that, under broadest reasonable interpretation, covers performance of the limitation in the mind and/or with pen and paper but for the recitation of generic computer components. More specifically, the mental process of a human saying a specific phrase aloud and having text written down that includes both what they said and an alternate phrasing to say, determining how they spoke the parts that should remain the same, using a learned understanding of human language, including context of the alternate phrasing, to determine how the alternate words should be spoken, and saying out loud the alternate phrasing with the appropriate prosody. The neural network reads to rules for how to understand and produce written and spoken language. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind and/or with pen and paper but for the recitation of generic computer components, then it falls within the --Mental Processes-- grouping of abstract ideas. Accordingly, the claim(s) recite(s) an abstract idea.
This judicial exception is not integrated into a practical application because the recitation of a processor in claim 1, a device, processor, and memory in claim 14, and a storage medium and processor in claim 20, reads to generalized computer components, based upon the claim interpretation wherein the structure is interpreted using [00321-343] in the specification. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim(s) is/are directed to an abstract idea.
The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using generalized computer components to obtain, obtain, obtain, and generate, amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim(s) is/are not patent eligible.
With respect to claim(s) 2 and 15, the claim(s) recite(s) obtaining a position, and concatenating speech, which reads on a human determining where in the transcript the new words are supposed to be, and saying out loud the entire phrase with the new words, using the same prosody for the unchanged parts and using the determined prosody for the new parts. No additional limitations are present.
With respect to claim(s) 3 and 16, the claim(s) recite(s) obtaining and obtaining, which reads on a human evaluating only a section of what was said and using that section to make determinations. No additional limitations are present.
With respect to claim(s) 4 and 17, the claim(s) recite(s) where the frame is in relation to the text, which reads on a human using sections of speech and text that are next to each other. No additional limitations are present.
With respect to claim(s) 5 and 18, the claim(s) recite(s) obtaining a speech feature using particular information, which reads on a human using an understanding of human language and specific information to determine the prosody for a new text phrase. No additional limitations are present.
With respect to claim(s) 6, 7, and 19, the claim(s) recite(s) obtaining a first vector and (claims 6 and 19) obtaining a speech feature, which reads on a human using a specific set of rules to write a numerical descriptor of prosody and using the numerical descriptor and a second set of rules to determine prosody for the new phrase. The encoder and decoder read to specific sets of rules for how to use specific input information to produce a specific result.
With respect to claim(s) 8 and 9, the claim(s) recite(s) predicting durations, correcting a duration, and obtaining a speech feature, which reads on a human determining how long it should and/or did take to say specific sounds and altering the length of time, and using the corrected length of time with a numerical descriptor and previously determined prosody to determine a prosody for the new phrase. The recitation of a prediction network reads to a human learning how to estimate the length of time it takes to say specific sounds.
With respect to claim(s) 10, the claim(s) recite(s) decoding in a forward or reverse order, which reads on a human evaluating information in a specific direction. No additional limitations are present.
With respect to claim(s) 11, the claim(s) recite(s) decoding vectors in specific orders to obtain speech features, which reads on a human evaluating information in sections in a specific direction to determine a prosody for a new phrase. No additional limitations are present.
With respect to claim(s) 12, the claim(s) recite(s) concatenating speech features, which reads on a human determining prosody based on putting specific sounds right next to each other. No additional limitations are present.
With respect to claim(s) 13, the claim(s) recite(s) determining speech features, and concatenating speech features, which reads on a human determining prosody for different sections of text using a specific process and information, and determining prosody based on putting specific sounds right next to each other. No additional limitations are present.
These claims further do not remedy the judicial exception being integrated into a practical application and further fail to include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 14-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jin et al. (“VoCo: Text-based Insertion and Replacement in Audio Narration”, ACM, 2017), as found in the IDS, hereinafter Jin, in view of Ingel et al. (U.S. PG Pub No. 2020/0169591), hereinafter Ingel.
Regarding claims 1, 14, and 20, Jin teaches
(claim 1) A speech processing method (method (Abstract)), comprising:
(claim 14) A speech processing device, comprising: a processor, and a memory coupled to the processor to store instructions, which when executed by the processor, cause the speech processing device to perform operations, the operations comprising (a method running on a system with a computer program where data is retained in a database (Abstract, Sec. 5.2)):
(claim 20) A non-transitory machine-readable storage medium having instructions stored therein, which when executed by a processor ,cause the processor to perform operations, the operations comprising (a method running on a system with a computer program where data is retained in a database (Abstract, Sec. 5.2)):
obtaining, by a processor, an original speech and a second text other than a first text in a target text, both the target text and an original text corresponding to the original speech comprise the first text, and a speech in the original speech corresponding to the first text is a non-edited speech (input is an existing recording coupled with a transcript, i.e. obtaining an original speech…first text in a target text…an original text corresponding to the original speech comprise the first text and a speech in the original speech corresponding to the first text is a non-edited speech, as well as the text and location of the word to be inserted, such as a replacement of a misspoken word or inserting an adjective for emphasis, i.e. obtaining…a second text other than a first text in a target text…both the target text and an original text corresponding to the original speech comprise the first text (Intro para 2-3, Sec. 3 intro and 3.1), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2));
obtaining, by the processor a first speech feature based on the non-edited speech (for target voices and transcripts, i.e. based on the non-edited speech, per-frame MFCC and F0 feature vectors are extracted, i.e. obtain a first speech feature (Sec. 4 intro, Sec. 4.1 para 1-3), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2));
obtaining, by the processor based on the first speech feature and the second text …, a second speech feature corresponding to the second text, …(the query frame of the source voice generated from the text of the new word, i.e. based on the…second text, and snippets of the target voice that sound similar to the query are determined by comparing MFCCs and feature vectors, i.e. based on the first speech feature, to obtain a matching unit, i.e. obtaining…a second speech feature corresponding to the second text (Sec. 4 intro and Sec 4.1), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)); and
generating, by the processor based on the second speech feature, [[a]] the target edited speech corresponding to the second text (the new word, i.e. corresponding to the second text, is synthesized into audio using a source voice, converted into the sound of the target speaker using the matching unit information, i.e. based on the second speech feature, and inserted into the given context, i.e. generating…a target edited speech (Sec. 4 intro and Sec 4.1), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)).
While Jin provides finding matching units of audio to synthesize the new word, Jin does not specifically teach the use of a neural network, and thus does not teach
obtaining,…based on the first speech feature and the second text by using a neural network, a second speech feature corresponding to the second text, wherein the second speech feature of the second text generated by using the first speech feature of a context of the second text.
Ingel, however, teaches obtaining,…based on the first speech feature and the second text by using a neural network, a second speech feature corresponding to the second text, wherein the second speech feature of the second text generated by using the first speech feature of a context of the second text (an artificial neural network is used to analyze the source audio data to extract components, i.e. obtaining…based on the first speech feature…by using a neural network, including characteristics of the speech included in the source audio data, such as timbre prosody, rhythm, tempo, accent, emotional state, etc., i.e. using the first speech feature of a context, where the original text from the source audio data may be transformed or edited, i.e. second text, and an artificial NN may be used to generate audio data corresponding to desired speech characteristics from transformed text, such as maintaining the manner in which an original sentence was said, i.e. obtaining based on…the second text a second speech feature corresponding to the second text wherein the second speech feature of the second text generated by using the first speech feature of a context of the second text [0133-4],[0140-2],[0154],[0162],[0409]).
Jin and Ingel are analogous art because they are from a similar field of endeavor in text-based speech editing. Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the finding matching units of audio to synthesize the new word teachings of Jin with the use of neural networks to generate new audio data using source speech characteristics for edited text as taught by Ingel. It would have been obvious to combine the references to enable improved artificial voice generation that sound as the individuals speak in the original audio stream (Ingel [0003-4]).
Regarding claims 2 and 15, Jin in view of Ingel teaches claims 1 and 14, and Jin further teaches
obtaining, by the processor, a position of the second text in the target text (the user selects an audio recording and its corresponding transcript, and the user then inserts and/or replaces words via typing at a specific location, i.e. obtaining a position Figs. 1, 2,(Intro para 3, Sec. 3 intro, Sec. 3.1), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)); and
concatenating, by the processor based on the position, the target edited speech and the non-edited speech to obtain a target speech corresponding to the target text (the synthesized word, i.e. target edited speech, is inserted into the audio, i.e. the non-edited speech, at the given location, i.e. based on the position, using concatenation of the audio snippets at their stitch points (Sec. 4 intro, Sec. 4.1), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)).
Regarding claims 3 and 16, Jin in view of Ingel teaches claims 1 and 14, and Jin further teaches
obtaining, by the processor at least one speech frame in the non-edited speech (the target audio is segmented into frames (Sec. 4 intro, Sec. 4.1 para 1-3), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)); and
obtaining, by the processor the first speech feature based on the at least one speech frame, wherein the first speech feature represents a feature of the at least one speech frame, and the first speech feature is a feature vector or a sequence (for target voices and transcripts, per-frame, i.e. based on the at least one speech frame, MFCC and F0 feature vectors are extracted, i.e. obtaining the first speech feature wherein the first speech feature represents a feature of the at least one speech frame and the first speech feature is a feature vector or a sequence (Sec. 4 intro, Sec. 4.1 para 1-3), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)).
Regarding claims 4 and 17, Jin in view of Ingel teaches claims 3 and 16, and Jin further teaches
a text corresponding to the at least one speech frame is in the first text adjacent to the second text (target voices aligned with the transcripts are divided into frames, i.e. a text corresponding to the at least one speech frame is in the first text, where the context frames correspond to the original text adjacent to the text being edited, i.e. adjacent to the second text Figs. 1, 2, 4, and 5,(Sec. 3.1, Sec. 4 intro, Sec. 4.1 para 1-3)).
Regarding claims 5 and 18, Jin in view of Ingel teaches claims 1 and 14, and Jin further teaches
obtaining, by the processor based on the first speech feature, the target text, and annotation information by using the neural network, the second speech feature corresponding to the second text, wherein the annotation information annotates the second text in the target text (for source and target voices and transcripts, per-frame MFCC and F0 feature vectors are extracted, i.e. based on the first speech feature the target text, where each frame is assigned a phoneme label, i.e. annotation information annotates the second text in the target text, and a query frame of the source voice is generated from the text of the new word, i.e. corresponding to the second text, and snippets of the target voice that sound similar to the query are determined by comparing MFCCs and feature vectors, to obtain a matching unit, i.e. obtaining…the second speech feature corresponding to the second text (Sec. 4 intro, Sec. 4.1 para 1-3), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)).
Where Ingel teaches that neural networks are used to generate speech using characteristics of source audio [0133-4],[0140-2],[0154],[0162],[0409].
And where the motivation to combine is the same as previously presented.
Claim(s) 6-8, 10, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jin, in view of Ingel, and further in view of Arik et al. (U.S. PG Pub No. 2019/0122651), hereinafter Arik.
Regarding claims 6 and 19, Jin in view of Ingel teaches claims 1 and 14.
While Jin in view of Ingel provides using a neural network, Jin in view of Ingel does not specifically teach an encoder that produces a vector corresponding to text and a decoder that produces a speech feature based on a vector, and thus does not teach
the neural network comprises an encoder and a decoder,…
obtaining, by the processor, based on the second text by using the encoder, a first vector corresponding to the second text; and
obtaining the second speech feature based on the first vector and the first speech feature by using the decoder.
Arik, however, teaches the neural network comprises an encoder and a decoder (an attention-based sequence-to-sequence model including an encoder and decoder Fig. 1,[0032],[0035-9]),
obtaining, by the processor, based on the second text by using the encoder, a first vector corresponding to the second text (the encoder begins with an embedding layer that converts characters, i.e. based on the second text by using the encoder, into vector representations, i.e. obtaining…a first vector corresponding to the second text [0037],[0043],[0061], where a processor causes the steps to be performed, i.e. by the processor [0105]); and
obtaining the second speech feature based on the first vector and the first speech feature by using the decoder (the decoder may use, i.e. by using the decoder, speaker embeddings associated with a speaker, i.e. first speech feature, as well as the attention key and value vector representations converted from the embedding representations, i.e. based on the first vector, to predict the mel-band log magnitude spectrograms that correspond to the output audio, i.e. obtaining the second speech feature [0038],[0043],[0053],[0063],[0072],[0110]).
Where Jin specifically teaches the text to be inserted and synthesis of the new words in the voice of the recording Sec. 3.1.
Jin, Ingel, and Arik are analogous art because they are from a similar field of endeavor in text to speech synthesis. Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the using a neural network teachings of Jin, as modified by Ingel, with the use of an encoder to generate vector representations of input text and a decoder to generate spectrograms from the vectors as taught by Arik. It would have been obvious to combine the references to enable maintaining naturalness in speech synthesis with ten times faster training and mitigation of common error modes (Arik Abstract).
Regarding claim 7, Jin in view of Ingel and Arik teaches claim 6, and Arik further teaches
obtaining, by the processor, the first vector based on the target text by using the encoder (the encoder begins with an embedding layer that converts characters, i.e. based on the target text by using the encoder, into vector representations, i.e. obtaining…a first vector corresponding to the second text [0037],[0043],[0061], where a processor causes the steps to be performed, i.e. by the processor [0105]).
Where Jin specifically teaches the text to be inserted and synthesis of the new words in the voice of the recording Sec. 3.1.
And where the motivation to combine is the same as previously presented.
Regarding claim 8, Jin in view of Ingel and Arik teaches claim 6, and Jin further teaches
predicting, by the processor, first duration and second duration based on the target text by using a prediction network, wherein the first duration is phoneme duration corresponding to the first text in the target text, and the second duration is phoneme duration corresponding to the second text in the target text (the duration of phonemes are determined, such as through the timing features of snippets in the target speaker’s speech recording and transcript, i.e. predicting first duration…based on the target text…the first duration is phoneme duration corresponding to the first text in the target text, and matching cost defined as the distance between the query exemplar and the matching exemplar includes duration of phonemes of the query and the candidate segments, i.e. predicting…second duration based on the target text…the second duration is phoneme duration corresponding to the second text in the target text (Sec. 3.3, Sec. 4 para 1-5, Sec. 4.1, Sec. 4.4), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)); and
correcting, by the processor, the second duration based on the first duration and third duration, to obtain first corrected duration, wherein the third duration is phoneme duration of the first text in the original speech (the duration of phonemes are determined, such as through the timing features of snippets in the target speaker’s speech recording and transcript, i.e. first duration and third duration…the third duration is phoneme duration of the first text in the original speech, and matching cost defined as the distance between the query phoneme and the matching exemplar includes duration of phonemes of the query and the candidate segments, i.e. correcting the second duration…to obtain first corrected duration (Sec. 3.3, Sec. 4 para 1-5, Sec. 4.1, Sec. 4.4), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)); and
the obtaining the second speech feature based on the first vector and the first speech feature by using the decoder comprises:
obtaining, by the processor, the second speech feature based on the first vector, the first speech feature, and the first corrected duration by using the decoder (the query frame of the source voice generated from the text of the new word being converted to phoneme sequences, i.e. based on the first vector, and snippets of the target voice that sound similar to the query are determined by comparing MFCCs and feature vectors, i.e. based on…the first speech feature, and using phoneme duration as part of the matching cost, i.e. based on…the first corrected duration, to obtain a matching unit, i.e. obtaining…the second speech feature (Sec. 3.3, Sec. 4 para 1-5, Sec 4.1, Sec. 4.4), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)).
Where Ingel teaches the identification of duration of speech sounds, tempo, and articulation rate by an artificial neural network, i.e. prediction network [0142-4].
Where Arik teaches the text being converted specifically into vector representations and the use of a decoder [0037-8],[0043],[0053],[0061],[0063],[0072], [0110].
And where the motivation to combine is the same as previously presented.
Regarding claim 10, Jin in view of Ingel and Arik teaches claim 6, and Arik further teaches
decoding, by the processor based on the decoder and the first speech feature, the first vector from the target text in a forward order or a reverse order to obtain the second speech feature (the decoder may use, i.e. decoding based on the decoder, speaker embeddings associated with a speaker, i.e. first speech feature, as well as the attention key and value vector representations converted from the embedding representations, i.e. decoding…the first vector from the target text, to predict the mel-band log magnitude spectrograms that correspond to the output audio, i.e. obtain the second speech feature, where the decoder generates audio in an autoregressive manner by predicting a group of r future audio frames conditioned on the past audio frames, i.e. in a forward order [0038],[0043],[0053],[0063],[0072],[0110], where a processor causes the steps to be performed, i.e. by the processor [0105]).
Where the motivation to combine is the same as previously presented.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jin, in view of Ingel, in view of Arik, and further in view of Morrison et al. (“CONTEXT-AWARE PROSODY CORRECTION FOR TEXT-BASED SPEECH EDITING”, ICASSP 2021, 13 May 2021), as found in the IDS, hereinafter Morrison.
Regarding claim 9, Jin in view of Ingel and Arik teaches claim 6.
While Jin in view of Ingel and Arik provides identifying different timing of speech and adjusting dubbed audio to match the length of the original audio, Jin in view of Ingel and Arik does not specifically teach predicting durations of all phonemes corresponding to second text and correcting the fourth duration based on the speaking speed, and using the corrected duration with other inputs to obtain a speech feature, and thus does not teach
predicting, by the processor, fourth duration based on the second text by using a prediction network, wherein the fourth duration is total duration of all phonemes corresponding to the second text;
obtaining, by the processor, a speaking speed of the original speech; and
correcting, by the processor, the fourth duration based on the speaking speed to obtain second corrected duration; and
the obtaining the second speech feature based on the first vector and the first speech feature by using the decoder comprises:
obtaining, by the processor, the second speech feature based on the first vector, the first speech feature, and the second corrected duration by using the decoder.
Morrison, however, teaches predicting, by the processor, fourth duration based on the second text by using a prediction network, wherein the fourth duration is total duration of all phonemes corresponding to the second text (a duration generator, i.e. prediction network, predicts the phoneme duration information for all words of an edit region by generating a duration value in seconds for each phoneme in the input text (Sec. 2.1));
obtaining, by the processor, a speaking speed of the original speech (a number of frames occupied by the original phoneme is determined (Sec. 2.3)); and
correcting, by the processor, the fourth duration based on the speaking speed to obtain second corrected duration (a value for time-stretching rate, obtain second corrected duration, is determined based on the ratio between the original and generated phoneme durations, i.e. correcting the fourth duration based on the speaking speed (Sec. 2.3)); and
the obtaining the second speech feature based on the first vector and the first speech feature by using the decoder comprises:
obtaining, by the processor, the second speech feature based on the first vector, the first speech feature, and the second corrected duration by using the decoder (the decoder may use, i.e. by using the decoder, speaker embeddings associated with a speaker, i.e. first speech feature, as well as the attention key and value vector representations converted from the embedding representations, i.e. based on the first vector, and duration predictions as the key and value vectors are per timestep, i.e. second corrected duration, to predict the mel-band log magnitude spectrograms that correspond to the output audio, i.e. obtaining the second speech feature [0032],[0038],[0043],[0050],[0053],[0063],[0072],[0110]).
Where Morrison specifically teaches the use of a time-stretching rate (Sec. 2.3).
Jin, Ingel, Arik, and Morrison are analogous art because they are from a similar field of endeavor in text-based speech editing. Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the identifying different timing of speech and adjusting dubbed audio to match the length of the original audio teachings of Jin, as modified by Ingel and Arik, with the use of neural networks to predict and adjust phoneme durations for edited text as taught by Morrison. It would have been obvious to combine the references to enable generation of more natural sounding text-based speech editing (Morrison Abstract).
Claim(s) 11-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jin, in view of Ingel, in view of Arik, and further in view of Zheng et al. (“Forward–Backward Decoding Sequence for Regularizing End-to-End TTS” IEEE, 2019), hereinafter Zheng.
Regarding claim 11, Jin in view of Ingel and Arik teaches claim 6, and Jin further teaches
the second text is in a middle area of the target text (words can be cut, copied, or pasted within the text of a transcript, such as replacing ‘papers’ with ‘videos’ in the middle of a sentence Fig. 2).
While Jin in view of Ingel and Arik provides decoding speaker embeddings and vector representations to predict spectrograms, Jin in view of Ingel and Arik does not specifically teach decoding in a forward and reverse order to obtain different speech features, and thus does not teach
decoding, by the processor based on the decoder and the first speech feature, the first vector from the target text in a forward order to obtain a third speech feature;
decoding, by the processor based on the decoder and the first speech feature, the first vector from the target text in a reverse order to obtain a fourth speech feature; and
obtaining, by the processor, the second speech feature based on the third speech feature and the fourth speech feature.
Zheng, however, teaches decoding, by the processor based on the decoder and the first speech feature, the first vector from the target text in a forward order to obtain a third speech feature (the embedding sequence of the text output by the encoder, i.e. first vector from the target text, is consumed by a forward decoder to summarize current and past elements of the sequence and output a predicted mel spectrogram, i.e. decoding…based on the first speech feature…in a forward order to obtain a third speech feature, and where model regularization can include the use of a text sequence and its target mel spectrograms, i.e. first speech feature (Sec. II.B., Sec. III));
decoding, by the processor based on the decoder and the first speech feature, the first vector from the target text in a reverse order to obtain a fourth speech feature (the embedding sequence of the text output by the encoder, i.e. first vector from the target text, is consumed by a backward decoder to summarize current and future elements of the sequence and output a predicted mel spectrogram, i.e. decoding…based on the first speech feature…in a reverse order to obtain a fourth speech feature, and where model regularization can include the use of a text sequence and its target mel spectrograms, i.e. first speech feature (Sec. II.B., Sec. III)); and
obtaining, by the processor the second speech feature based on the third speech feature and the fourth speech feature (the distance between the forward and backward hidden states is regularized such that the decoder outputs mel-spectrograms, i.e. obtaining the second speech feature based on the third speech feature and the fourth speech feature (Sec. II.B., Sec. III)).
Where Jin specifically teaches the text to be inserted and synthesis of the new words in the voice of the recording Sec. 3.1, as well as that the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2).
Jin, Ingel, Arik, and Zheng are analogous art because they are from a similar field of endeavor in text to speech synthesis. Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the decoding speaker embeddings and vector representations to predict spectrograms teachings of Jin, as modified by Ingel and Arik, with the maximization of agreement between the generated spectrograms from L2R and R2L end-to-end TTS models as taught by Zheng. It would have been obvious to combine the references to enable improvement of both robustness and overall naturalness of neural end-to-end TTS performance (Zheng Abstract).
Regarding claim 12, Jin in view of Ingel, Arik, and Zheng teaches claim 11, and Ingel further teaches
the second text comprises a third text and a fourth text, the third speech feature is corresponding to the third text, and the fourth speech feature is corresponding to the fourth text (the edited text can include a new word or phrase, which is broken into phoneme sequences or snippets, i.e. the second text comprises a third text and a fourth text, where the query frame of the source voice generated from the text of the new word, and snippets of the target voice that sound similar to the query are determined by comparing MFCCs and feature vectors, to obtain a matching unit for each snippet, i.e. the third speech feature is corresponding to the third text and the fourth speech feature is corresponding to the fourth text (Sec. 3, Sec. 4 para 1-5, Sec 4.1)); and
the obtaining the second speech feature based on the third speech feature and the fourth speech feature comprises:
concatenating, by the processor, the third speech feature and the fourth speech feature to obtain the second speech feature (the feature vectors of consecutive frames are concatenated, i.e. concatenating the third speech feature and the fourth speech feature, into exemplars, i.e. obtain the second speech feature (Sec. 3, Sec. 4 para 1-5, Sec 4.1), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)).
Regarding claim 13, Jin in view of Ingel, Arik, and Zheng teaches claim 11, and Zheng further teaches
third speech feature is corresponding to the second text obtained by the decoder based on the forward order, and the fourth speech feature is corresponding to the second text and that is obtained by the decoder based on the reverse order (the embedding sequence of the text output by the encoder, i.e. second text, is consumed by a forward decoder to summarize current and past elements of the sequence and output a predicted mel spectrogram, i.e. third speech feature is corresponding to the second text obtained by the decoder based on the forward order, and is consumed by a backward decoder to summarize current and future elements of the sequence and output a predicted mel spectrogram, i.e. the fourth speech feature is corresponding to the second text and that is obtained by the decoder based on the reverse order (Sec. II.B., Sec. III)); and
the obtaining the second speech feature based on the third speech feature and the fourth speech feature comprises:
determining, by the processor in the third speech feature and the fourth speech feature, a speech feature whose similarity is greater than a first threshold as a transitional speech feature (a regularization term is used to minimize the L2 distance, i.e. determining…a speech feature whose similarity is greater than a first threshold as a transitional speech feature, between the forward and backward hidden states, i.e. in the third speech feature and the fourth speech feature (Sec. II.B., Sec. III)); and
…wherein the fifth speech feature is captured from the third speech feature based on a position of the transitional speech feature in the third speech feature, and the sixth speech feature is captured from the fourth speech feature based on a position of the transitional speech feature in the fourth speech feature (the decoder performs location sensitive attention during forward and backward decoding, i.e. based on a position of the transitional speech feature in the third speech feature…based on a position of the transitional speech feature in the fourth speech feature, and the decoder predicts mel spectrograms, i.e. the fifth speech feature is captured from the third speech feature…the sixth speech feature is captured from the fourth speech feature (Sec. II.B., Sec. III)).
Where Jin further teaches concatenating, by the processor, a fifth speech feature and a sixth speech feature to obtain the second speech feature (feature vectors are concatenated to form exemplars (Sec. 4.1), where the method runs on a system with a computer program where data is retained in a database, i.e. by the processor (Abstract, Sec. 5.2)).
And where the motivation to combine is the same as previously presented.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICOLE A K SCHMIEDER whose telephone number is (571)270-1474. The examiner can normally be reached 8:00 - 5:00 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached at (571) 272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICOLE A K SCHMIEDER/Primary Examiner, Art Unit 2659
Prosecution Insights