DETAILED ACTION
This office action is in response to the applicant’s Request for Continued Examination (RCE), received on 02/26/2026. Claims 1, 11, and 19 have been amended. Claims 1-4, 6-14, 15-20 are pending and have been considered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/26/2026 has been entered.
Response to Arguments
Applicant’s arguments, see pgs. 7-9, filed 02/26/2026, with respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. 103 (Li in view of Wang) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Chang et al. (US-20220108681-A1), hereinafter Chang. Chang discloses “deep neural network-based non-autoregressive voice synthesizing method and a system therefor. A deep neural network-based non-autoregressive voice synthesizing system according to an embodiment may comprise: a voice feature vector column synthesizing unit which constitutes a non-recursive deep neural network based on multiple decoders, and gradually produces a voice feature vector column through the multiple decoders from a template including temporal information of a voice; and a voice reconstituting unit which transforms the voice feature vector column into voice data, wherein the voice feature vector column synthesizing unit produces a template input, and produces a voice feature vector column by adding, to the template input, sentence data refined through an attention mechanism” (abstract). Specifically, Fig. 5 and associated paragraphs of the specification will be discussed. See updated rejections below.
Applicant’s arguments with respect to claim(s) 4, 6, 8, 14, 16, 18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7, 9-13, 17, 19, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. ("Parallel Rescoring with Transformer for Streaming On-Device Speech Recognition"), hereinafter Li, in view of Chang et al. (US-20220108681-A1), hereinafter Chang
Regarding claim 1, Li discloses a computer-implemented method executed by data processing hardware that causes the data processing hardware to perform operations comprising:
receiving an initial alignment ([Figure 1, “RNN-T hypothesis”], [The output from a first pass tracks to an initial alignment] for a candidate hypothesis generated by a transducer decoder ([Figure 1, RNN-T decoder]) model during a first pass based on an initial sequence of audio encodings characterizing an utterance ([Figure 1, “1st pass”]), the candidate hypothesis corresponding to a candidate transcription for the utterance and the initial alignment for the candidate hypothesis comprising a first sequence of output labels ([Figure 1, “RNN-T hypothesis”], [2.1, Par. 1] RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens [Word piece token generated from a first pass track to output labels corresponding to a candidate hypothesis]) each corresponding to a blank symbol or a hypothesized sub-word unit ([Figure 1, “RNN-T hypothesis”] [An output sequence from a first pass being sent to a second pass tracks to a candidate hypothesis]);
receiving a subsequent sequence of audio encodings characterizing the utterance ([Figure 1, “acoustic frames”]), where, ([2.1] In the 1st-pass, each acoustic frame xt is passed through RNN-T encoder… RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion ); and,
during a first refinement step, generating, using a non-autoregressive decoder configured to receive the initial alignment for the candidate hypothesis generated by the transducer decoder model during the first pass and the subsequent sequence of audio encodings ([Introduction, Par. 5] Unlike beam search, where the Transformer decoder still has to run autoregressively, the rescoring scenario allows parallel processing of the full hypothesis sequence), a corresponding first refined alignment for a first rescored sequence of output labels each corresponding to a blank symbol or a hypothesized sub-word unit ([Figure 1, “Transformer rescorer”], [Figure 2, “Self-attention”]).
Li does not disclose:
during a second refinement step subsequent to the first refinement step, generating, using the non-autoregressive decoder configured to receive the first rescored sequence of output labels generated during the first refinement step by the non-autoregressive decoder and without using the initial alignment for the candidate hypothesis generated by the transducer decoder model, a second refined alignment for a second rescored sequence of output labels each corresponding to a blank symbol of a hypothesized sub-word unit.
Chang discloses:
during a second refinement step subsequent to the first refinement step ([Fig. 5, M-th Single Decoder 210 within Speech Feature Vector Sequence Synthesis Unit 200 after First Single Decoder], [0071] The speech feature vector sequence synthesis unit 200 corresponds to the decoder of all models, [0037] FIG. 1 is a diagram illustrating an overall configuration of a non-autoregressive speech synthesis system, [0058] the speech feature vector sequence synthesis unit may construct a multi-decoder-based non-regressive DNN, [Disclosing the unit 200 to be a decoder for all models, wherein the decoder is comprised of decoders, indicates each decoder within the decoding unit to be a decoding refinement step]), generating, using the non-autoregressive decoder configured to receive the first rescored sequence of output labels generated during the first refinement step by the non-autoregressive decoder and without using the initial alignment for the candidate hypothesis generated by the transducer decoder model ([0073] The multiple decoders 210 have a structure including several decoders 210 having the same structure, and have a structure in which the output of the decoder 210 in a previous step is inputted as an input to the decoder 210 in a next step, [As seen in the previously cited Fig. 5, the M-th single decoder, i.e. the decoder used for the second refinement step, only receives output from the first single decoder, wherein that decoder receives the initial alignment, indicating the decoding unit 200 to be the non-autoregressive decoder with a second refinement step which does not use the initial alignment, but receives it for a first refinement step]), a second refined alignment for a second rescored sequence of output labels each corresponding to a blank symbol of a hypothesized sub-word unit ([Fig. 5, Mel-filterbank Speech Feature Vector String 204], [0073] the Mel-filterbank speech feature vector sequence 204…includes sentence information and speech information in a previous time, [Sentence and/or speech information represented in a vector string indicates each component of that vector to be a hypothesized sub-word unit]).
Li and Chang are considered analogous art within non-autoregressive decoding for speech recognition. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li to incorporate the teachings of Chang, because of the novel way to divide the non-autoregressive decoding operation into multiple decoders, i.e. refinements, which improves the learning efficiency of each refinement step, improving the accuracy of overall output from the decoding operations when combined (Chang, [0074]).
Regarding claim 2, Li in view of Chang discloses the computer-implemented method of claim 1,
Li further discloses:
wherein the non-autoregressive decoder comprises a plurality of transformer layers ([Figure 2, “Transformer rescorer”]) each configured to:
perform self-attention on text features associated with a previous alignment, the previous alignment comprising the initial alignment or the first refined alignment ([Introduction, Par. 4] Transformer applies self-attention to capture the sequential relation among input features, [Figure 2, “Self-attention”] [Being performed on an initial alignment as disclosed in Li, could be applied to a first/second refined alignment as disclosed in Chang to the system of Li with no change in functionality]);
use the self-attention performed on the text features as a query to perform cross- attention on the subsequent sequence of audio encodings representing both a key and value to provide a transformer layer output ([Figure 2, “Cross-attention”]).
Regarding claim 3, Li in view of Chang discloses: the computer-implemented method of claim 2.
Li further discloses:
wherein each respective transformer layer subsequent to an initial transformer layer in the plurality of transformer layers receives the transformer layer output from a corresponding previous transformer layer as the text features ([Figure 2], where, [2.1, Par. 1] Here y is a sequence of word-piece tokens, and, [2.2, Par. 1] The Transformer rescorer takes the RNN-T’s hypothesis as input and feed the tokens to the self-attention layer)
Regarding claim 7, Li in view of Chang discloses: the computer-implemented method of claim 1.
Li further discloses:
wherein the operations further comprise generating, by a causal encoder ([3.4] Thus, we report results with causal self-attention for the experiments throughout the paper) during the first pass, the initial sequence of audio encodings based on a sequence of acoustic frames corresponding to an utterance ([Figure 1], [2.1, Par. 1] RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens).
Regarding claim 9, Li in view of Chang discloses: the computer method of claim 7.
Li further discloses:
wherein the transducer decoder model generates the candidate hypothesis using the initial sequence of audio encodings ([Figure 1, 1st pass], [2.1, Par. 1] In the 1st-pass, each acoustic frame xt is passed through RNN-T encoder, consisting of a multi-layer LSTM [8], to get encoder output. RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens).
Regarding claim 10, Li in view of Chang discloses: the computer-implemented method of claim 1.
Li further discloses:
wherein the candidate transcription of the candidate hypothesis comprises a sequence of output labels each corresponding to a hypothesized sub-word unit ([Figure 1, RNN-T hypothesis], [2.1, Par. 1] In the 1st-pass, each acoustic frame xt is passed through RNN-T encoder, consisting of a multi-layer LSTM [8], to get encoder output. RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens).
Regarding claim 11, Li discloses:
a system comprising:
data processing hardware ([3.1] All models are implemented in TensorFlow using the Lingvo toolkit and trained on 8 x 8 Tensor Processing Units (TPU) slices with a global batch size of 4096. [Inherent in compiling/running TensorFlow/Lingvo, an open-source software library written in C++ and Python]); and
memory hardware in communication with the data processing hardware ([3.1] All models are implemented in TensorFlow using the Lingvo toolkit and trained on 8 x 8 Tensor Processing Units (TPU) slices with a global batch size of 4096. [Inherent in implementing TensorFlow/Lingvo, an open-source software library written in C++ and Python]), the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
receiving an initial alignment ([Figure 1, “RNN-T hypothesis”], [The output from a first pass tracks to an initial alignment] for a candidate hypothesis generated by a transducer decoder ([Figure 1, RNN-T decoder]) model during a first pass based on an initial sequence of audio encodings characterizing an utterance ([Figure 1, “1st pass”]), the candidate hypothesis corresponding to a candidate transcription for the utterance and the initial alignment for the candidate hypothesis comprising a first sequence of output labels ([Figure 1, “RNN-T hypothesis”], [2.1, Par. 1] RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens [Word piece token generated from a first pass track to output labels corresponding to a candidate hypothesis]) each corresponding to a blank symbol or a hypothesized sub-word unit ([Figure 1, “RNN-T hypothesis”] [An output sequence from a first pass being sent to a second pass tracks to a candidate hypothesis]);
receiving a subsequent sequence of audio encodings characterizing the utterance ([Figure 1, “acoustic frames”]), where, ([2.1] In the 1st-pass, each acoustic frame xt is passed through RNN-T encoder… RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion ); and,
during a first refinement step, generating, using a non-autoregressive decoder configured to receive the initial alignment for the candidate hypothesis generated by the transducer decoder model during the first pass and the subsequent sequence of audio encodings ([Introduction, Par. 5] Unlike beam search, where the Transformer decoder still has to run autoregressively, the rescoring scenario allows parallel processing of the full hypothesis sequence), a corresponding first refined alignment for a first rescored sequence of output labels each corresponding to a blank symbol or a hypothesized sub-word unit ([Figure 1, “Transformer rescorer”], [Figure 2]).
Li does not disclose:
during a second refinement step subsequent to the first refinement step, generating, using the non-autoregressive decoder configured to receive the first rescored sequence of output labels generated during the first refinement step by the non-autoregressive decoder and without using the initial alignment for the candidate hypothesis generated by the transducer decoder model, a second refined alignment for a second rescored sequence of output labels each corresponding to a blank symbol of a hypothesized sub-word unit.
Chang discloses:
during a second refinement step subsequent to the first refinement step ([Fig. 5, M-th Single Decoder 210 within Speech Feature Vector Sequence Synthesis Unit 200 after First Single Decoder], [0071] The speech feature vector sequence synthesis unit 200 corresponds to the decoder of all models, [0037] FIG. 1 is a diagram illustrating an overall configuration of a non-autoregressive speech synthesis system, [0058] the speech feature vector sequence synthesis unit may construct a multi-decoder-based non-regressive DNN, [Disclosing the unit 200 to be a decoder for all models, wherein the decoder is comprised of decoders, indicates each decoder within the decoding unit to be a decoding refinement step]), generating, using the non-autoregressive decoder configured to receive the first rescored sequence of output labels generated during the first refinement step by the non-autoregressive decoder and without using the initial alignment for the candidate hypothesis generated by the transducer decoder model ([0073] The multiple decoders 210 have a structure including several decoders 210 having the same structure, and have a structure in which the output of the decoder 210 in a previous step is inputted as an input to the decoder 210 in a next step, [As seen in the previously cited Fig. 5, the M-th single decoder, i.e. the decoder used for the second refinement step, only receives output from the first single decoder, wherein that decoder receives the initial alignment, indicating the decoding unit 200 to be the non-autoregressive decoder with a second refinement step which does not use the initial alignment, but receives it for a first refinement step]), a second refined alignment for a second rescored sequence of output labels each corresponding to a blank symbol of a hypothesized sub-word unit ([Fig. 5, Mel-filterbank Speech Feature Vector String 204], [0073] the Mel-filterbank speech feature vector sequence 204…includes sentence information and speech information in a previous time, [Sentence and/or speech information represented in a vector string indicates each component of that vector to be a hypothesized sub-word unit]).
Li and Chang are considered analogous art within non-autoregressive decoding for speech recognition. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li to incorporate the teachings of Chang, because of the novel way to divide the non-autoregressive decoding operation into multiple decoders, i.e. refinements, which improves the learning efficiency of each refinement step, improving the accuracy of overall output from the decoding operations when combined (Chang, [0074]).
Regarding claim 12, Li in view of Chang discloses the system of claim 11.
Li further discloses:
wherein the non-autoregressive decoder comprises a plurality of transformer layers ([Figure 2, “Transformer rescorer”]) each configured to:
perform self-attention on text features associated with a previous alignment, the previous alignment comprising the initial alignment or the first refined alignment ([Introduction, Par. 4] Transformer applies self-attention to capture the sequential relation among input features, [Figure 2, “Self-attention”] [Being performed on an initial alignment as disclosed in Li, could be applied to a first/second refined alignment as disclosed in Chang to the system of Li with no change in functionality]);
use the self-attention performed on the text features as a query to perform cross- attention on the subsequent sequence of audio encodings representing both a key and value to provide a transformer layer output ([Figure 2, “Cross-attention”]).
Regarding claim 13, Li in view of Chang discloses: the system of claim 12.
Li further discloses:
wherein each respective transformer layer subsequent to an initial transformer layer in the plurality of transformer layers receives the transformer layer output from a corresponding previous transformer layer as the text features ([Figure 2], where, [2.1, Par. 1] Here y is a sequence of word-piece tokens, and, [2.2, Par. 1] The Transformer rescorer takes the RNN-T’s hypothesis as input and feed the tokens to the self-attention layer)
Regarding claim 17, Li in view of Chang discloses: the system of claim 11.
Li further discloses:
wherein the operations further comprise generating, by a causal encoder ([3.4] Thus, we report results with causal self-attention for the experiments throughout the paper) during the first pass, the initial sequence of audio encodings based on a sequence of acoustic frames corresponding to an utterance ([Figure 1], [2.1, Par. 1] RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens).
Regarding claim 19, Li in view of Chang discloses: the system of claim 17.
Li further discloses:
wherein the transducer decoder model generates the candidate hypothesis using the initial sequence of audio encodings ([Figure 1, 1st pass], [2.1, Par. 1] In the 1st-pass, each acoustic frame xt is passed through RNN-T encoder, consisting of a multi-layer LSTM [8], to get encoder output. RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens).
Regarding claim 20, Li in view of Chang discloses: the system of claim 11.
Li further discloses:
wherein the candidate transcription of the candidate hypothesis comprises a sequence of output labels each corresponding to a hypothesized sub-word unit ([Figure 1, RNN-T hypothesis], [2.1, Par. 1] In the 1st-pass, each acoustic frame xt is passed through RNN-T encoder, consisting of a multi-layer LSTM [8], to get encoder output. RNN-T decoder takes the acoustic features from RNN-T encoder to generate the hypotheses in a streaming fashion, denoted as y = (y1, …, ys) where s is the label sequence length. Here y is a sequence of word-piece tokens).
Claim(s) 4, 6, 14, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Chang, further in view of Fan et al. ("CASS-NAT: CTC Alignment-Based Single Step Non-Autoregressive Transformer for Speech Recognition"), hereinafter "CASS-NAT".
Regarding claim 4, Li in view of Chang discloses: the computer-implemented method of claim 2.
Li further discloses:
wherein: the refined alignment comprises the first refined alignment and the rescored sequence of output labels comprises the first rescored sequence of output labels ([Fig. 1, “Transformer Rescorer”], [The refined alignment comprises a rescored sequence of output labels from a first alignment, i.e. that from the RNN-T hypothesis]);
or the refined alignment comprises the second refined alignment and the rescored sequence of output labels comprises the second rescored sequence of output labels ([The examiner would like to note that, due to the disjunctive construction of the claim, this element does not require a mapping]).
Li in view of Chang does not disclose:
wherein a final transformer layer in the plurality of transformer layers provides the transformer layer output to a final SoftMax layer configured to predict a refined alignment for the corresponding rescored sequence of output labels.
“CASS-NAT” discloses:
wherein a final transformer layer in the plurality of transformer layers provides the transformer layer output to a final SoftMax layer configured to predict the corresponding new alignment for the corresponding rescored sequence of output labels ([Fig. 1, “Linear + SoftMax”] [Passing output from a decoder into a SoftMax layer is equivalent to a SoftMax layer being a final transform in a series of transform layers])
Li, Chang, and "CASS-NAT" are considered analogous art within non-autoregressive speech recognition. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li in view of Chang to incorporate the teachings of "CASS-NAT", because of the novel way to implement a final SoftMax layer to reduce the word error rate of output generated when compared to a ground truth sequence ("CASS-NAT", Table 1-3).
Regarding claim 6, Li in view of Chang discloses: the computer implemented method of claim 1.
Li further discloses:
wherein the refined alignment comprises the first refined alignment and the previous alignment comprises the initial alignment ([Fig. 1, “Transformer Rescorer”], [The refined alignment comprises a rescored sequence of output labels from a first alignment, i.e. that from the RNN-T hypothesis]).
Chang further discloses:
wherein the refined alignment comprises the second refined alignment and the first refined alignment ([Fig. 5, Addition of output from First Signal Decoder and M-th Single Decoder]).
Li in view of Chang does not disclose:
wherein generating the corresponding new alignment for the corresponding rescored sequence of output labels comprises inserting, deleting, or substituting one or more output labels of the initial alignment for the candidate hypothesis
“CASS-NAT” discloses:
wherein generating the corresponding new alignment for the corresponding rescored sequence of output labels comprises inserting, deleting, or substituting one or more output labels of the initial alignment for the candidate hypothesis ([3.3 Analysis of the CTC Alignments, Par 1] For MR (mismatch rate), only deletion and insertion errors were counted as mismatch since substitution errors will not change the end boundaries and the number of predicted tokens [The indication of mismatch rate based on deletion, insertion, substitution indicates these operations were performed on the text]).
Li, Chang, and "CASS-NAT" are considered analogous art within speech recognition using non-autoregressive transformers. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li in view of Chang to incorporate the teachings of "CASS-NAT", because of the novel way to correct or adjust alignment of generated text to reduce the mismatch rate of generated and original texts. ("CASS-NAT", Table 3).
Regarding claim 14, Li in view of Chang discloses: the system of claim 12.
Li further discloses:
wherein: the refined alignment comprises the first refined alignment and the rescored sequence of output labels comprises the first rescored sequence of output labels ([Fig. 1, “Transformer Rescorer”], [The refined alignment comprises a rescored sequence of output labels from a first alignment, i.e. that from the RNN-T hypothesis]);
or the refined alignment comprises the second refined alignment and the rescored sequence of output labels comprises the second rescored sequence of output labels ([The examiner would like to note that, due to the disjunctive construction of the claim, this element does not require a mapping]).
Li in view of Chang does not disclose:
wherein a final transformer layer in the plurality of transformer layers provides the transformer layer output to a final SoftMax layer configured to predict the corresponding new alignment for the corresponding rescored sequence of output labels.
“CASS-NAT” discloses:
wherein a final transformer layer in the plurality of transformer layers provides the transformer layer output to a final SoftMax layer configured to predict the corresponding new alignment for the corresponding rescored sequence of output labels ([Fig. 1, “Linear + SoftMax”] [Passing output from a decoder into a SoftMax layer is equivalent to a SoftMax layer being a final transform in a series of transform layers])
Li, Chang, and "CASS-NAT" are considered analogous art within non-autoregressive speech recognition. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li in view of Chang to incorporate the teachings of "CASS-NAT", because of the novel way to implement a final SoftMax layer to reduce the word error rate of output generated when compared to a ground truth sequence ("CASS-NAT", Table 1-3).
Regarding claim 16, Li in view of Chang discloses: the system of claim 11.
Li further discloses:
wherein the refined alignment comprises the first refined alignment and the previous alignment comprises the initial alignment ([Fig. 1, “Transformer Rescorer”], [The refined alignment comprises a rescored sequence of output labels from a first alignment, i.e. that from the RNN-T hypothesis]).
Chang further discloses:
wherein the refined alignment comprises the second refined alignment and the first refined alignment ([Fig. 5, Addition of output from First Signal Decoder and M-th Single Decoder]).
Li in view of Chang does not disclose:
wherein generating the corresponding new alignment for the corresponding rescored sequence of output labels comprises inserting, deleting, or substituting one or more output labels of the initial alignment for the candidate hypothesis
“CASS-NAT” discloses:
wherein generating the corresponding new alignment for the corresponding rescored sequence of output labels comprises inserting, deleting, or substituting one or more output labels of the initial alignment for the candidate hypothesis ([3.3 Analysis of the CTC Alignments, Par 1] For MR (mismatch rate), only deletion and insertion errors were counted as mismatch since substitution errors will not change the end boundaries and the number of predicted tokens [The indication of mismatch rate based on deletion, insertion, substitution indicates these operations were performed on the text]).
Li, Chang, and "CASS-NAT" are considered analogous art within speech recognition using non-autoregressive transformers. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li in view of Chang to incorporate the teachings of "CASS-NAT", because of the novel way to correct or adjust alignment of generated text to reduce the mismatch rate of generated and original texts. ("CASS-NAT", Table 3).
Claim(s) 8 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Chang, further in view of van den Oord et al. (US-20180075343-A1), hereinafter van den Oord.
Regarding claim 8, Li in view of Chang discloses: the computer-implemented method of claim 7.
Li in view of Chang does not disclose:
wherein the subsequent sequence of audio encodings are encoded by a non-causal encoder based on the initial sequence of audio encodings.
van den Oord discloses:
wherein the subsequent sequence of audio encodings are encoded by a non-causal encoder based on the initial sequence of audio encodings ([0131] In these implementations, the output subnetwork 520 also includes one or more additional layers, e.g., one or more non-causal convolutional layers followed by a SoftMax output layer, configured to receive the mean pooled representation and to generate a score distribution for an output in the output sequence at a position corresponding to the coarser frame)
Li, Chang, and van den Oord are considered analogous art within speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li in view of Chang to incorporate the teachings of van den Oord, because of the novel way to generate mean pooled representations for alternate representations corresponding to coarser frames allowing for multiple possible final representations of the same initial utterance (van den Oord, [0131]).
Regarding claim 18, Li in view of Chang discloses: the system of claim 17.
Li in view of Chang does not disclose:
wherein the subsequent sequence of audio encodings are encoded by a non-causal encoder based on the initial sequence of audio encodings.
van den Oord discloses:
wherein the subsequent sequence of audio encodings are encoded by a non-causal encoder based on the initial sequence of audio encodings ([0131] In these implementations, the output subnetwork 520 also includes one or more additional layers, e.g., one or more non-causal convolutional layers followed by a SoftMax output layer, configured to receive the mean pooled representation and to generate a score distribution for an output in the output sequence at a position corresponding to the coarser frame)
Li, Chang, and van den Oord are considered analogous art within speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Li in view of Chang to incorporate the teachings of van den Oord, because of the novel way to generate mean pooled representations for alternate representations corresponding to coarser frames allowing for multiple possible final representations of the same initial utterance (van den Oord, [0131]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zhang et al. (US-20230153548-A1) discloses “A translation method, an electronic device and a storage medium, which relate to the field of artificial intelligence technologies, such as machine learning technologies, information processing technologies, are disclosed. An implementation includes: acquiring an intermediate translation result generated by each of multiple pre-trained translation models for a to-be-translated specified sentence in a same iteration of a translation process, so as to obtain multiple intermediate translation results; acquiring a co-occurrence word based on the multiple intermediate translation results; and acquiring a target translation result of the specified sentence based on the co-occurrence word.” (abstract). See Fig. 1C/1D. The examiner would like to note that this art does not beat the EFD of the instant application.
Bradbury et al. (US-20190130249-A1) discloses “A method for sequence-to-sequence prediction using a neural network model includes A method for sequence-to-sequence prediction using a neural network model, generating an encoded representation based on an input sequence using an encoder of the neural network model, predicting a fertility sequence based on the input sequence, generating an output template based on the input sequence and the fertility sequence, and predicting an output sequence based on the encoded representation and the output template using a decoder of the neural network model. The neural network model includes a plurality of model parameters learned according to a machine learning process. Each item of the fertility sequence includes a fertility count associated with a corresponding item of the input sequence” (abstract). See entire document.
Volkovs et al. (US-20230119108-A1) discloses “An autoencoder model includes an encoder portion and a decoder portion. The encoder encodes an input token sequence to an input sequence representation that is decoded by the decoder to generate an output token sequence. The autoencoder model may decode multiple output tokens in parallel, such that the decoder may be applied iteratively. The decoder may receive an output estimate from a prior iteration to predict output tokens. To improve positional representation and reduce positional errors and repetitive tokens, the autoencoder may include a trained layer for combining token embeddings with positional encodings. In addition, the model may be trained with a corrective loss based on output predictions when the model receives a masked input as the output estimate” (abstract). Fig. 3 is being referenced for identification of the state of the art.
Bai et al. (“Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition”) discloses “a non-autoregressive end-to-end speech recognition system called LASO (listen attentively, and spell once). Because of the non-autoregressive property, LASO predicts a textual token in the sequence without the dependence on other tokens. Without beam-search, the one-pass propagation much reduces inference time cost of LASO. And because the model is based on the attention based feedforward structure, the computation can be implemented in parallel efficiently” (abstract). See structure of Fig. 1.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THEODORE JOHN WITHEY whose telephone number is (703)756-1754. The examiner can normally be reached Monday - Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached on ((571) 272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THEODORE WITHEY/Examiner, Art Unit 2655
/ANDREW C FLANDERS/Supervisory Patent Examiner, Art Unit 2655
Prosecution Insights