DETAILED ACTION
This action is in response to the application filed on March 31st, 2025. Claims 1-22 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 7, 10-15, 18, 21, and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dimitriadis et al. (US Pat. Pub. No. 2023/0297777 A1 hereinafter Dimitriadis), in view of Sklyar et al. (Sklyar, Ilya et al. “Separator-Transducer-Segmenter: Streaming Recognition and Segmentation of Multi-party Speech.” ArXiv abs/2205.05199 (2022): n. pag. hereinafter Sklyar).
Regarding claim 1, Dimitriadis discloses a computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising (Dimitriadis, [0048]: "The following description of the method 100 is provided with reference to the software and hardware components described above and shown in FIGS. 1-5. It will be appreciated that the method 100 also may be performed in other contexts using other suitable hardware and software components."): receiving a plurality of training samples each comprising a corresponding sequence of training input features paired with a corresponding sequence of ground-truth output tokens, the sequence of ground-truth output tokens comprising a set of ground-truth common tokens and a set of ground-truth special tokens (Dimitriadis, [0022]: "The raw text data 54B is tokenized by tokenizer 30 to produce a set of tokenized text data 31, which includes a sequence of tokens 32. Any of a variety of tokenizers may be used, as appropriate, and specific examples are given below. User specifier 34 is a software module that receives the user identification data 54A, determines user-specific tokens 52A for each user based on the user identification data 54A, and inserts the user-specific tokens 52A into the token set, for example at the start and end of each sentence, to thereby demarcate the utterance within a user-specific token set 36....The user-specific token sets 36 are inputted into the multi-user personalized NLP model 38 during training, along with the corresponding ground truth classifications 54C, the model is trained based on the training data set 54, and a trained multi-user NLP model 38A is outputted."). However, Dimitriadis fails to expressly recite for each training sample in the plurality of training samples: processing, using a sequence transduction model, the corresponding sequence of training input features to obtain one or more output token sequence hypotheses, each output token sequence hypothesis comprising one or more predicted common tokens; and determining a per sample token-level loss based on, for each corresponding output token sequence hypothesis obtained for the training sample: a number of special token insertions each associated with a corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens; and a number of special token deletions each associated with a corresponding ground-truth special token in the set of ground-truth special tokens that does not appear in corresponding output token sequence hypothesis; and training the sequence transduction model to minimize additive error rate based on the per sample token-level losses determined for the plurality of training samples.
Sklyar teaches for each training sample in the plurality of training samples: processing, using a sequence transduction model, the corresponding sequence of training input features to obtain one or more output token sequence hypotheses, each output token sequence hypothesis comprising one or more predicted common tokens (Sklyar, Pg. 2, 2.1: "STS model inherits its separator and transducer functionalities from a multi-turn RNN-T (MT-RNN-T) [26]. MT-RNN-T extends the standard RNN-T [25] to overlapping speech recognition with multiple output channels N, where N is a maximum number of simultaneously active speakers in the audio."; " The encoder takes acoustic features x as input and produces high-level disentangled acoustic representations hn as output: hn = RecEnc(SepEncn(MixEnc(x))). (1) In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. "); and determining a per sample token-level loss based on, for each corresponding output token sequence hypothesis obtained for the training sample: a number of special token insertions each associated with a corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens; and a number of special token deletions each associated with a corresponding ground-truth special token in the set of ground-truth special tokens that does not appear in corresponding output token sequence hypothesis; and training the sequence transduction model to minimize additive error rate based on the per sample token-level losses determined for the plurality of training samples (Sklyar, Pg. 2, 2.2: " The model is trained in a multi-task fashion by jointly optimizing RNN-T and masking losses: L =LRNN−T +γ∗ L2(hn ◦mn), n (3) where mn is an inverse binary mask of speech activity for output channel n and γ is a weight of masking loss. This approach enforces encoder outputs hn for each output channel n to be close to 0 in frames where there is no active speaker turn as signed to this output channel."; "Besides FastEmit, we also explore another emission regularization method, dynamic latency penalty, applied specifically to the <eot> token. This approach was originally proposed for end-of-speech token emission latency regularization in [33] and investigated for endpoint detection in multi-speaker ASR in [29]. In our case, for each <eot> token in the target transcription, we apply the following dynamic penalty to the probability of <eot> emission in log domain: log P(eot |xt)− = max(0,α(t −τ −tend)) (4) where α is a tunable scale of a late <eot> emission penalty, τ is a <eot> token frame buffer and tend is a ground-truth end of-turn frame. This penalty increases over time and enforces timely emission of the <eot> token.").
Dimitriadis and Sklyar are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis to incorporate the teachings of Sklyar to train the language model based on special tokens and a token-based loss. This helps improve the system’s ability to segment speech with a high accuracy (Sklyar, Pg. 1, 1). As such, the system is able to provide better outputs to the user.
Regarding claim 2, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Dimitriadis further discloses determining the per sample token-level loss is further based on a total number of ground-truth output tokens in the corresponding sequence of ground-truth output tokens and a respective number of predicted common token errors relative to the set of ground-truth common tokens in the corresponding sequence of ground-truth tokens (Dimitriadis, [0025]-[0026]: "During the training stage of the personalized NLP model 38 configured as a transformer sequence classifier as shown in FIG. 2, cross-entropy loss for classification can be minimized according to the following equations. Here, x denotes the input set of tokenized text data (i.e., the user-specific token sets 36), u.sub.id denotes the user identifier for the user to whom the set of tokenized text data belongs, and y is the class label for x. For parametrizations of the user identifiers 52, parameter tying may be used, where the user identifiers 52 use the same set of parameters for their embeddings as the rest of the tokens. In other words, user embedding parameters of the predetermined user-specific tokens 52 may be tied to embedding parameters of the transformer classification model 38. These principles may be applied to the transformer sequence-to-sequence model of FIG. 3 as well.). However, Dimitriadis fails to expressly recite when determining the per sample token-level loss, the number of special token insertions and the number of special token deletions are each weighted higher than the respective number of predicted common token errors to force the sequence transduction model to reduce special token insertion and deletion rates during training.
Sklyar further teaches when determining the per sample token-level loss, the number of special token insertions and the number of special token deletions are each weighted higher than the respective number of predicted common token errors to force the sequence transduction model to reduce special token insertion and deletion rates during training (Sklyar, Pg. 2, 2.2.4: "Besides FastEmit, we also explore another emission regularization method, dynamic latency penalty, applied specifically to the <eot> token. This approach was originally proposed for end-of-speech token emission latency regularization in [33] and investigated for endpoint detection in multi-speaker ASR in [29]. In our case, for each <eot> token in the target transcription, we apply the following dynamic penalty to the probability of <eot> emission in log domain: log P(eot |xt)− = max(0,α(t −τ −tend)) (4) where α is a tunable scale of a late <eot> emission penalty, τ is a <eot> token frame buffer and tend is a ground-truth end of-turn frame. This penalty increases over time and enforces timely emission of the <eot> token."). The same motivation for claim 1 applies equally to claim 2.
Regarding claim 3, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches wherein the operations further comprise, for each training sample in the plurality of training samples: processing, using the sequence transduction model, the corresponding sequence of training input features to predict probability distributions over possible output tokens; and training the sequence transduction model based on a negative log of the probability distributions for the corresponding sequence of ground-truth output tokens conditioned on the corresponding sequence of training input features (Sklyar, Pg. 2, 2.1: "In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. In this case, first separation encoder learns to focus on the very first speaker turns, and the second on the follow-up speaker turn if it exists. As a result, DAT computes RNN-T loss only N times: LRNN−T =− [sum] log P(yn|hn). n (2)"). The same motivation for claim 1 applies equally to claim 3.
Regarding claim 4, the rejection of claim 3 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches training the sequence transduction model based on the negative log of the probability distributions comprises initially training the sequence transduction model based on the negative log of the probability distributions to initialize the sequence transduction model; and training the sequence transduction model to minimize additive error rate based on the per sample token-level losses comprises fine tuning the initialized sequence transduction model based on minimizing the additive error rate based on the per sample token-level losses (Sklyar, Pg. 2, 2.1: "In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. In this case, first separation encoder learns to focus on the very first speaker turns, and the second on the follow-up speaker turn if it exists. As a result, DAT computes RNN-T loss only N times: LRNN−T =−[sum] log P(yn|hn). n (2)"). The same motivation for claim 1 applies equally to claim 4.
Regarding claim 7, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches the sequence of training input features comprises a sequence of input audio frames characterizing multiple utterances spoken by at least two different speakers (Sklyar, Pg. 2, 2.2.3: " The model is trained in a multi-task fashion by jointly optimizing RNN-T and masking losses: L =LRNN−T +γ∗ L2(hn ◦mn), n (3) where mn is an inverse binary mask of speech activity for output channel n and γ is a weight of masking loss. This approach enforces encoder outputs hn for each output channel n to be close to 0 in frames where there is no active speaker turn assigned to this output channel."); the set of ground-truth common tokens in each corresponding sequence of ground-truth output tokens comprises a set of word or sub-word unit tokens that form a ground-truth transcription of the multiple spoken utterances characterized by the sequence of input audio frames (Sklyar, Pg. 2, 2.2.3: "To further regularize segmentation capability of the STS model, during training we expose it to the ground-truth segmentation information that is encoded in speech activity labels. "); the set of ground-truth special tokens in each corresponding sequence of ground-truth output tokens comprises a set of one or more ground-truth speaker change tokens each indicating a respective location where a speaker change occurs in the ground-truth transcription of the multiple utterances (Sklyar, Pg. 1, 1: " In this work we attack this issue by explicit turn boundary modeling with start-of-turn (<sot>) and end-of turn (<eot>) tokens in a novel separator-transducer-segmenter (STS) model. We further improve recognition and segmentation accuracy of this model through FastEmit [31] and masking loss that penalizes leakage of acoustic information into encoder outputs corresponding to non-active speech frames."); the one or more predicted common tokens of each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses comprises a sequence of predicted word or sub-word unit tokens that form a respective candidate transcription of the multiple utterances (Sklyar, Pg. 2, 2.1: " The encoder takes acoustic features x as input and produces high-level disentangled acoustic representations hn as output: hn = RecEnc(SepEncn(MixEnc(x))). (1) In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. "); and each corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens comprises a predicted speaker change token indicating a respective location in the respective candidate transcription where a respective speaker change event is detected by the sequence transduction model (Sklyar, Pg. 2, 2.2.4: "Besides FastEmit, we also explore another emission regularization method, dynamic latency penalty, applied specifically to the <eot> token. This approach was originally proposed for end-of-speech token emission latency regularization in [33] and investigated for endpoint detection in multi-speaker ASR in [29]. In our case, for each <eot> token in the target transcription, we apply the following dynamic penalty to the probability of <eot> emission in log domain: log P(eot |xt)− = max(0,α(t −τ −tend)) (4) where α is a tunable scale of a late <eot> emission penalty, τ is a <eot> token frame buffer and tend is a ground-truth end of-turn frame. This penalty increases over time and enforces timely emission of the <eot> token."). The same motivation for claim 1 applies equally to claim 7.
Regarding claim 10, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches wherein the sequence transduction model comprises a recurrent neural network- transducer (RNN-T) model architecture (Sklyar, Pg. 2, 2.1: "STS model inherits its separator and transducer functionalities from a multi-turn RNN-T (MT-RNN-T) [26]."). The same motivation for claim 1 applies equally to claim 10.
Regarding claim 11, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches wherein the sequence transduction model comprises at least one of a character recognition model, a speech recognition model, an endpointing model, a speaker turn detection model, or a machine translation model (Sklyar, Pg. 1, 1: " In this work we attack this issue by explicit turn boundary modeling with start-of-turn (<sot>) and end-of turn (<eot>) tokens in a novel separator-transducer-segmenter (STS) model. We further improve recognition and segmentation accuracy of this model through FastEmit [31] and masking loss that penalizes leakage of acoustic information into encoder outputs corresponding to non-active speech frames."). The same motivation for claim 1 applies equally to claim 11.
Regarding claim 12, Dimitriadis discloses a system comprising data processing hardware; memory hardware in communication with the data processing hardware and storing instructions, that when executed by the data processing hardware, cause the data processing hardware to perform operations comprising (Dimitriadis, [0063]: "One aspect provides a personalized natural language processing system comprising: at least one processor, communicatively coupled to non-volatile memory storing a natural language processing (NLP) model personalized for use by multiple users and instructions that, when executed by the processor, cause the processor to"): receiving a plurality of training samples each comprising a corresponding sequence of training input features paired with a corresponding sequence of ground-truth output tokens, the sequence of ground-truth output tokens comprising a set of ground-truth common tokens and a set of ground-truth special tokens (Dimitriadis, [0022]: "The raw text data 54B is tokenized by tokenizer 30 to produce a set of tokenized text data 31, which includes a sequence of tokens 32. Any of a variety of tokenizers may be used, as appropriate, and specific examples are given below. User specifier 34 is a software module that receives the user identification data 54A, determines user-specific tokens 52A for each user based on the user identification data 54A, and inserts the user-specific tokens 52A into the token set, for example at the start and end of each sentence, to thereby demarcate the utterance within a user-specific token set 36....The user-specific token sets 36 are inputted into the multi-user personalized NLP model 38 during training, along with the corresponding ground truth classifications 54C, the model is trained based on the training data set 54, and a trained multi-user NLP model 38A is outputted."). However, Dimitriadis fails to expressly recite for each training sample in the plurality of training samples: processing, using a sequence transduction model, the corresponding sequence of training input features to obtain one or more output token sequence hypotheses, each output token sequence hypothesis comprising one or more predicted common tokens; and determining a per sample token-level loss based on, for each corresponding output token sequence hypothesis obtained for the training sample: a number of special token insertions each associated with a corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens; and a number of special token deletions each associated with a corresponding ground-truth special token in the set of ground-truth special tokens that does not appear in corresponding output token sequence hypothesis; and training the sequence transduction model to minimize additive error rate based on the per sample token-level losses determined for the plurality of training samples.
Sklyar teaches for each training sample in the plurality of training samples: processing, using a sequence transduction model, the corresponding sequence of training input features to obtain one or more output token sequence hypotheses, each output token sequence hypothesis comprising one or more predicted common tokens (Sklyar, Pg. 2, 2.1: "STS model inherits its separator and transducer functionalities from a multi-turn RNN-T (MT-RNN-T) [26]. MT-RNN-T extends the standard RNN-T [25] to overlapping speech recognition with multiple output channels N, where N is a maximum number of simultaneously active speakers in the audio."; " The encoder takes acoustic features x as input and produces high-level disentangled acoustic representations hn as output: hn = RecEnc(SepEncn(MixEnc(x))). (1) In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. "); and determining a per sample token-level loss based on, for each corresponding output token sequence hypothesis obtained for the training sample: a number of special token insertions each associated with a corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens; and a number of special token deletions each associated with a corresponding ground-truth special token in the set of ground-truth special tokens that does not appear in corresponding output token sequence hypothesis; and training the sequence transduction model to minimize additive error rate based on the per sample token-level losses determined for the plurality of training samples (Sklyar, Pg. 2, 2.2: " The model is trained in a multi-task fashion by jointly optimizing RNN-T and masking losses: L =LRNN−T +γ∗ L2(hn ◦mn), n (3) where mn is an inverse binary mask of speech activity for output channel n and γ is a weight of masking loss. This approach enforces encoder outputs hn for each output channel n to be close to 0 in frames where there is no active speaker turn as signed to this output channel."; "Besides FastEmit, we also explore another emission regularization method, dynamic latency penalty, applied specifically to the <eot> token. This approach was originally proposed for end-of-speech token emission latency regularization in [33] and investigated for endpoint detection in multi-speaker ASR in [29]. In our case, for each <eot> token in the target transcription, we apply the following dynamic penalty to the probability of <eot> emission in log domain: log P(eot |xt)− = max(0,α(t −τ −tend)) (4) where α is a tunable scale of a late <eot> emission penalty, τ is a <eot> token frame buffer and tend is a ground-truth end of-turn frame. This penalty increases over time and enforces timely emission of the <eot> token.").
Dimitriadis and Sklyar are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis to incorporate the teachings of Sklyar to train the language model based on special tokens and a token-based loss. This helps improve the system’s ability to segment speech with a high accuracy (Sklyar, Pg. 1, 1). As such, the system is able to provide better outputs to the user.
Regarding claim 13, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Dimitriadis further discloses determining the per sample token-level loss is further based on a total number of ground-truth output tokens in the corresponding sequence of ground-truth output tokens and a respective number of predicted common token errors relative to the set of ground-truth common tokens in the corresponding sequence of ground-truth tokens (Dimitriadis, [0025]-[0026]: "During the training stage of the personalized NLP model 38 configured as a transformer sequence classifier as shown in FIG. 2, cross-entropy loss for classification can be minimized according to the following equations. Here, x denotes the input set of tokenized text data (i.e., the user-specific token sets 36), u.sub.id denotes the user identifier for the user to whom the set of tokenized text data belongs, and y is the class label for x. For parametrizations of the user identifiers 52, parameter tying may be used, where the user identifiers 52 use the same set of parameters for their embeddings as the rest of the tokens. In other words, user embedding parameters of the predetermined user-specific tokens 52 may be tied to embedding parameters of the transformer classification model 38. These principles may be applied to the transformer sequence-to-sequence model of FIG. 3 as well.). However, Dimitriadis fails to expressly recite when determining the per sample token-level loss, the number of special token insertions and the number of special token deletions are each weighted higher than the respective number of predicted common token errors to force the sequence transduction model to reduce special token insertion and deletion rates during training.
Sklyar further teaches when determining the per sample token-level loss, the number of special token insertions and the number of special token deletions are each weighted higher than the respective number of predicted common token errors to force the sequence transduction model to reduce special token insertion and deletion rates during training (Sklyar, Pg. 2, 2.2.4: "Besides FastEmit, we also explore another emission regularization method, dynamic latency penalty, applied specifically to the <eot> token. This approach was originally proposed for end-of-speech token emission latency regularization in [33] and investigated for endpoint detection in multi-speaker ASR in [29]. In our case, for each <eot> token in the target transcription, we apply the following dynamic penalty to the probability of <eot> emission in log domain: log P(eot |xt)− = max(0,α(t −τ −tend)) (4) where α is a tunable scale of a late <eot> emission penalty, τ is a <eot> token frame buffer and tend is a ground-truth end of-turn frame. This penalty increases over time and enforces timely emission of the <eot> token."). The same motivation for claim 12 applies equally to claim 13.
Regarding claim 14, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches wherein the operations further comprise, for each training sample in the plurality of training samples: processing, using the sequence transduction model, the corresponding sequence of training input features to predict probability distributions over possible output tokens; and training the sequence transduction model based on a negative log of the probability distributions for the corresponding sequence of ground-truth output tokens conditioned on the corresponding sequence of training input features (Sklyar, Pg. 2, 2.1: "In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. In this case, first separation encoder learns to focus on the very first speaker turn, and the second on the follow-up speaker turn if it exists. As a result, DAT computes RNN-T loss only N times: LRNN−T =−log P(yn|hn). n (2)"). The same motivation for claim 12 applies equally to claim 14.
Regarding claim 15, the rejection of claim 14 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches training the sequence transduction model based on the negative log of the probability distributions comprises initially training the sequence transduction model based on the negative log of the probability distributions to initialize the sequence transduction model; and training the sequence transduction model to minimize additive error rate based on the per sample token-level losses comprises fine tuning the initialized sequence transduction model based on minimizing the additive error rate based on the per sample token-level losses (Sklyar, Pg. 2, 2.1: "In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. In this case, first separation encoder learns to focus on the very first speaker turn, and the second on the follow-up speaker turn if it exists. As a result, DAT computes RNN-T loss only N times: LRNN−T =−log P(yn|hn). n (2)"). The same motivation for claim 12 applies equally to claim 15.
Regarding claim 18, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches the sequence of training input features comprises a sequence of input audio frames characterizing multiple utterances spoken by at least two different speakers (Sklyar, Pg. 2, 2.2.3: " The model is trained in a multi-task fashion by jointly optimizing RNN-T and masking losses: L =LRNN−T +γ∗ L2(hn ◦mn), n (3) where mn is an inverse binary mask of speech activity for output channel n and γ is a weight of masking loss. This approach enforces encoder outputs hn for each output channel n to be close to 0 in frames where there is no active speaker turn as signed to this output channel."); the set of ground-truth common tokens in each corresponding sequence of ground-truth output tokens comprises a set of word or sub-word unit tokens that form a ground-truth transcription of the multiple spoken utterances characterized by the sequence of input audio frames (Sklyar, Pg. 2, 2.2.3: "To further regularize segmentation capability of the STS model, during training we expose it to the ground-truth segmentation information that is encoded in speech activity labels. "); the set of ground-truth special tokens in each corresponding sequence of ground-truth output tokens comprises a set of one or more ground-truth speaker change tokens each indicating a respective location where a speaker change occurs in the ground-truth transcription of the multiple utterances (Sklyar, Pg. 1, 1: " In this work we attack this issue by explicit turn boundary modeling with start-of-turn (<sot>) and end-of turn (<eot>) tokens in a novel separator-transducer-segmenter (STS) model. We further improve recognition and segmentation accuracy of this model through FastEmit [31] and masking loss that penalizes leakage of acoustic information into encoder outputs corresponding to non-active speech frames."); the one or more predicted common tokens of each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses comprises a sequence of predicted word or sub-word unit tokens that form a respective candidate transcription of the multiple utterances (Sklyar, Pg. 2, 2.1: " The encoder takes acoustic features x as input and produces high-level disentangled acoustic representations hn as output: hn = RecEnc(SepEncn(MixEnc(x))). (1) In order to associate hn with prediction network outputs for each label sequence yn we employ deterministic assignment training (DAT) method [23], which forces the model to learn to associate its output with the speaker order in the audio. "); and each corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens comprises a predicted speaker change token indicating a respective location in the respective candidate transcription where a respective speaker change event is detected by the sequence transduction model (Sklyar, Pg. 2, 2.2.4: "Besides FastEmit, we also explore another emission regularization method, dynamic latency penalty, applied specifically to the <eot> token. This approach was originally proposed for end-of-speech token emission latency regularization in [33] and investigated for endpoint detection in multi-speaker ASR in [29]. In our case, for each <eot> token in the target transcription, we apply the following dynamic penalty to the probability of <eot> emission in log domain: log P(eot |xt)− = max(0,α(t −τ −tend)) (4) where α is a tunable scale of a late <eot> emission penalty, τ is a <eot> token frame buffer and tend is a ground-truth end of-turn frame. This penalty increases over time and enforces timely emission of the <eot> token."). The same motivation for claim 12 applies equally to claim 18.
Regarding claim 21, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches wherein the sequence transduction model comprises a recurrent neural network- transducer (RNN-T) model architecture (Sklyar, Pg. 2, 2.1: "STS model inherits its separator and transducer functionalities from a multi-turn RNN-T (MT-RNN-T) [26]."). The same motivation for claim 12 applies equally to claim 21.
Regarding claim 22, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. Sklyar further teaches wherein the sequence transduction model comprises at least one of a speech recognition model, a character recognition model, an endpointing model, a speaker turn detection model, or a machine translation model (Sklyar, Pg. 1, 1: " In this work we attack this issue by explicit turn boundary modeling with start-of-turn (<sot>) and end-of turn (<eot>) tokens in a novel separator-transducer-segmenter (STS) model. We further improve recognition and segmentation accuracy of this model through FastEmit [31] and masking loss that penalizes leakage of acoustic information into encoder outputs corresponding to non-active speech frames."). The same motivation for claim 12 applies equally to claim 22.
Claim(s) 5 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dimitriadis, in view of Sklyar, as applied to claims 1-4, 7, 10-15, 18, 21, and 22 above, and further in view of Prabhavalkar et al. (US Pat. Pub. No. 2020/0027444 A1 hereinafter Prabhavalkar).
Regarding claim 5, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. However, Dimitriadis, in view of Sklyar, fails to expressly recite processing the corresponding sequence of training input features to obtain one or more output token sequence hypotheses comprises processing the corresponding sequence of training input features to obtain an N-best list of output token sequence hypotheses, each corresponding output token sequence hypothesis in the N-best list having a respective probability score assigned by the sequence transduction model; and determining the per sample token-level loss is further based on the respective probability score of the corresponding output token sequence hypotheses.
Prabhavalkar teaches processing the corresponding sequence of training input features to obtain one or more output token sequence hypotheses comprises processing the corresponding sequence of training input features to obtain an N-best list of output token sequence hypotheses, each corresponding output token sequence hypothesis in the N-best list having a respective probability score assigned by the sequence transduction model; and determining the per sample token-level loss is further based on the respective probability score of the corresponding output token sequence hypotheses (Prabhavalkar, [0027]: "In some implementations, training the speech recognition model using the loss function having a minimum word error rate criterion comprises determining a word error rate expectation for a training example by: determining, based on output of the speech recognition model in response to processing of input data for the training example, a set of hypotheses using beam search decoding; identifying an n-best list of hypotheses based on probabilities indicated by the output of the speech recognition model; limiting evaluation for the word error rate expectation to the n-best list of decoded hypotheses; and approximating the word error rate expectation by weighting a measure of errors in each hypothesis by an amount of the probability distribution that the output of the speech recognition model concentrates on the hypothesis.").
Dimitriadis, Sklyar, and Prabhavalkar are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis, as modified by the separator-transducer-segmenter of Sklyar, to incorporate the teachings of Prabhavalkar to create an n-best list of outputs. This allows the system to limit outputs used for further processing to only ones with a high enough probability (Prabhavalkar, [0027]). As such, the system can ensure that outputs that are unlikely to be accurate are filtered out as soon as possible.
Regarding claim 16, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. However, Dimitriadis, in view of Sklyar, fails to expressly recite processing the corresponding sequence of training input features to obtain one or more output token sequence hypotheses comprises processing the corresponding sequence of training input features to obtain an N-best list of output token sequence hypotheses, each corresponding output token sequence hypothesis in the N-best list having a respective probability score assigned by the sequence transduction model; and determining the per sample token-level loss is further based on the respective probability score of the corresponding output token sequence hypotheses.
Prabhavalkar teaches processing the corresponding sequence of training input features to obtain one or more output token sequence hypotheses comprises processing the corresponding sequence of training input features to obtain an N-best list of output token sequence hypotheses, each corresponding output token sequence hypothesis in the N-best list having a respective probability score assigned by the sequence transduction model; and determining the per sample token-level loss is further based on the respective probability score of the corresponding output token sequence hypotheses (Prabhavalkar, [0027]: "In some implementations, training the speech recognition model using the loss function having a minimum word error rate criterion comprises determining a word error rate expectation for a training example by: determining, based on output of the speech recognition model in response to processing of input data for the training example, a set of hypotheses using beam search decoding; identifying an n-best list of hypotheses based on probabilities indicated by the output of the speech recognition model; limiting evaluation for the word error rate expectation to the n-best list of decoded hypotheses; and approximating the word error rate expectation by weighting a measure of errors in each hypothesis by an amount of the probability distribution that the output of the speech recognition model concentrates on the hypothesis.").
Dimitriadis, Sklyar, and Prabhavalkar are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis, as modified by the separator-transducer-segmenter of Sklyar, to incorporate the teachings of Prabhavalkar to create an n-best list of outputs. This allows the system to limit outputs used for further processing to only ones with a high enough probability (Prabhavalkar, [0027]). As such, the system can ensure that outputs that are unlikely to be accurate are filtered out as soon as possible.
Claim(s) 6 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dimitriadis, in view of Sklyar, as applied to claims 1-4, 7, 10-15, 18, 21, and 22 above, and further in view of Limeres Limeres et al. (US Pat. Pub. No. 2023/0360646 A1 hereinafter Limeres).
Regarding claim 6, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. However, Dimitriadis, in view of Sklyar, fails to expressly recite the sequence of training input features comprises a sequence of input audio frames characterizing an utterance that includes a particular key phrase; the set of ground-truth common tokens in each corresponding sequence of ground-truth output tokens comprises a set of word or sub-word unit tokens that form a ground-truth transcription of the utterance characterized by the sequence of input audio frames; the set of ground-truth special tokens in each corresponding sequence of ground-truth output tokens comprises at least one ground-truth keyword token indicating a respective location in the ground-truth transcription immediately after the particular key phrase appears in the ground- truth transcription; the one or more predicted common tokens of each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses comprises a sequence of predicted word or sub-word tokens that form a respective candidate transcription of the utterance; and each corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens comprises a predicted key phrase token indicating a respective location in the respective candidate transcription immediately after the sequence transduction model predicts that the particular key phrase is detected.
Limeres teaches the sequence of training input features comprises a sequence of input audio frames characterizing an utterance that includes a particular key phrase (Limeres, [0043]: "Furthermore, it is also possible to extend the prefix penalties operation from only working intra-command (as previously exemplified) by adding two additional tokens to the lexicon, e.g., “<start_of_command>” and “<end_of_command>” to bookend any occurrence of a command in the training data. In this manner, the E2E ASR system can learn to predict when a command is going to be uttered and/or has been uttered. Until the token “<start_of_command>” is predicted, the E2E ASR system would produce prefix penalties to mask all the command entries, thereby preventing any attention to them in the first attention mechanism 2007, but subsequently enabling such attention once the token <start_of_command> occurs, which attention is enabled until the “<end_of_command>” token appears."); the set of ground-truth common tokens in each corresponding sequence of ground-truth output tokens comprises a set of word or sub-word unit tokens that form a ground-truth transcription of the utterance characterized by the sequence of input audio frames (Limeres, [0046]: "This loss would consist of a cross-entropy between the ground-truth penalties and the predicted prefix penalties (after the sigmoid function module 208). The ground-truth penalties for the training data can be trivially computed by analyzing the input transcription and checking which bias entry fits at each sequence step (each step being a sub-word token of the transcription sequence, and the notion of fitting as exemplified above). This means that for each training utterance of length U (measured in sub-word tokens), and for a bias list of size Z, the ground-truth penalties would be a matrix of U×Z, with values of zeros/ones depending on whether a bias entry z fits/does not fit at the sequence step u."); the set of ground-truth special tokens in each corresponding sequence of ground-truth output tokens comprises at least one ground-truth keyword token indicating a respective location in the ground-truth transcription immediately after the particular key phrase appears in the ground- truth transcription (Limeres, [0043]: "Furthermore, it is also possible to extend the prefix penalties operation from only working intra-command (as previously exemplified) by adding two additional tokens to the lexicon, e.g., “<start_of_command>” and “<end_of_command>” to bookend any occurrence of a command in the training data."); the one or more predicted common tokens of each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses comprises a sequence of predicted word or sub-word tokens that form a respective candidate transcription of the utterance (Limeres, [0022]: "The output of the example ASR module 11 will contain the transcription (output 1004 shown in FIG. 1) of the conversational speech occurring during the doctor-patient encounter (denoted as “ . . . ” next to “D” for doctor and “P” for patient in FIG. 1) as well as the commands uttered during the doctor-patient encounter."); and each corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens comprises a predicted key phrase token indicating a respective location in the respective candidate transcription immediately after the sequence transduction model predicts that the particular key phrase is detected (Limeres, [0043]: "Furthermore, it is also possible to extend the prefix penalties operation from only working intra-command (as previously exemplified) by adding two additional tokens to the lexicon, e.g., “<start_of_command>” and “<end_of_command>” to bookend any occurrence of a command in the training data.").
Dimitriadis, Sklyar, and Limeres are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis, as modified by the separator-transducer-segmenter of Sklyar, to incorporate the teachings of Limeres to use special tokens to indicate key phrases in an input. This helps remove any ambiguity that might arise from words that could also be used in normal conversation (Limeres, [0022]). Therefore, the system is able to better interpret commands from a user and differentiate them from other speech.
Regarding claim 17, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. However, Dimitriadis, in view of Sklyar, fails to expressly recite the sequence of training input features comprises a sequence of input audio frames characterizing an utterance that includes a particular key phrase; the set of ground-truth common tokens in each corresponding sequence of ground-truth output tokens comprises a set of word or sub-word unit tokens that form a ground-truth transcription of the utterance characterized by the sequence of input audio frames; the set of ground-truth special tokens in each corresponding sequence of ground-truth output tokens comprises at least one ground-truth keyword token indicating a respective location in the ground-truth transcription immediately after the particular key phrase appears in the ground- truth transcription; the one or more predicted common tokens of each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses comprises a sequence of predicted word or sub-word tokens that form a respective candidate transcription of the utterance; and each corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens comprises a predicted key phrase token indicating a respective location in the respective candidate transcription immediately after the sequence transduction model predicts that the particular key phrase is detected.
Limeres teaches the sequence of training input features comprises a sequence of input audio frames characterizing an utterance that includes a particular key phrase (Limeres, [0043]: "Furthermore, it is also possible to extend the prefix penalties operation from only working intra-command (as previously exemplified) by adding two additional tokens to the lexicon, e.g., “<start_of_command>” and “<end_of_command>” to bookend any occurrence of a command in the training data. In this manner, the E2E ASR system can learn to predict when a command is going to be uttered and/or has been uttered. Until the token “<start_of_command>” is predicted, the E2E ASR system would produce prefix penalties to mask all the command entries, thereby preventing any attention to them in the first attention mechanism 2007, but subsequently enabling such attention once the token <start_of_command> occurs, which attention is enabled until the “<end_of_command>” token appears."); the set of ground-truth common tokens in each corresponding sequence of ground-truth output tokens comprises a set of word or sub-word unit tokens that form a ground-truth transcription of the utterance characterized by the sequence of input audio frames (Limeres, [0046]: "This loss would consist of a cross-entropy between the ground-truth penalties and the predicted prefix penalties (after the sigmoid function module 208). The ground-truth penalties for the training data can be trivially computed by analyzing the input transcription and checking which bias entry fits at each sequence step (each step being a sub-word token of the transcription sequence, and the notion of fitting as exemplified above). This means that for each training utterance of length U (measured in sub-word tokens), and for a bias list of size Z, the ground-truth penalties would be a matrix of U×Z, with values of zeros/ones depending on whether a bias entry z fits/does not fit at the sequence step u."); the set of ground-truth special tokens in each corresponding sequence of ground-truth output tokens comprises at least one ground-truth keyword token indicating a respective location in the ground-truth transcription immediately after the particular key phrase appears in the ground- truth transcription (Limeres, [0043]: "Furthermore, it is also possible to extend the prefix penalties operation from only working intra-command (as previously exemplified) by adding two additional tokens to the lexicon, e.g., “<start_of_command>” and “<end_of_command>” to bookend any occurrence of a command in the training data."); the one or more predicted common tokens of each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses comprises a sequence of predicted word or sub-word tokens that form a respective candidate transcription of the utterance (Limeres, [0022]: "The output of the example ASR module 11 will contain the transcription (output 1004 shown in FIG. 1) of the conversational speech occurring during the doctor-patient encounter (denoted as “ . . . ” next to “D” for doctor and “P” for patient in FIG. 1) as well as the commands uttered during the doctor-patient encounter."); and each corresponding predicted special token that appears in the corresponding output token sequence hypothesis but does not appear in the corresponding sequence of ground-truth output tokens comprises a predicted key phrase token indicating a respective location in the respective candidate transcription immediately after the sequence transduction model predicts that the particular key phrase is detected (Limeres, [0043]: "Furthermore, it is also possible to extend the prefix penalties operation from only working intra-command (as previously exemplified) by adding two additional tokens to the lexicon, e.g., “<start_of_command>” and “<end_of_command>” to bookend any occurrence of a command in the training data.").
Dimitriadis, Sklyar, and Limeres are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis, as modified by the separator-transducer-segmenter of Sklyar, to incorporate the teachings of Limeres to use special tokens to indicate key phrases in an input. This helps remove any ambiguity that might arise from words that could also be used in normal conversation (Limeres, [0022]). Therefore, the system is able to better interpret commands from a user and differentiate them from other speech.
Claim(s) 8, 9, 19, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dimitriadis, in view of Sklyar, as applied to claims 1-4, 7, 10-15, 18, 21, and 22 above, and further in view of Thomson et al. (US Pat. Pub. No. 2020/0175961 A1 hereinafter Thomson).
Regarding claim 8, the rejection of claim 1 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. However, Dimitriadis, in view of Sklyar, fails to expressly recite wherein the operations further comprise, for each training sample in plurality of training samples: determining a customized Levenshtein distance between each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses obtained for the corresponding sequence of training input features and the corresponding sequence of ground-truth output tokens; and based on the customized Levenshtein distance: identifying the number of special token insertions for each corresponding output token sequence hypothesis; and identifying the number of special token deletions for each corresponding output token sequence hypothesis.
Thomson teaches wherein the operations further comprise, for each training sample in plurality of training samples: determining a customized Levenshtein distance between each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses obtained for the corresponding sequence of training input features and the corresponding sequence of ground-truth output tokens; and based on the customized Levenshtein distance: identifying the number of special token insertions for each corresponding output token sequence hypothesis; and identifying the number of special token deletions for each corresponding output token sequence hypothesis (Thomson, [0878]: "In some embodiments, the comparer 4504 may be configured to compare the monitored transcription with the reference transcription. In these and other embodiments, the comparer 4504 may compare the monitored transcription with the reference transcription by determining an edit distance or Levenshtein distance there between. In some embodiments, the comparison process by the comparer 4504 may be implemented as follows: (1) the comparer 4504 may align the monitored transcription and the reference transcription; (2) the comparer 4504 may compare each aligned pair of tokens from the monitored transcription and the reference transcription. The pair of tokens may include a first token from the monitored transcription and a second token from the reference transcription; (3) the comparer 4504 may provide an indication, such as a match or no match with respect to each aligned pair of tokens, to the counter 4506. For example, the comparer 4504 may output a zero when a pair of tokens match and a one if there is no match between a pair of tokens; and (4) the number of differences are counted or averaged by the counter 4506 to determine an average disagreement rate, edit distance, and/or Levenshtein distance.").
Dimitriadis, Sklyar, and Thomson are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis, as modified by the separator-transducer-segmenter of Sklyar, to incorporate the teachings of Thomson to use a levenshtein distance to judge the difference between training outputs and ground-truth data. This is a known way of comparing output data to ground-truth data (Thomson, [0878]). As such, the system is able to properly judge the accuracy of its outputs.
Regarding claim 9, the rejection of claim 8 is incorporated. Dimitriadis, in view of Sklyar and Thomson, discloses all of the elements of the current invention as stated above. Thomson further teaches wherein the customized Levenshtein distance determined between each corresponding output token sequence hypothesis and the corresponding sequence of ground-truth output tokens prevents the sequence transduction model from allowing substitutions between special tokens and common tokens during training of the sequence transduction model (Thomson, [0878]: "In some embodiments, the comparer 4504 may be configured to compare the monitored transcription with the reference transcription. In these and other embodiments, the comparer 4504 may compare the monitored transcription with the reference transcription by determining an edit distance or Levenshtein distance there between. In some embodiments, the comparison process by the comparer 4504 may be implemented as follows: (1) the comparer 4504 may align the monitored transcription and the reference transcription; (2) the comparer 4504 may compare each aligned pair of tokens from the monitored transcription and the reference transcription. The pair of tokens may include a first token from the monitored transcription and a second token from the reference transcription; (3) the comparer 4504 may provide an indication, such as a match or no match with respect to each aligned pair of tokens, to the counter 4506. For example, the comparer 4504 may output a zero when a pair of tokens match and a one if there is no match between a pair of tokens; and (4) the number of differences are counted or averaged by the counter 4506 to determine an average disagreement rate, edit distance, and/or Levenshtein distance."). The same motivation for claim 8 applies equally to claim 9.
Regarding claim 19, the rejection of claim 12 is incorporated. Dimitriadis, in view of Sklyar, discloses all of the elements of the current invention as stated above. However, Dimitriadis, in view of Sklyar, fails to expressly recite wherein the operations further comprise, for each training sample in plurality of training samples: determining a customized Levenshtein distance between each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses obtained for the corresponding sequence of training input features and the corresponding sequence of ground-truth output tokens; and based on the customized Levenshtein distance: identifying the number of special token insertions for each corresponding output token sequence hypothesis; and identifying the number of special token deletions for each corresponding output token sequence hypothesis.
Thomson teaches wherein the operations further comprise, for each training sample in plurality of training samples: determining a customized Levenshtein distance between each corresponding output token sequence hypothesis of the one or more output token sequence hypotheses obtained for the corresponding sequence of training input features and the corresponding sequence of ground-truth output tokens; and based on the customized Levenshtein distance: identifying the number of special token insertions for each corresponding output token sequence hypothesis; and identifying the number of special token deletions for each corresponding output token sequence hypothesis (Thomson, [0878]: "In some embodiments, the comparer 4504 may be configured to compare the monitored transcription with the reference transcription. In these and other embodiments, the comparer 4504 may compare the monitored transcription with the reference transcription by determining an edit distance or Levenshtein distance there between. In some embodiments, the comparison process by the comparer 4504 may be implemented as follows: (1) the comparer 4504 may align the monitored transcription and the reference transcription; (2) the comparer 4504 may compare each aligned pair of tokens from the monitored transcription and the reference transcription. The pair of tokens may include a first token from the monitored transcription and a second token from the reference transcription; (3) the comparer 4504 may provide an indication, such as a match or no match with respect to each aligned pair of tokens, to the counter 4506. For example, the comparer 4504 may output a zero when a pair of tokens match and a one if there is no match between a pair of tokens; and (4) the number of differences are counted or averaged by the counter 4506 to determine an average disagreement rate, edit distance, and/or Levenshtein distance.").
Dimitriadis, Sklyar, and Thomson are analogous arts because they each belong to the same field of natural language processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the personalized natural language processing system of Dimitriadis, as modified by the separator-transducer-segmenter of Sklyar, to incorporate the teachings of Thomson to use a levenshtein distance to judge the difference between training outputs and ground-truth data. This is a known way of comparing output data to ground-truth data (Thomson, [0878]). As such, the system is able to properly judge the accuracy of its outputs.
Regarding claim 20, the rejection of claim 19 is incorporated. Dimitriadis, in view of Sklyar and Thomson, discloses all of the elements of the current invention as stated above. Thomson further teaches wherein the customized Levenshtein distance determined between each corresponding output token sequence hypothesis and the corresponding sequence of ground-truth output tokens prevents the sequence transduction model from allowing substitutions between special tokens and common tokens during training of the sequence transduction model (Thomson, [0878]: "In some embodiments, the comparer 4504 may be configured to compare the monitored transcription with the reference transcription. In these and other embodiments, the comparer 4504 may compare the monitored transcription with the reference transcription by determining an edit distance or Levenshtein distance there between. In some embodiments, the comparison process by the comparer 4504 may be implemented as follows: (1) the comparer 4504 may align the monitored transcription and the reference transcription; (2) the comparer 4504 may compare each aligned pair of tokens from the monitored transcription and the reference transcription. The pair of tokens may include a first token from the monitored transcription and a second token from the reference transcription; (3) the comparer 4504 may provide an indication, such as a match or no match with respect to each aligned pair of tokens, to the counter 4506. For example, the comparer 4504 may output a zero when a pair of tokens match and a one if there is no match between a pair of tokens; and (4) the number of differences are counted or averaged by the counter 4506 to determine an average disagreement rate, edit distance, and/or Levenshtein distance."). The same motivation for claim 19 applies equally to claim 20.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYLER J BECKER whose telephone number is (703)756-1271. The examiner can normally be reached M-Th, 7:15am-5:45pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TYLER BECKER/ Examiner, Art Unit 2657
/DANIEL C WASHBURN/ Supervisory Patent Examiner, Art Unit 2657
