DETAILED ACTION
This communication is in response to the Amendments and Arguments filed on March 24, 2026. Claims 1-7 and 21-33 are pending and have been examined. Hence, this action has been made FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The reply filed on March 24, 2026 has been entered. Applicant’s arguments with respect to claims 1-7 and 21-33 have been considered but are not persuasive. Applicant’s arguments with respect to claim 6 has been considered but are moot in view of new ground(s) of rejection caused by the amendments.
With respect to the applicant’s arguments to claim rejections under 35 U.S.C § 101, Applicant has amended each of the independent claims and asserts that “As noted by Director Squires, "Examiners and panels should not evaluate claims at such a high level of generality." See Ex parte Desjardins at 9 and 10. However, the Office Action does just that by parsing portions of claim 1 to extract singular features or to generalize a complex, multi-step process.” (emphasis added) The examiner respectfully disagrees with these assertions. The applicant asserts that claim rejections under 35 USC 101 generalize a complex, multi-step process, but the supposed complex, multi-step process is nowhere to be found in the claim language. As amended, the independent claims merely disclose performing ASR and inserting a multi-frame blank in a sequence of audio frames with the intention to perform inferencing operations using a neural network. The rejection of claim 1 under 35 U.S.C. 101, as explained in detail below, clearly addresses both of these limitations. Further, there is no additional language in claim 1 that relates to a step-wise process for training the neural network using the modified sequence as the applicant asserts.
Applicant further asserts that “embodiments of the present disclosure are directed toward improvements to machine learning model training and deployment through the incorporation of training for the inclusion of multi-frame blanks.” The examiner respectfully disagrees with these assertions. With respect to improvements to machine learning model training, there is no such language in the independent claims that relate to training a neural network. The independent claims also lack claim language related to the “incorporation of training” as the applicant asserts.
Applicant further asserts that “there is a problem with current ASR systems. The specification and claims provide a solution to this technical field, namely, implementing multi-frame blank emissions to reduce a number of processed frames that are evaluated at inference.” The examiner respectfully disagrees with these assertions. It is not clear from the independent claim language how multi-frame blanks are actually integrated into the process of ASR in a way that explicitly provides the improvements set forth in the specification of the instant application. The independent claims as amended recite removing at least two audio frames from the input of a neural network, but fail to explain how this modified input interacts with the ASR process in a way that would bring forth the claimed improvement. Simply reciting the removal of frames without explaining how said removal integrates into the neural network or the ASR process cannot be considered valid justification for improving a technical field.
With respect to the applicant’s arguments to claim rejections under 35 U.S.C § 102, the applicant asserts that “a "valid part" of a token cannot reasonably be interpreted as an "omitted" audio frame. Indeed, Zhao directly indicates that "the inference result is 'AϕϕϕBϕϕsϕCϕϕ'. After removing blank tokens, the final result is 'AB s C', which matches the alignment of this utterance." See id at [0049]. Accordingly, any alleged removal is done after inference, not before.” The examiner respectfully disagrees with these arguments. Zhao et al. bears no language regarding a “valid part of a token.” Rather, ¶ [0050] of the Zhao et al. states that “in
y
2
, almost half of the valid part is blank, so that blank tokens dominate in the pre-training process. In contrast with the example label tensor 310 (
y
2
), the label tensor 320 (
y
3
) only retains the non-blank portion of
y
2
.” In other words, it is not the token in which Zhao et al. refers to as having a valid part, but the label tensor tasked with processing said tokens. Accordingly, by “only retain[ing] the non-blank portion
y
2
”, label tensor
y
3
discloses removing two or more blank portions from the input of a neural network before inferencing. To show that label tensor
y
3
performs inferencing, see ¶ [0047] of Zhao et al., “320 (also referred to herein as “
y
3
”) represent[s] a one-hot vector and [is] based on the 8-frame utterance ‘A B s C’ with the alignment ‘A A A B B s C C’ shown in FIG. 3A. … In each of the three label tensors shown in FIG. 3B, the horizontal axis represents the time dimension from left to right, and the vertical axis represents the output token dimension from top to bottom.” Computing tokens sequentially over a period of time is considered analogous to inferencing.
The applicant further asserts that “claims 2, 7, 22, 27, and 29 are allowable at least for depending from an allowable independent claim. In addition, Applicant respectfully submits that at least some of claims 2, 7, 22, 27, and 29 additionally recite patentable subject matter not taught by Zhao.” The examiner respectfully disagrees. As explained above, independent claim 1 does not comprise allowable subject matter. Further, the Applicant states that at least some of claims 2, 7, 22, 27, and 29 additionally recite patentable subject matter not taught by Zhao, but fails to indicate or explain said patentable subject matter.
With respect to the applicant’s arguments to claim rejections under 35 U.S.C § 103, the applicant asserts that “Claims 3-5, 23-25, and 30-32 each depend, directly or indirectly, from one of claims 1, 21, and 28 described above. Accordingly, Applicant respectfully submits that claims 3-5, 23-25, and 30-32 are allowable at least for depending from an allowable independent claim. In addition, Applicant respectfully submits that at least some of claims 3-5, 23-25, and 30-32 additionally recite patentable subject matter not taught or otherwise rendered obvious by Zhao and Graves, individually or in combination.” The examiner respectfully disagrees. As explained above, independent claim 1 does not recite allowable subject matter. Further, the Applicant states that at least some of claims3-5, 23-25, and 30-32 additionally recite patentable subject matter not taught or rendered obvious by Zhao in view of Graves, but fails to indicate or explain said patentable subject matter.
The applicant’s arguments with respect to claim 6 have been considered but are moot in view of new ground(s) of rejection caused by the amendments.
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required:
Claim 26, line 3: “configuration file” lacks antecedent basis from specification
Claim 33, line 3: “configuration file” lacks antecedent basis from specification
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
The following terms in the claims have been given the following interpretations in light of the specification:
Forward/backward weight: paragraph [0033], “In certain embodiments, a forward-backward algorithm is incorporated with an RNN-T that includes both forward weights (
a
) and backward weights (
β
), which are represented as:
a
t
,
u
=
a
t
-
1
,
u
Ø
t
-
1
,
u
+
a
t
,
u
-
1
y
t
,
u
-
1
,
β
(
t
,
u
)
=
β
(
t
+
1
,
u
)
Ø
(
t
,
u
)
)
+
β
(
t
,
u
+
1
)
y
(
t
,
u
)
”
Thus, a forward/backward weight is the forward/backward function utilized in a forward-backward algorithm for training an RNN-T model. This definition is used for purposes of searching for prior art, but cannot be incorporated into the claims.
Should applicant wish different definitions, Applicant should point to the portions of the specification that clearly show a different definition.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7 and 21-33 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. All of the claims are method claims (1-7), apparatus/machine claims (21-33) or manufacture claim under (Step 1), but under Step 2A all of these claims recite abstract ideas and specifically mental processes. These mental processes are more particularly recited in claims 1, 21, and 28 as:
performing, using a neural network (NN), one or more automatic speech recognition (ASR) operations with respect to a sequence of audio frames…
Under Step 2A Prong One, claims 1, 21, and 28 are directed to an abstract idea and specifically a mental process. As detailed above, the steps of “performing…” may be practically performed in the human mind with the use of a physical aid such as a pen and paper. For example, a human could receive a sequence of audio frame values, modify the sequence of audio frames by replacing multiple sequential audio frames with a multi-frame blank, and recognize speech from the modified sequence of audio frames by simulating a neural network algorithm using pen and paper.
Under Step 2A Prong Two, this judicial exception is not integrated into a practical application because claims 1-7 and 21-33 do not recite additional elements that integrate the exception into a practical application. In particular, claims 1, 21, and 28 recite the additional elements of a processor (¶ [0104]) and a neural network (¶ [0018]). These additional elements are recited at a high level of generality and merely equate to “apply it” or otherwise merely uses a generic computer as a tool to perform an abstract which are not indicative of integration into a practical application as per MPEP 2106.05(f). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Under Step 2B, the claims do not recite additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using a computer is noted as a general computer {processor (¶ [0104]); neural network (¶ [0018])}. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are not patent eligible.
With respect to claims 2, 22, and 29, the claim relates to training a neural network with a probability lattice. This relates to a human using a probability lattice in their neural network simulation to formally recognize speech from the sequence of audio frames. The additional limitation of a “neural network” is recited at a high level of generality (¶ [0018]) and merely equates to “apply it” or otherwise merely uses a generic computer as a tool to perform an abstract which are not indicative of integration into a practical application as per MPEP 2106.05(f). No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claims 3, 23, and 30, the claim relates to a neural network comprising an RNN-T, and the RNN-T comprising one of a forward weight, backward weight, or multi-frame probability. This relates to a human following the structure of an RNN-T to update the weights of their simulated neural network process. The additional limitations of a “neural network” and a “recurrent neural network transducer” are recited at a high level of generality and merely equate to “apply it” (¶ [0018]-[0019]) or otherwise merely uses a generic computer as a tool to perform an abstract which are not indicative of integration into a practical application as per MPEP 2106.05(f). No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claims 4, 24, and 31, the claim relates to omitting a multi-frame probability for boundary conditions. This relates to a human ignoring a multi-frame blank if it comes at the beginning or end of a recognized word of speech. No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claims 5, 25, and 32, the claim relates to specifying the relative values of a neural network output. This relates to a human automatically assigning any multi-frame blank token a probability value of 1 during their neural network simulation. The additional limitation of a “neural network” is recited at a high level of generality (¶ [0018]) and merely equates to “apply it” or otherwise merely uses a generic computer as a tool to perform an abstract which are not indicative of integration into a practical application as per MPEP 2106.05(f). No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claims 6, the claim relates to training an NN using an under normalized output distribution. This limitation is directed towards mathematical concept which is not patentable subject matter according to MPEP 2106.04(a)(2)(I). The additional limitation of a “neural network” is recited at a high level of generality (¶ [0018]) and merely equates to “apply it” or otherwise merely uses a generic computer as a tool to perform an abstract which are not indicative of integration into a practical application as per MPEP 2106.05(f). No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claims 7 and 27, the claim relates to outputting probability values for at least two multi-frame blanks that differ in their number of frames. This relates to a human estimating their confidence in their recognized speech output, their recognized speech output comprising at least two multi-frame blanks that differ in the number of frames each multi-frame blank comprises. The additional limitation of a “neural network” is recited at a high level of generality (¶ [0018]) and merely equates to “apply it” or otherwise merely uses a generic computer as a tool to perform an abstract which are not indicative of integration into a practical application as per MPEP 2106.05(f). No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claims 26 and 33, the claim relates to configuring a multi-frame blank to comprise a set number of frames. This relates to a human setting a constant number of frames for any given multi-frame blank when deciding what frames to replace in the sequence of audio frames. No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
For all of the above reasons, taken alone or in combination, claims 1-7 and 21-33 recite a non-statutory mental process.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 7, 21-22, and 27-29 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by US Patent Publication 20210312905 A1, (Zhao et al.).
Claim 1
Regarding claim 1, Zhao et al. disclose a method, comprising:
performing, using a neural network (NN) (Zhao et al. ¶ [0024], "Techniques for utilizing external alignments to pre-train RNN-T models are provided."), one or more automatic speech recognition (ASR) operations with respect to a sequence of audio frames (Zhao et al. ¶ [0079], "The speech processing module 1215 may be configured to perform automatic speech recognition (ASR) on the audio data to output textual content representing spoken content included in the audio data."), wherein an output of the NN corresponding to at least one audio frame of the sequence of audio frames corresponds to a multi- frame blank (Zhao et al. ¶ [0050], "In order to provide the blank information during the pre-training stage, a short pause (space token less than 3 frames) of each utterance is set to blank."), the multi-frame blank causing two or more subsequent audio frames of the sequence of audio frames to be omitted from an input provided to the NN (Zhao et al. ¶ [0050], "In contrast with the example label tensor 310 (
y
2
), the label tensor 320 (
y
3
) only retains the non-blank portion of
y
2
. ... a short pause (space token less than 3 frames) of each utterance is set to blank. That means some space in the valid part of the label tensor will become blank. Thus, a part of the alignment path in the label matrix becomes blank." See Figure 3C, component 320, which illustrates omitting multi-frame blanks from an input provided to the NN) to perform one or more inferencing operations using the NN (Zhao et al. ¶ [0079], "The speech processing module 1215 may be configured to perform automatic speech recognition (ASR) on the audio data to output textual content representing spoken content included in the audio data." ASR is considered analogous to an inferencing operation).
Claim 2
Regarding claim 2, the rejection of claim 1 is incorporated. Zhao et al. further disclose wherein the NN is trained using a probability lattice representing a probability of interpretations corresponding to a training sequence of input audio frames (Zhao et al. ¶ [0045], "FIGS. 3B and 3C show examples of creating three-dimensional token-aligned training data that may be used in whole-network training of the RNN-T. The training data represents utterances that are aligned with frame boundaries of the frames of audio data associated with the utterance. ... One way to address the problem of whole-network training is to only compute the CE for the alignment path of the label matrix." ¶ [0037], "The CE loss represents the entropy or difference between a reference probability distribution for a particular input and the actual probability distribution output by the model." See Figs. 3B and 3C, which illustrate examples of probability lattices).
Claim 7
Regarding claim 7, the rejection of claim 1 is incorporated. Zhao et al. further disclose wherein the NN outputs a probability or confidence value for a plurality of output tokens (Zhao et al. ¶ [0033], "A forward-backward algorithm is executed on the three-dimensional output from the Softmax operation 120 to compute the total probability
P
(
y
|
x
)
of the output sequence
y
, conditioned on the input sequence
x
."), the plurality of output tokens including a first multi-frame blank token corresponding to a first number of frames and a second multi-frame blank token corresponding to a second number of frames different from the first number of frames (Zhao et al. ¶ [0049], "Thus, by directly performing the decoding on
y
2
of the given example, the inference result is ‘AϕϕϕBϕϕsϕCϕϕ’." "ϕϕϕ" is considered analogous to a first multi-frame blank. "ϕϕ" is considered analogous to a second multi-frame blank.).
Claim 21
Regarding claim 21, Zhao et al. disclose a system, comprising:
at least one processor (Zhao et al. ¶ [0092], "In some examples, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules.") to:
generate an output, for at least one audio frame of a sequence of audio frames, corresponding to a multi-frame blank, the multi-frame blank including two or more subsequent audio frames of the sequence of audio frames (Zhao et al. ¶ [0050], "In order to provide the blank information during the pre-training stage, a short pause (space token less than 3 frames) of each utterance is set to blank."); and
perform, using a neural network (NN) (Zhao et al. ¶ [0024], "Techniques for utilizing external alignments to pre-train RNN-T models are provided."), one or more automatic speech recognition operations on the sequence of audio frames (Zhao et al. ¶ [0079], "The speech processing module 1215 may be configured to perform automatic speech recognition (ASR) on the audio data to output textual content representing spoken content included in the audio data.") and to omit the two or more subsequent audio frames associated with the multi-frame blank from an input provided to the NN (Zhao et al. ¶ [0050], "In contrast with the example label tensor 310 (
y
2
), the label tensor 320 (
y
3
) only retains the non-blank portion of
y
2
. ... In order to provide the blank information during the pre-training stage, a short pause (space token less than 3 frames) of each utterance is set to blank. That means some space in the valid part of the label tensor will become blank. Thus, a part of the alignment path in the label matrix becomes blank." See Figure 3C, component 320, which illustrates omitting multi-frame blanks from an input to an NN) to perform one or more inferencing operations during the one or more automatic speech recognition operations ( (Zhao et al. ¶ [0079], "The speech processing module 1215 may be configured to perform automatic speech recognition (ASR) on the audio data to output textual content representing spoken content included in the audio data.").
Claim 22
Regarding claim 22, the rejection of claim 21 is incorporated. The limitations of claim 22 are similar to the limitations of claim 2 and therefore are rejected for similar reasons as described above.
Claim 27
Regarding claim 27, the rejection of claim 21 is incorporated. The limitations of claim 27 are similar to the limitations of claim 7 and therefore are rejected for similar reasons as described above.
Claim 28
Regarding claim 28, Zhao et al. disclose a processor, comprising:
one or more processing units to perform (Zhao et al. ¶ [0092], "In some examples, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules."), using a neural network (NN) (Zhao et al. ¶ [0024], "Techniques for utilizing external alignments to pre-train RNN-T models are provided."), one or more automatic speech recognition (ASR) operations to a sequence of audio frames (Zhao et al. ¶ [0079], "The speech processing module 1215 may be configured to perform automatic speech recognition (ASR) on the audio data to output textual content representing spoken content included in the audio data."), at least one output of the NN corresponding to a multi-frame blank associated with at least one audio frame of the sequence of audio frames (Zhao et al. ¶ [0050], "In order to provide the blank information during the pre-training stage, a short pause (space token less than 3 frames) of each utterance is set to blank."), and omitting, from an input provided to the NN to perform one or more inferencing operations for the one or more ASR operations (Zhao et al. ¶ [0079], "The speech processing module 1215 may be configured to perform automatic speech recognition (ASR) on the audio data to output textual content representing spoken content included in the audio data." ASR is considered analogous to an inferencing operation), two or more subsequent audio frames of the sequence of audio frames associated with the multi-frame blank (Zhao et al. ¶ [0050], "In contrast with the example label tensor 310 (
y
2
), the label tensor 320 (
y
3
) only retains the non-blank portion of
y
2
. ... In order to provide the blank information during the pre-training stage, a short pause (space token less than 3 frames) of each utterance is set to blank. That means some space in the valid part of the label tensor will become blank. Thus, a part of the alignment path in the label matrix becomes blank." See Figure 3C, component 320, which illustrates omitting multi-frame blanks from an input before inferencing).
Claim 29
Regarding claim 29, the rejection of claim 28 is incorporated. The limitations of claim 29 are similar to the limitations of claim 2 and therefore are rejected for similar reasons as described above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-5, 23-25, and 30-32 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 20210312905 A1 (Zhao et al.) in view of "Sequence Transduction with Recurrent Neural Networks" (Graves).
Claim 3
Regarding claim 3, the rejection of claim 1 is incorporated. Zhao et al. disclose all the elements of the claimed invention as stated above. Zhao et al. further disclose wherein the NN comprises a recurrent neural network transducer (RNN-T) (Zhao et al. ¶ [0024], "Techniques for utilizing external alignments to pre-train RNN-T models are provided."), wherein one or more parameters of the RNN-T are updated based at least on one or more loss functions (Zhao et al. ¶ [0033], "A forward-backward algorithm is executed on the three-dimensional output from the Softmax operation 120 to compute the total probability
P
(
y
|
x
)
of the output sequence
y
, conditioned on the input sequence
x
. The negative log-loss of the target sequence is used as the object function to train the model" See Equation 3).
Zhao et al. do not explicitly disclose all of weights.
However, Graves discloses wherein the NN comprises a recurrent neural network transducer (RNN-T) (Graves pg. 2, Section 1, Paragraph 7, "Section 2 defines the RNN transducer, showing how it can be trained and applied to test data"), wherein one or more parameters of the RNN-T are updated based at least on one or more loss functions (Graves pg. 4, Section 2.5, Paragraph 1, "Given an input sequence
x
and a target sequence
y
*
, the natural way to train the model is to minimise the log-loss
L
=
-
l
n
P
r
(
y
*
|
x
)
of the target sequence.") comprising at least one of: one or more forward weights (Graves pg. 3, Section 2.4, Paragraph 1, "Define the forward variable
α
(
t
,
u
)
as the probability of outputting
y
[
1
:
u
]
during
f
[
1
:
t
]
. The forward variables for all
1
≤
t
≤
T
and
0
≤
u
≤
U
can be calculated recursively using
α
(
t
,
u
)
=
α
(
t
-
1
,
u
)
∅
(
t
-
1
,
u
)
+
α
(
t
,
u
-
1
)
y
(
t
,
u
-
1
)
" Forward variable
α
is considered analogous to a forward weight. See claim interpretation section), one or more backward weights (Graves pg. 3, Section 2.4, Paragraph 2, "Define the backward variable
β
(
t
,
u
)
as the probability of outputting
y
[
u
+
1
:
U
]
during
f
[
t
:
T
]
. Then
β
(
t
,
u
)
=
β
(
t
+
1
,
u
)
∅
(
t
,
u
)
+
β
(
t
,
u
+
1
)
y
(
t
,
u
)
with initial condition
β
(
T
,
U
)
=
∅
(
T
,
U
)" Backward variable
β
is considered analogous to a backward weight. See claim interpretation section), or a multi-frame probability.
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify Zhao et al.’s RNN-T model to include Graves’ forward-backward algorithm because such a modification is the result of simple substitution of one known element for another producing a predictable result. More specifically, Zhao et al.’s forward-backward algorithm and Graves’ forward-backward algorithm perform the same general and predictable function, the predictable function being updating the RNN-T. Since each individual element and its function are shown in the prior art, albeit shown in separate references, the difference between the claimed subject matter and the prior art rests not on any individual element or function but in the very combination itself - that is in the substitution of Zhao et al.’s forward-backward algorithm by replacing it with Graves’ forward-backward algorithm. Thus, the simple substitution of one known element for another producing a predictable result renders the claim obvious.
Claim 4
Regarding claim 4, the rejection of claim 3 is incorporated. Zhao et al. in view of Graves disclose all the elements of the claimed invention as stated above. Zhao et al. further disclose wherein the multi-frame probability is omitted for boundary conditions (Zhao et al. ¶ [0049], "Thus, by directly performing the decoding on
y
2
of the given example, the inference result is ‘A ϕϕϕBϕϕsϕCϕϕ’. After removing blank tokens, the final result is ‘A B s C’, which matches the alignment of this utterance." Removing multi-frame blanks (denoted by two or more ϕ) is considered analogous to omitting multi-frame probabilities for boundary conditions).
Claim 5
Regarding claim 5, the rejection of claim 1 is incorporated. Zhao et al. further disclose wherein the output of the NN includes values corresponding to a plurality of output tokens (Zhao et al. ¶ [0033], "A forward-backward algorithm is executed on the three-dimensional output from the Softmax operation 120 to compute the total probability
P
(
y
|
x
)
of the output sequence
y
, conditioned on the input sequence
x
.").
Zhao et al. do not explicitly disclose a blank token corresponding to the highest value in an output.
However, Graves discloses wherein the output of the NN includes values corresponding to a plurality of output tokens (Graves pg. 5, Section 2.6, Paragraph 2, "Let
P
r
(
y
)
be the approximate probability of emitting some output sequence
y
found by the search so far. ... Pseudocode for a width
W
beam search for the output sequence with highest length-normalised probability given some length
T
transcription sequence is given in Algorithm 1."), and further wherein a highest value of the values of the output corresponds to a [multi-frame] blank token of the output tokens (Graves pg. 2, Section 2, Paragraph 1-2, “Let
y
=
(
y
1
,
y
2
,
…
,
y
U
)
be a length
U
output sequence belonging to the set
Y
*
of all sequences over some output space
Y
. … Define the extended output space
Y
-
as
Y
∪
∅
, where
∅
denotes the null output. The intuitive meaning of
∅
is ‘output nothing,’” pg. 3, Section 2.3, Paragraph 1, “all possible input-output alignments are assigned a probability” pg. 5, Section 2.6, Algorithm 1, "
P
r
(
∅
)
=
1
" Graves outputs a blank token, which is assigned a probability value of 1. A probability value of 1 is considered analogous to the highest value of the values output).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify Zhao et al.’s RNN-T model to incorporate Graves RNN-T model output.
The suggestion/motivation for doing so is similar to the suggestion/motivation described above with respect to claim 3.
Claim 23
Regarding claim 23, the rejection of claim 21 is incorporated. The limitations of claim 23 are similar in scope to that of claim 3 and therefore are rejected for similar reasons as described above.
Claim 24
Regarding claim 24, the rejection of claim 23 is incorporated. The limitations of claim 24 are similar in scope to that of claim 4 and therefore are rejected for similar reasons as described above.
Claim 25
Regarding claim 25, the rejection of claim 21 is incorporated. The limitations of claim 25 are similar in scope to that of claim 5 and therefore are rejected for similar reasons as described above.
Claim 30
Regarding claim 30, the rejection of claim 28 is incorporated. The limitations of claim 30 are similar in scope to that of claim 3 and therefore are rejected for similar reasons as described above.
Claim 31
Regarding claim 31, the rejection of claim 30 is incorporated. The limitations of claim 31 are similar in scope to that of claim 4 and therefore are rejected for similar reasons as described above.
Claim 32
Regarding claim 32, the rejection of claim 28 is incorporated. The limitations of claim 32 are similar in scope to that of claim 5 and therefore are rejected for similar reasons as described above.
Claim 6 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 20210312905 A1 (Zhao et al.) in view of “Alleviating ASR Long-Tailed Problem by Decoupling the Learning of Representation and Classification” (Deng et al.).
Claim 6
Regarding claim 6, the rejection of claim 1 is incorporated. Zhao et al. disclose all the elements of the claimed invention as stated above.
Zhao et al. disclose wherein the NN is trained using an output distribution that is [under] normalized to encourage multi-frame blank inclusion (Zhao et al. ¶ [0033], "A forward-backward algorithm is executed on the three-dimensional output from the Softmax operation 120 to compute the total probability
P
(
y
|
x
)
of the output sequence
y
, conditioned on the input sequence
x
. The negative log-loss of the target sequence is used as the object function to train the model" ¶ [0065]-[0066], "generating the three-dimensional ground truth label may include identifying a space token representing a pause in the utterance and replacing that space token with a blank token to facilitate training of the RNN-T." A softmax is considered analogous to normalizing).
Zhao et al. do not explicitly disclose all of an under normalized output distribution.
However, Deng et al. disclose an output distribution that is under normalized to encourage [multi-frame] blank inclusion (Deng pg. 346, Section (III)(D)(1), Paragraph 1, "The attention-based [weighted softmax] is designed for the inference of attention-based ASR models, with the specific process illustrated as follows: [Eqn. 33, 34] where
Π
=
π
1
,
…
,
π
U
,
π
m
is the frequency of vocabulary’s
m
-th token appearing in training data transcripts,
α
is a hyperparameter, and
π
n
-
α
and
S
t
n
respectively represent the weight and probability of the
n
-th token at
t
step. For tokens that have not appeared in the training data transcripts, such as
[
b
l
a
n
k
]
, we set its corresponding weight to 1." pg. 345, Section (III)(B)(3), Paragraph 4, "Due to large token size, the
π
i
of each token is very small." Weighting non-blank tokens to a lower value is considered analogous to under normalizing. Under normalizing non-blank tokens while allowing blank tokens the maximum weight value is considered analogous to encouraging blank inclusion).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify Zhou et al.’s ASR recognition system to incorporate Deng et al.’s weighted softmax.
The suggestion/motivation for doing so would have been that, “[weighted softmax] does improve the ASR accuracy for tail tokens, though HWER1% / HCER5% may increase. Therefore, a proper intermediate value of α can provide a good trade-off between TWER1% / TCER5% and HWER1% / HCER5%, so as to improve the overall recognition ability, that is, to reduce WER/CER,” as noted by Deng et al. in pg. 351, Section (VI)(F), Paragraph 3.
Claims 26 and 33 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 20210312905 A1 (Zhao et al.) in view of US Patent Publication 20230360653 A1 (Yan et al.). Yan et al. claims a priority date of 05/04/2022 for provisional application 63/338,159.
Claim 26
Regarding claim 26, the rejection of claim 21 is incorporated. Zhao et al. disclose all the elements of the claimed invention as stated above.
Zhao et al. do not explicitly disclose all of specifying a number of contiguous frames corresponding to a multi-frame blank.
However, Yan et al. discloses wherein a number of contiguous audio frames corresponding to the multi-frame blank is specified within a configuration file (Yan et al. ¶ [0028], "A constant number c is configured to indicate the toleration of contiguous silence frames. ... The tolerance number c is set to 25, which means that if there are contiguous 25 frames below the threshold, the upcoming frames will be discarded until a higher power frame arrives" App. 63/337,159, Page 9, Paragraph [0027], “A constant number c is configured to indicate the toleration of contiguous silence frames. … The tolerance number c is set to 25, which means that if there are contiguous 25 frames below the threshold, the upcoming frames will be discarded until a higher power frame arrives”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify Zhao et al.’s RNN-T model to include Yan et al.’s configurable multi-frame blank because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, Zhao et al.’s RNN-T model as modified by Yan et al.’s configurable multi-frame blank can yield a predictable result of providing more user control since the ability to configure the number of frames that comprise a multi-frame blank would allow a user to experiment with which configurable number might provide the best model performance. Thus, a person of ordinary skill would have appreciated including in Zhao et al.’s RNN-T mode the ability to do Yan et al.’s multi-frame blank configuration since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim 33
Regarding claim 33, the rejection of claim 28 is incorporated. The limitations of claim 33 are similar in scope to that of claim 26 and therefore are rejected for similar reasons as described above.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
“Mitigating Neural Network Overconfidence with Logit Normalization” to Wei et al. discloses normalizing logits in a way that reduces the magnitude of each logit during normalization. This is similar to the claimed process in claim 6, i.e. under normalization of an output distribution.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACOB B VOGT whose telephone number is (571)272-7028. The examiner can normally be reached Monday - Friday 9:30am - 7pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D Shah can be reached at (571)270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACOB B VOGT/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
04/16/2026
Prosecution Insights