Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. Examiner recommends further wording in the title towards the type of speech decoding performed.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1,3-8,15-17 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Moritz et al (20200312306).
As per claims 1,15, Moritz et al (20200312306) teaches a speech recognition method (para 0013 – speech recognition system producing a transcription of an utterance – para 0013), comprising:
obtaining a first speech feature of a speech to-be-recognized, wherein the first speech feature comprises a plurality of speech segment features corresponding to a plurality of speech segments in the speech to-be-recognized (as, processing speech features by using an alignment based decoder – see para 0014, into para 0015, explaining the alignment network and then the alignment decoding; as to claim 15 – using speech recognition to process large input sequences – para 0041);
decoding the first speech feature using a first decoder to obtain a plurality of first decoding results corresponding to a plurality of words in the speech to-be-recognized, wherein each first decoding result of the plurality of first decoding results indicates a first recognition result of a word corresponding to the first decoding result (as performing a first decoding process using an alignment decoder – para 0016);
extracting a second speech feature from the first speech feature based on first a priori information, wherein the first a priori information comprises the plurality of first decoding results (as using a priori knowledge for joint training, and using the initial results of the a priori knowledge as anchor points to find a sequence of input frames – see para 0015; see para 0016, wherein the a priori knowledge comes from the alignment decoder, and that information is fed into the attention-based decoder, for a ‘second-pass’ at decoding),
and the second speech feature comprises a plurality of first word-level audio features corresponding to the plurality of words (as, the decoded results of the attention based decoder are representations of the input acoustic signal – para 0038);wherein the extracting comprises:
for each word of the plurality of words, performing an attention mechanism calculation by using the first decoding result corresponding to the word as a query feature Q (as, the first decoding result is from the alignment decoder which is fed into the attention based decoder – see para 0015, explaining the alignment process/results, and then see last half of para 0015, wherein the attention decoder uses the alignment information from the alignment decoder, to generate a transcript of the input) and using the first speech feature as a key feature K (as feature values for the acoustic signal – para 0036) and a value feature V, to obtain the first word level audio feature corresponding to the word (as further using, the attention based probabilities to improve the accuracy of the label sequence – para 0087 into para 0088);
and decoding the second speech feature using a second decoder to obtain a plurality of second decoding results corresponding to the plurality of words, wherein each second decoding result of the plurality of second decoding results indicates a second recognition result of a word corresponding to the second decoding result (as, generating a second transcription word recognition result based on the attention based decoder – para 0045).
As per claim 3, Moritz et al (20200312306) teaches the method of claim 2, further comprises:
before decoding the second speech feature, performing global encoding on the plurality of first word-level audio features (as using alignment encoder to produce encoder states – first half of para 0013) corresponding to the plurality of words using a second encoder to enhance the second speech feature (and following with an attention based neural network to determine the transcription representation based on encoder states from the alignment network – last half of para 0013).
As per claim 4, Moritz et al (20200312306) teaches the method of claim 1, wherein decoding the second speech feature using the second decoder to obtain the plurality of second decoding results corresponding to the plurality of words comprises:
for each of the plurality of words, using the first decoding result corresponding to the word as a query feature of the second decoder (as, additionally noted above, the alignment decoder outputs a result as a prompt to the attention based neural network decoder – para 0013-0015, as well as figure 1, alignment decoder output into the attention based decoder using partition module information derived from the alignment decoder input),
and using the second speech feature as a key feature and a value feature of the second decoder to obtain the second decoding result corresponding to the word output by the second decoder (as, using the partition information, from the alignment decoder, as a key feature and value -- para 0053, for the attention based decoder neural network, and outputting a transcript (words) – see fig. 3, output 316).
As per claim 5, Moritz et al (20200312306) teaches the method of claim 4, wherein the second decoder comprises a forward decoder and a backward decoder, the forward decoder and the backward decoder are both configured to: for each word of the plurality of words, use the first decoding result of the word as a query feature for input, and use the second speech feature as a key feature and a value feature for input, wherein the forward decoder is configured to apply a left-to-right temporal mask to input features, and the backward decoder is configured to apply a right-to-left temporal mask to input features (as, the neural networks implemented, can be bidirectional LSTM’s – one is a forward LSTM and the other is a backward LSTM – see para 0075; as to the first/second decoding steps with a query feature, second speech feature – see mappings above in claims 1-4).
As per claim 6, Moritz et al (20200312306) teaches the method of claim 5, wherein for each of the plurality of words, using the first decoding result of the word as the query feature of the second decoder and using the second speech feature as the key feature and the value feature of the second decoder to obtain the second decoding result corresponding to the word output by the second decoder (see mapping above, in claims 1-5) comprises:
fusing a plurality of forward decoding features corresponding to the plurality of words output by the forward decoder and a plurality of backward decoding features corresponding to the plurality of words output by the backward decoder to obtain a plurality of fusion features corresponding to the plurality of words (as generating a hidden vector, as the output of the bidirectional LSTM (BLSTM) is obtained as a concatenations of hidden vectors of the forward and backward LSTM’s – see last half of para 0075; this concatenation is mathematically explained in para 0078);
and obtaining the plurality of second decoding results based on the plurality of fusion features (as outputting the transcription – fig. 3, subblock 316).
As per claim 7, Moritz et al (20200312306) teaches the method of claim 4, further comprises:
for each word of the plurality of words, using N.sub.th decoding result of the word as a query feature of the second decoder, and using the second speech feature as a key feature and a value feature of the second decoder to obtain N+1.sub.th decoding result corresponding to the word output by the second decoder, wherein N is an integer greater than or equal to 2 (as, using the previous state alignment vectors for the current vector – see para 0083, after equation 27…the “next state” corresponds to the claimed “N+1”).
As per claim 8, Moritz et al (20200312306) teaches the method of claim 1, further comprises:
extracting a third speech feature from the first speech feature based on second a prior information, wherein the second a prior information comprises the plurality of second decoding results (as, in an alternate embodiment, and as an intermediate step (compared to the mappings above to claims 1-7), generating a higher representation of the input acoustic signal as an alternative sequence of the states of the acoustic information – para 0038; in other words, the embodiment mapped to claims 1-7, is, acoustic features [Wingdings font/0xE0] alignment features [Wingdings font/0xE0] decoded transcript; para 0038 teaches acoustic features [Wingdings font/0xE0] further acoustic representation [Wingdings font/0xE0] alignment features [Wingdings font/0xE0] decoded transcript),
and the third speech feature comprises a plurality of second word-level audio features corresponding to the plurality of words; and decoding the third speech feature using the second decoder to obtain a plurality of third decoding results corresponding to the plurality of words (as, using alternative representations of the acoustic signal, and input that information into the attention based decoder to generate a transcript – see end of para 0038),
wherein each third decoding result of the plurality of third decoding results indicates a third recognition result of a word corresponding to the second decoding result (as generating a character sequence, as the first decoding step and then, generating words from the character sequence – see para 0065).
Claim 16 is a method claim that performs the steps found throughout claims 1-8 above and as such, claim 16 is similar in scope and content to claims 1-8 above and therefore, claim 16 is rejected under similar rationale as presented against claims 1-8 above. Furthermore, as to the ground-truth information as a data source in claim 16, examiner notes that Moritz et al (20200312306) teaches the use of data resources for training a speech recognition network can be labeled/unlabeled/sound-transcription relational databases – para 0098, 0099. Examiner notes, that it is old and notoriously well known in the art of end-to-end speech recognition systems to access ground-truth type databases to increase the probability of accurate data training, at the cost of a premium for the ‘ground-truth’ labeling.
Claim 17 is a device claim that performs the steps found throughout claims 1-8 above and as such, claim 17 is similar in scope and content to claims 1-8 above and therefore, claim 17 is rejected under similar rationale as presented against claims 1-8 above. Furthermore, as to claim 17, Moritz et al (20200312306) teaches a processor executing stored/memory instructions to perform the disclosed method in Moritz et al (20200312306) – see para 0023.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 9-14 are rejected under 35 U.S.C. 103 as being unpatentable over Moritz et al (20200312306) in view of Rao et al (20220262350).
As per claim 9, Moritz et al (20200312306) teaches the method of claim 1 (see mapping above to claim 1), wherein obtaining the first speech feature of the speech to-be-recognized comprises:
obtaining an original speech feature of the speech to-be-recognized (as processing speech for acoustic features – para 0013 – 0015);
and truncating the original speech feature to obtain the plurality of speech segment features (as fixing the length of the partitions – para 0053; examiner notes, that the partition is determined by the CTC based alignment decoder – see para 0015, 0019),
Moritz et al (20200312306) teaches the use of a CTC neural network for the alignment decoding, as noted above (and applied throughout to claims 1-7 above); however, Moritz et al (20200312306) does not explicitly state the well known use of spikes in CTC processing to mark the areas of common overlap of features; Rao et al (20220262350) teaches CTC acoustic processing performing subword alignments (para 0005) using determined spikes to generate alignment of the features (see para 0038, “the statistics associated with the phone label spikes…”). Therefore, it would have been obvious to one of ordinary skill in the art of CTC based acoustic processing to further define the bidirectional LSTM model of Moritz et al (20200312306) to call the statistical information as spikes in tracking the alignment information, as taught by Rao et al (20220262350), because it would advantageously show in a more efficient manner, the location in the sequence that the subword (phone alignment) is most appropriate (see Rao et al (20220262350) , para 0040).
As per claim 10, the combination of Moritz et al (20200312306) in view of Rao et al (20220262350) teaches the method of claim 9, wherein the plurality of speech segment features are sequentially obtained by performing streaming truncation (see Moritz et al (20200312306), fixing the length of the partition – para 0053) on the original speech feature, and decoding the first speech feature using the first decoder comprises: sequentially performing streaming decoding on the plurality of speech segment features using the first decoder (as sequential processing of the acoustic signal in a streaming fashion -- Moritz et al (20200312306), para 0067).
As per claims 11,12, examiner notes that the claim features in claims 11,12 are similar to claim features found throughout in claims 1-8 and as such, claims 11,12 are similar in scope and content to the claims features in claims 1-8; therefore, these commonly found claim features in claims 11,12 are rejected under similar rationale as presented against claims 1-8 above using the Moritz et al (20200312306) reference. As a further explanation, examiner notes the ‘historical’ claim feature in claims 11,12 mirror the “a priori” knowledge that is used in Moritz et al (20200312306) e.g., para 0015; and that the claimed “concatenation” is the “fused” feature found above; and that the combination of the vectors for the forward-backward bi-directional LSTM match this claim elements, as well as, the CTC based alignment decoder output is the prompt to the attention based decoder, as explained in detail in claims 1-8 above.
As per claim 13, the combination of Moritz et al (20200312306) in view of Rao et al (20220262350) teaches the method of claim 9, wherein truncating the original speech feature to obtain the plurality of speech segment features comprises: truncating the original speech feature based on a predetermined time length, (see Moritz et al (20200312306), fixing the length of the partition – para 0053) , and using the speech segment feature of the speech segment in which each spike of the plurality of spikes is located as the speech segment feature corresponding to the spike (Rao et al (20220262350), see para 0038, “the statistics associated with the phone label spikes…”).
As per claim 14, the combination of Moritz et al (20200312306) in view of Rao et al (20220262350) teaches the method of claim 9, wherein truncating the original speech feature to obtain the plurality of speech segment features comprises: truncating the original speech feature ( see Moritz et al (20200312306), fixing the length of the partition – para 0053)
based on the plurality of spikes, and using the speech segment feature of the speech segment between every two adjacent spikes as the speech segment feature corresponding to one of the two adjacent spikes (Rao et al (20220262350), see para 0038, “the statistics associated with the phone label spikes…”; furthermore, see Figure 2 of Rao et al showing the spike locations of the phone labels).
Response to Arguments
Applicant's arguments filed 6/23/2026 have been fully considered but they are not persuasive. As per applicants arguments against the Moritz et al (20200312306) reference regarding apriori knowledge, examiner argues that the alignment procedure operates on words – see para 0041, “encoder states in the sequence…such as characters, bits, words, etc.”; which also meets applicants concern regarding “word-level audio features”. As per applicants arguments against the amended claim language, examiner notes the recited sections to Moritz et al (20200312306), as shown above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see related art listed on the PTO-892 form.
Furthermore, the following references were found that match certain features in the claims/specification:
Wieman et al (20210256386) teaches CTC acoustic feature processing operating on sub-word levels for further recognition (para 0043), using ground -truth basis and attention layers in the neural network architecture.
Hori (20190189115) teaches the use of historical word information with fwd/backward bidirectional LSTM
Zhang et al (20210158799) teaches a speech end-to-end recognition system using a combination of CTC and attention models, with a streaming multi-layer truncation model (para 0044).
Prabhavalkar et al (20200402501) teaches a first and second attention models, processing acoustic features into biased phrases (para 0011), to optimize word error rates – para 0033-0034).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Opsasnick, telephone number (571)272-7623, who is available Monday-Friday, 9am-5pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Mr. Richemond Dorvil, can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/Michael N Opsasnick/Primary Examiner, Art Unit 2658 09/23/2026
