DETAILED ACTION
Introduction
1. This office action is in response to Applicant’s submission filed on 11/28/2024. Claims 1-20 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1-5, 9-15, 19, and 20 are rejected under 35 U.S.C. 103 as unpatentable over “ADAPTABLE MULTI-DOMAIN LANGUAGE MODEL FOR TRANSFORMER ASR” (Lee et al., hereinafter “Lee”) in view of “DAMAGE CONTROL DURING DOMAIN ADAPTATION FOR TRANSDUCER BASED AUTOMATIC SPEECH RECOGNITION” (Majumdar et al., hereinafter “Maj”).
With regard to Claim 1, Lee describes:
“A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising:
obtaining an automatic speech recognition (ASR) model pre-trained on an initial training data set, the ASR model comprising a plurality of layers; (Section I, paragraph 3 describes that ASR model may be pre-trained. Figure 2 shows the model has a plurality of layers.)
augmenting the ASR model with a recurrent adapter comprising a controller and a plurality of adapter heads, wherein the controller and the plurality of adapter heads are shared with each layer of the plurality of layers of the ASR model; (Figure 2 shows that the model has a plurality of adapter heads Nd. The adapter heads are connected at least indirectly to the other layers.)
receiving an adaptation training data set comprising a plurality of spoken utterances, each respective spoken utterance of the plurality of spoken utterances in the adaptation training data set is paired with a respective transcription of the respective spoken utterance; and (Section 3, paragraph 2 describes that additional training data including text and corresponding utterances is received and used to train the model.)
Lee does not explicitly describe “adapting the ASR model augmented with the recurrent adapter to the adaptation training data set while parameters of the ASR model are frozen.”
However, Maj describes ““adapting the ASR model augmented with the recurrent adapter to the adaptation training data set while parameters of the ASR model are frozen.”
Section 3, paragraph 1 of Maj describes that a pre-trained model has the original parameters frozen while the model is retrained.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the parameter freezing as described by Maj into the system of Lee to avoid gradient updates, as described in Section 3, paragraph 1 of Maj.
With regard to Claim 2, Lee describes “each adapter head of the plurality of adapter heads comprises a simple linear projection matrix architecture.”
Figure 2 shows the simple linear architecture of the adapter heads.
With regard to Claim 3, Lee describes “each adapter head of the plurality of adapter heads comprises a feed-forward network (FFN) architecture.”
Figure 2 shows that each of the adapter heads includes FFN architecture.
With regard to Claim 4, Lee describes “each spoken utterance of the plurality of spoken utterances of the adaptation training data set is spoken by a speaker with atypical speech.” Section 3, paragraph 3 describes that a data set from a noisy environment may be used, which would be atypical speech.
With regard to Claim 5, Lee does not explicitly describe “a number of the plurality of spoken utterances in the adaptation training data set is less than a number of utterances in the initial training data set used to pre-train the ASR model.” However, Section 3 of describes many different training sets of different sizes. As there are only three possible choices (pre-train set > adaptation training set, pre-train set = adaptation training set, and pre-train set < adaptation training set), it would have been obvious to try each of these limited number of solutions to determine the best. See MPEP 2143(E). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the relative training data set sizes into the system of Lee.
With regard to Claim 9, Lee describes “the adaptation training data set comprises anonymized utterances in a single language.” Section 3, paragraph 2 describes a data set of all Korean utterances which are anonymized.
With regard to Claim 10, Lee describes “augmenting the ASR model with the recurrent adapter further comprises inserting the controller and the plurality of adapter heads of the recurrent adapter into each layer of the ASR model.” Figure 2 shows that the plurality of adapter heads are included in each layer of the ASR model.
With respect to Claims 11-15, 19, and 20, method Claim 1 and system Claim 11 are related as a system programmed to perform the same method, with each claimed system function corresponding to each claimed method step. Accordingly, Claims 11-15, 19, and 20 are similarly rejected under the same rationale as applied above with respect to Claims 1-5, 9, and 10.
5. Claims 6-8 and 16-18 are rejected under 35 U.S.C. 103 as unpatentable over Lee in view of Maj and further in view of “Self-Supervised Speech Representation Learning: A Review” (Mohamed et al., hereinafter “Moh”).
With regard to Claim 6, Lee in view of Maj does not explicitly describe this feature. However, Moh describes “the initial training data set comprises a set of un-transcribed speech utterances.” Section IV(A), paragraph 1 describes that a pre-training data set may be speech only.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the pre-training data as described by Moh into the system of Lee in view of Maj to use readily available data sets, as described in Section IV(A), paragraph 1 of Moh.
With regard to Claim 7, Lee in view of Maj does not explicitly describe this feature. However, Moh describes “the ASR model is pre-trained on the set of un- transcribed speech utterances using BERT-based Speech pre-training with random projection quantizer (BEST-RQ).” Table I, “Predictive Models” describes the use of BERT and BEST-RQ. Section IV(A), paragraph 1 describes the use of speech only data sets.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the pre-training data and models as described by Moh into the system of Lee in view of Maj to use readily available data sets and models, as described in Table I and Section IV(A), paragraph 1 of Moh.
With regard to Claim 8, Lee in view of Maj does not explicitly describe this feature. However, Moh describes “the speech utterances in the set of un-transcribed speech utterances comprise multilingual speech utterances.” Section IV(A), paragraph 2 of Moh describes that the pre-training data set may be multilingual.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the pre-training data as described by Moh into the system of Lee in view of Maj to use readily available data sets, as described in Section IV(A), paragraph 2 of Moh.
With respect to Claims 16-18, method Claim 1 and system Claim 11 are related as a system programmed to perform the same method, with each claimed system function corresponding to each claimed method step. Accordingly, Claims 16-18 are similarly rejected under the same rationale as applied above with respect to Claims 6-8.
Conclusion
6. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Pat. No. 12,230,258 (Biadsy et al.) also describes a device that uses adapters in a recurrent neural network.
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD TRACY whose telephone number is (571)272-8332. The examiner can normally be reached Monday-Friday 9 AM- 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EDWARD TRACY JR./Examiner, Art Unit 2656
Prosecution Insights