DETAILED ACTION
This action is in response to the initial filing of application no. 18/355,166 on 08/05/2025.
Response to Amendment
Applicant’s amendment filed on 08/05/2025 has been entered. Claims 1 and 3 have been amended. Claims 4 and 5 have been canceled. No claims have been added.
The 35 USC 101 rejection of claims 1 – 3 made in the prior office action has been obviated by applicant’s amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupta et al. (US 12,112,752) (“Gupta”) in view of in view of Donofrio et al. (“US 2020/0143820”) (“Donofrio”), and further in view of Thai et al. (“Synthetic Data Augmentation for Improving Low-Resource ASR”) (“Thai”), and further in view of Chen et al. (US 2020/0160836) (“Chen”) and further, and further in view of and further in view of Johnson et al. (US 2021/0089887) (“Johnson”).
For claim 1, Gupta discloses a method of training a transcoder algorithm having a parameter space using a training dataset comprising audio samples having high resource and low resource language components, the method comprising: by one or more computing devices: evaluating batches of audio samples from a training dataset based on at least one language component (features that represent the natural language input or acoustic characteristics of the spoken user input) (the various features are clustered together based on feature representation homogeneity loss to maximize the homogeneity of feature representations in a cluster, i.e. each cluster comprises similar data representations/similar language and acoustic features, Fig.1A, 120a, 122, 124, 140a; column 5 lines 54 – column 6 lines 40, 45 – column 7 line 2; column 7 lines 25 – column 8 line 45; column 16 lines 23 – 65); and training a pre-trained neural network (ASR) using a cohort of the batches of audio samples (column 9 lines 35 – 55 column 12 lines 50 – column 13 line 40). Yet, Gupta fails to teach the following: the language component is selected from a group consisting of accent, dialect, accented and language specific domain, accented and language specific subdomain, dialect and language specific domain, dialect and language specific subdomain; creating, using at least two audio samples from the batches of audio samples, one or more additional audio samples; adding the one or more additional audio samples to the batches of audio samples; segmenting, the batches of audio samples, into one or more minibatches by sampling the batches of audio samples; generating, one or more encodings for each audio sample of the one or more minibatches; generating, using the one or more encodings, one or more predictions for each audio sample of the one or more minibatches; comparing the one or more true labels for each audio sample of the one or more batches, to determine one or more losses for each audio sample of the one or more minibatches; aggregating, for each audio sample of the one or more minibatches, the one or more losses to form an aggregated loss score; and tuning the parameter space of the transcoder algorithm using the aggregated loss score, wherein the generating steps, the comparing step, and the aggregating step are performed iteratively using the one or more minibatches.
However, Donofrio discloses a system and method for generating a speech transcript (Abstract), comprising the following: a language component comprises an accent ([0029]); and a language component further comprises a vocabulary, wherein the vocabulary can be domain specific (medicine or engineering) or subdomain specific (patent litigation, bankruptcy case) ([0029]).
Additionally, Thai discloses a method for applying deep learning to automatic speech recognition (Abstract), comprising the following: up-sampling audio samples of a low resource data set to create two or more additional audio samples(voice conversion is applied to a low-resource data set, Fig.1; 1. Introduction, 2. Background and 4.2 Multistage transfer learning); and adding the up-sampled audio samples to training data set to train an ASR model (Fig.1; 1. Introduction, 2. Background and 4.2 Multistage transfer learning).
Furthermore, Chen discloses a system and method for performing speech recognition (Abstract), comprising the following: a language component comprises a dialect ([0004]). Furthermore, Chen discloses that training a neural network comprises the following: generating, one or more encodings for audio samples of batches of samples of a training dataset (The audio samples are encoded using an encoder of speech recognition model/LAS model. The batches of audio samples comprise over 35M English utterances with varying dialects, Fig.2A, 212; [0051] [0055 – 0057] [0060] [0067] [0081] [0082]); generating one or more predictions for each audio sample (The predictions, a sequence of graphemes, are generated by inputting encodings into the decoder/attention network, Fig.2A, 214 and 216; [0051] [0055 – 0057] [0060] [0067] [0083 – 0085]); comparing the one or more true labels for each audio sample of the one or more batches to determine one or more losses for each audio sample of the one or more batches (Fig.2a, 230; [0051] [0055 – 0060] [0085 – 0088]); and tuning the parameter space of the transcoder algorithm (attention network/decoder of the ASR/LAS model) using the loss score ([0014] [0030] [0086 – 0088]).
Moreover, Johnson discloses a system and method for training a machine learning model(Abstract), comprising the following: segmenting a batch of samples into minibatches ([0008] [0024] [0027] [0058]); aggregating, for each sample of the one or more minibatches, the one or more losses to form an aggregated loss score (A loss is determined for the mini-batch, wherein the mini-batch comprises a plurality of samples, [0058]); and the parameters of a neural network model are updated based on the loss ([0058]), wherein the model is iteratively trained ([0061 – 0063]).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve Gupta’s invention in the same way that Donofrio’s invention has been improved to achieve the following, predictable results for the purpose of providing an ASR model with improved recognition of inputs which were not sufficiently represented in an set of training data (Gupta, column 2 lines 20 – 33): the language component further comprises an accent, language specific domain, or language specific sub-domain.
Additionally, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Gupta and Donfrio in the same way that Thai’s invention has been improved to achieve the following, predictable result for the purpose of providing an ASR model with improved recognition of inputs which were not sufficiently represented in an set of training data (Gupta, column 2 lines 20 – 33): the cohort comprising the language components which are used to train the pretrained ASR model further comprise low resource language audio samples (Gupta, audio samples comprising acoustic features and language which generate high word error rates similar to low resource languages, column 2 lines 20 – 33; column 6 lines 6 – 27; column 20 lines 56 – column 21 line 2) (Thai, Abstract, 1. Introduction); this cohort is further up-sampled to generate up-sampled audio samples, wherein up-sampling involves creating new samples; and the up-sampled audio samples are added to the batches of audio samples.
Moreover, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Gupta, Donfrio and Thai in the same way that Chen’s invention has been improved to achieve the following, predictable results for the purpose of providing an ASR model which accurately recognizes speech in multiple different languages or dialects (Chen, [0002 – 0007]): the language component of the audio samples further comprises dialect; one or more encodings for audio samples of batches of samples of a training dataset are generated; one or more predictions for each audio sample are generated; one or more true labels for each audio sample of the one or more batches are compared to determine one or more losses for each audio sample of the one or more batches; and parameter space of the transcoder algorithm (attention network/decoder network of a ASR model) is further tuned using the loss score.
Furthermore, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Gupta, Donfrio, Thai and Chen in the same way that Johnson’s invention has been improved to achieve the following, predictable results for the purpose of providing an ASR model which accurately recognizes speech in multiple different languages or dialects (Chen, [0002 – 0007]): a batch of samples is further segmented into minibatches; for each sample of the one or more minibatches, aggregating the one or more losses to form an aggregated loss score; and the parameters of the transcoder algorithm (neural network) are further updated based on the loss, wherein the model is iteratively trained (including the generating, comparing and aggregating steps as discussed above).
For claim 3, Gupta, Thai and Donofrio further disclose, wherein creating the additional audio samples further comprises combining the at least two audio samples from batches of audio samples, each of the at least two audio samples having a same as least one language component (Gupta, data points comprising similar language component features are clustered together, column 5 lines 54 – column 6 lines 40, 45 – column 7 line 2; column 7 lines 25 – column 8 line 45; column 16 lines 23 – column 17 line 6) (Donofrio, [0029]) (Thai, Fig.1; 1. Introduction, 2. Background and 4.2 Multistage transfer learning); or combining the at least two audio samples from the batches of audio samples, each of the at least two audio samples having a different at least one language component.
Response to Arguments
Applicant’s arguments filed on 08/05/2025 have been considered, but are moot in view of the new ground(s) of rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SONIA L GAY whose telephone number is (571)270-1951. The examiner can normally be reached Monday-Friday 9-5 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at 571-272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SONIA L GAY/ Primary Examiner, Art Unit 2657