DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-6, 10-11, 14, 17-19, and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (From IDS: “Mitigating Unintended Memorization in Language Models via Alternate Teaching”) in view of Kim et al. (USPG 2018/0247642, hereinafter Kim).
Regarding claims 1 and 14, Liu discloses a computer-implemented method and system executed on data processing hardware that causes the data processing hardware to perform operations comprising:
obtaining an automatic speech recognition (ASR) model pre-trained on an initial training dataset (section 4.2, ASR model);
creating a set of canary speech utterances (section 4.2, generating “canaries”);
fine-tuning the ASR model on the set of sped-up canary speech utterances (sections 4.2-4.3, training LM); and
measuring un-intended memorization of the fine-tuned ASR model based on speech recognition results performed by the fine-tuned ASR model on the sped-up canary speech utterances (sections 4.2-4.3, measuring unintended memorization of the ASR model trained with canary speech).
Liu fails to explicitly disclose, however, Kim teaches data processing hardware (Kim: figure 1); and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations that include (Kim: figure 1): speeding up each canary speech utterance in the set of canary speech utterances (process in figure 2, time scale modification can speed up speech).
Since Liu and Kim are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of speeding up speech. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007).
Regarding claims 4-6 and 17-19, Liu further discloses wherein the initial training data set used to pre-train the ASR model comprises a set of un-transcribed speech utterances that each comprise audio-only data not paired with any corresponding transcription (section 4.2, training speech dataset not associated with transcription); wherein the set of un-transcribed speech utterances are multilingual (section 4.2, dataset from different user, and therefore can be in different languages); wherein a number of utterances in the initial training data set is greater than a number of utterances in the set of canary speech utterances (section 4.2 discusses about creating a few canary speech to insert into training data).
Regarding claims 10-11 and 23-24, the combination of Liu and Kim further discloses wherein speeding up each canary speech utterance in the set of canary speech utterances comprises speeding up each canary speech utterance to a speaking pace that is faster than a normal human speaking pace (Kim: process in figure 2 teaches speeding up speed at any rate faster than normal speaking speed); wherein the speaking pace of each sped-up canary speech utterance is four times faster than the normal human speaking pace (Kim: process in figure 2 teaches speeding up speed at any rate faster than normal speaking speed; can be 4 times higher).
Since Liu and Kim are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of speeding up speech. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007).
Claims 2-3 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kim, and further in view of Chen et al. (USPG 2021/0280170, hereinafter Chen).
Regarding claims 2 and 15, the modified Liu still fails to explicitly disclose, however, Chen further teaches wherein the operations further comprise: obtaining a set of transcribed speech utterances, each transcribed speech utterance paired with a corresponding ground-truth transcription, wherein fine-tuning the ASR model on the set of sped-up canary speech utterances further comprises fine-tuning the ASR model on the set of transcribed speech utterances (paragraphs 40-44, training ASR model based on comparing speech recognition results against a ground truth transcription).
Since the modified Liu and Chen are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of training ASR model based on a comparison between the speech recognition results and a ground truth transcription. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007).
Regarding claims 3 and 16, Liu further discloses wherein a number of utterances in the set of transcribed speech utterances is less than a number of utterances in the initial training data set used to pre-train the ASR model (section 4.2 discusses about creating a few training speech data, which is significantly speed data used to create a pre-trained model).
Claims 7 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kim, and further in view of Chiu et al. (USPG 2025/0118291, hereinafter Chiu).
Regarding claims 7 and 20, the modified Liu still fails to explicitly disclose, however, Chiu further teaches wherein the ASR model is pre-trained on the set of un-transcribed speech utterances using BERT-based Speech pre-training with random projection quantizer (BEST-RQ) (paragraph 85 and/or process in figures 1-2).
Since the modified Liu and Chiu are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of using BERT-based speech pre-training with BEST-RQ to train model. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007).
Claims 8-9 and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kim, and further in view of Huang et al. (From IDS: “Detecting Unintended Memorization in Language-Model-Fused ASR”).
Regarding claims 8-9 and 21-22, the modified Liu fails to explicitly disclose, however, Huang teaches wherein creating the set of canary speech utterances comprises: generating a set of text-only utterances from a language model; and converting, using a text-to-speech (TTS) system, each text-only utterances from the set of text-only utterances into a corresponding synthesized speech representation, wherein the synthesized speech representation converted from the set of text-only utterances form corresponding ones of the set of canary speech utterances (Introduction section 1 and section 2.1 discuss generating text and convert text, using TTS, to speech); and wherein the set of text-only utterances generated from the language model comprise a sequence of randomly sampled consonants and words from the language model (Introduction section 1 and section 2.1 discuss generating text and convert text, using TTS, to speech; the text utterance obviously contains both vowels and consonants).
Since the modified Liu and Huang are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of generating a set of text utterance and converting the text utterance to synthetic speech. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007).
Claims 12 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kim and further in view of Valentine et al. (USPG 2018/0061409, hereinafter Valentine).
Regarding claims 12 and 25, the modified Liu fails to explicitly disclose, however, Valentine teaches wherein the operations further comprise applying sensitivity-bounded training is applied when fine-tuning the ASR model (paragraph 66, ASR tuning parameters can be adjusted).
Since the modified Liu and Valentine are analogous in the art because they are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to use the known technique of tuning the speech recognizer by adjusting tuning parameters. One of ordinary skill in the art would have recognized that the results of the combination were predictable since the use of that known technique provides the rationale to arrive at a conclusion of obviousness. See KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007).
Allowable Subject Matter
Claims 13 and 26 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Beaufays et al. (USPG 2021/0104223) teach a speech synthesis method for synthesizing textual segments for training of speech recognition model that is considered pertinent to the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUYEN X VO whose telephone number is (571)272-7631. The examiner can normally be reached M-F, 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUYEN X VO/Primary Examiner, Art Unit 2656
Prosecution Insights