Phrase Extraction for ASR Models

§102 §103 §DOUBLEPATENT

Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . DETAILED ACTION This office action is in response to application 19063261, which was filed 02/25/25, and is a continuation of 18/609,362, now US Patent 12,260,875, which is a continuation of 17/643,848, now US Patent 11,955,134. Claims 1-20 are pending in the application and have been considered. Double Patenting The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b). Claims 1, 3-11, and 13-20 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1, 4-10, and 12 of US Patents 12,260,875 and 11,955,134. Specifically, a comparison of claim 1 in the present application with claim 1 of US Patents 12,260,875 and 11,955,134 yields the following: (Present application) (US Patent 12,260,875) (US Patent 11,955,134) (Examiner notes) 1. A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising: obtaining input data representing a data item; modifying the input data to obfuscate an identified feature within the data item; processing, using a trained model, the modified input data to generate a predicted representation of the data item; determining the predicted representation of the data item includes the identified feature by comparing the predicted representation of the data item to a ground-truth representation of the data item; and generating an output indicating that the trained model leaked the identified feature from a training data set used to train the model. 1. A computer-implemented method executed on data processing hardware that causes the data processing hardware to perform operations comprising: obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance; modifying the audio data to obfuscate the particular phrase recited in the utterance; and processing, using a trained automated speech recognition (ASR) model, the modified audio data to generate a predicted transcription of the utterance; determining whether the predicted transcription includes the particular phrase or another phrase substituted for the particular phrase from the ground-truth transcription that is associated with a same category of information as the particular phrase by comparing the predicted transcription of the utterance to the ground-truth transcription of the utterance; generating an output indicating that the trained ASR model leaked the other phrase from a training data set used to train the ASR model. 1. A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising: obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance; modifying the audio data to obfuscate a particular phrase recited in the utterance; processing, using a trained automated speech recognition (ASR) model, the modified audio data to generate a predicted transcription of the utterance; determining whether the predicted transcription includes the particular phrase by comparing the predicted transcription of the utterance to the ground-truth transcription of the utterance; generating an output indicating that the trained ASR model leaked the particular phrase from a training data set used to train the ASR model. “audio data” = “data item” “an identified feature” = “a particular phrase” “predicted representation” = “predicted transcription” See above See above As the table above demonstrates, while the language is not identical, each limitation of claim 1 of the present application is found in claim 1 of US Patents 12,260,875 and 11,955,134, thus claim 1 of the present application is anticipated by US Patents 12,260,875 and 11,955,134. Independent claim 11 corresponds to and is similarly considered anticipated by claims 10 and 12 of US Patents 12,260,875 and 11,955,134 respectively. Dependent claims 3-10 of the present application are anticipated by claims 4-9 of US Patents 12,260,875 and 11,955,134 as detailed below: (Present application) (US Patent 12,260,875) (US Patent 11,955,134) (Examiner notes) 3. The computer-implemented method of claim 1, wherein the operations further comprise identifying a segment of the input data that aligns with the identified feature in the ground-truth representation of the data item. 7. The computer-implemented method of claim 1, wherein modifying the audio data comprises: based on the ground-truth transcription, identifying a segment of the audio data that aligns with the particular phrase in the ground-truth transcription; 7. The computer-implemented method of claim 1, wherein modifying the audio data comprises: based on the ground-truth transcription, identifying a segment of the audio data that aligns with the particular phrase in the ground-truth transcription Audio data = input data Transcription = representation of the data item 4. The computer-implemented method of claim 3, wherein modifying the input data comprises performing data augmentation on the identified feature to obfuscate the identified feature within the data item. 7… performing data augmentation on the identified segment of the audio data to obfuscate the particular phrase recited in the utterance. 7… performing data augmentation on the identified segment of the audio data to obfuscate the particular phrase recited in the utterance. See above 5. The computer-implemented method of claim 4, wherein performing data augmentation on the identified feature comprises adding noise to the identified feature within the data item. 8. The computer-implemented method of claim 7, wherein performing data augmentation on the identified segment of the audio data comprises adding noise to the identified segment of the audio data. 8. The computer-implemented method of claim 7, wherein performing data augmentation on the identified segment of the audio data comprises adding noise to the identified segment of the audio data. See above 6. The computer-implemented method of claim 4, wherein performing data augmentation on the identified feature comprises removing the identified feature within the data item. 9. The computer-implemented method of claim 7, wherein performing data augmentation on the identified segment of the audio data comprises removing the identified segment of the audio data. 9. The computer-implemented method of claim 7, wherein performing data augmentation on the identified segment of the audio data comprises replacing the identified segment of the audio data with noise. See above 7. The computer-implemented method of claim 1, wherein the input data representing the data item is derived from the ground-truth representation of the data item. 1… obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance; processing the ground-truth transcription to identify a particular phrase included in the ground-truth transcription that is associated with sensitive data; modifying the audio data to obfuscate the particular phrase recited in the utterance; 1…. obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance; modifying the audio data to obfuscate a particular phrase recited in the utterance; Since the input audio data is essentially the ground truth audio data with a particular phrase obfuscated, it is considered “derived” from the ground truth audio data 8. The computer-implemented method of claim 1, wherein the input data comprises an audio waveform. 4. The computer-implemented method of claim 1, wherein the audio data comprises an audio waveform. 4. The computer-implemented method of claim 1, wherein the audio data comprises an audio waveform. none 9. The computer-implemented method of claim 8, wherein the audio waveform corresponds to human speech. 5. The computer-implemented method of claim 4, wherein the audio waveform corresponds to human speech. 5. The computer-implemented method of claim 4, wherein the audio waveform corresponds to human speech. none 10. The computer-implemented method of claim 8, wherein the audio waveform corresponds to synthesized speech. 6. The computer-implemented method of claim 4, wherein the audio waveform corresponds to synthesized speech. 6. The computer-implemented method of claim 4, wherein the audio waveform corresponds to synthesized speech. none The subject matter of the instant claims 13-20 corresponds to that shown above with regard to claims 3-10. These claims are also anticipated by claims 4-9 of US Patents 12,260,875 and 11,955,134 for similar reasons. Claims 2 and 12 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claim 1 of US Patents 12,260,875 and 11,955,134 in view of Zhang et al. (“The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks”. arXiv:1911.07135v2 [cs.LG] 18 Apr 2020). Consider claims 2 and 12, claim 1 of US Patents 12,260,875 and 11,955,134 does not, but Zhang discloses operations further comprise receiving a user input that explicitly identifies the feature to be obfuscated (in order to conduct the experiments described at 5.3.1, page 6, it is inherent that the system received a selection from the researchers indicating whether to apply the “center mask”, “T” mask, or blurring, Section 5.3.1, page 6, Figure 2, page 7; the former two explicitly identify features of the face which they obfuscate such as the eyes). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify claim 1 of US Patents 12,260,875 and 11,955,134 by receiving a user input that explicitly identifies the feature to be obfuscated in order to more comprehensively evaluate the tendency for networks to reveal private information in response to different attack types, as suggested by Zhang (Section 5.3.1, page 6). Doing so would have had the predictable effect of improving ability to protect against network attacks, as suggested by Zhang (page 2, Section 1). Claim Rejections - 35 USC § 102 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention. Claims 1-7 and 11-17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang et al. (“The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks”. arXiv:1911.07135v2 [cs.LG] 18 Apr 2020). Consider claim 1, Zhang discloses a computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations (a computer executing instructions from a memory is inherent for performing the proposed GMI attack, which is a computer vision task, page 2, Figure 1) comprising: obtaining input data representing a data item (a face image from the CelebFaces Attributes Dataset (CelebA), page 5, Section 5.1); modifying the input data to obfuscate an identified feature within the data item (the center mask blocks the central part of the face and hides most of the identity-revealing features, such as eyes and noise, while the face T mask is designed to obstruct all private features, or the image is blurred; Section 5.3.1, page 6); processing, using a trained model, the modified input data to generate a predicted representation of the data item (Result from Generator, Fig 5, Fig 6, page 11, Section B.2; Section 5.3.1, page 6); determining the predicted representation of the data item includes the identified feature by comparing the predicted representation of the data item to a ground-truth representation of the data item (evaluation metrics compare the reconstructed image to the target, i.e. ground truth image, Section 5.2, page 5, Fig. 2, page 7); and generating an output indicating that the trained model leaked the identified feature from a training data set used to train the model (ability of classifier to recognize that the target image had e.g. blond hair, bushy eyebrows, glasses, etc. from the recovered image indicates that the model unintentionally leaked that feature from its training data, Section 5.3.2, page 7, Section 1, page 1). Consider claim 11, Zhang discloses a system comprising: data processing hardware (data processing hardware is inherent for performing the proposed GMI attack, which is a computer vision task, page 2, Figure 1); and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (a computer executing instructions from a memory is inherent for performing the proposed GMI attack, which is a computer vision task, page 2, Figure 1) comprising: obtaining input data representing a data item (a face image from the CelebFaces Attributes Dataset (CelebA), page 5, Section 5.1); modifying the input data to obfuscate an identified feature within the data item (the center mask blocks the central part of the face and hides most of the identity-revealing features, such as eyes and noise, while the face T mask is designed to obstruct all private features, or the image is blurred; Section 5.3.1, page 6); processing, using a trained model, the modified input data to generate a predicted representation of the data item (Result from Generator, Fig 5, Fig 6, page 11, Section B.2; Section 5.3.1, page 6); determining the predicted representation of the data item includes the identified feature by comparing the predicted representation of the data item to a ground-truth representation of the data item (evaluation metrics compare the reconstructed image to the target, i.e. ground truth image, Section 5.2, page 5, Fig. 2, page 7); and generating an output indicating that the trained model leaked the identified feature from a training data set used to train the model (ability of classifier to recognize that the target image had e.g. blond hair, bushy eyebrows, glasses, etc. from the recovered image indicates that the model unintentionally leaked that feature from its training data, Section 5.3.2, page 7, Section 1, page 1). Consider claim 2, Zhang discloses the operations further comprise receiving a user input that explicitly identifies the feature to be obfuscated (in order to conduct the experiments described at 5.3.1, page 6, it is inherent that the system received a selection from the researchers indicating whether to apply the “center mask”, “T” mask, or blurring, Section 5.3.1, page 6, Figure 2, page 7; the former two explicitly identify features of the face which they obfuscate such as the eyes). Consider claim 3, Zhang discloses the operations further comprise identifying a segment of the input data that aligns with the identified feature in the ground-truth representation of the data item (e.g. mustache, glasses, page 7, Section 5.3.2). Consider claim 4, Zhang discloses modifying the input data comprises performing data augmentation on the identified feature to obfuscate the identified feature within the data item (blurring the image, i.e. augmenting the image data by adding a noise to blur the pixels, page 7, Section 5.3.2). Consider claim 5, Zhang discloses performing data augmentation on the identified feature comprises adding noise to the identified feature within the data item (blurring, Fig 2, page 7, Section 5.3.2). Consider claim 6, Zhang discloses performing data augmentation on the identified feature comprises removing the identified feature within the data item (the center and T mask remove facial features, Fig 2, page 7, Section 5.3.2). Consider claim 7, Zhang discloses the input data representing the data item is derived from the ground-truth representation of the data item (the target image, Fig 2, page 7, Section 5.3.2). Consider claim 12, Zhang discloses the operations further comprise receiving a user input that explicitly identifies the feature to be obfuscated (in order to conduct the experiments described at 5.3.1, page 6, it is inherent that the system received a selection from the researchers indicating whether to apply the “center mask”, “T” mask, or blurring, Section 5.3.1, page 6, Figure 2, page 7; the former two explicitly identify features of the face which they obfuscate such as the eyes). Consider claim 13, Zhang discloses the operations further comprise identifying a segment of the input data that aligns with the identified feature in the ground-truth representation of the data item (e.g. mustache, glasses, page 7, Section 5.3.2). Consider claim 14, Zhang discloses modifying the input data comprises performing data augmentation on the identified feature to obfuscate the identified feature within the data item (blurring the image, i.e. augmenting the image data by adding a noise to blur the pixels, page 7, Section 5.3.2). Consider claim 15, Zhang discloses performing data augmentation on the identified feature comprises adding noise to the identified feature within the data item (blurring, Fig 2, page 7, Section 5.3.2). Consider claim 16, Zhang discloses performing data augmentation on the identified feature comprises removing the identified feature within the data item (the center and T mask remove facial features, Fig 2, page 7, Section 5.3.2). Consider claim 17, Zhang discloses the input data representing the data item is derived from the ground-truth representation of the data item (the target image, Fig 2, page 7, Section 5.3.2). Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 8, 9, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (“The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks”. arXiv:1911.07135v2 [cs.LG] 18 Apr 2020) in view of Goshen et al. (US 20210303724). Consider claim 8, Zhang does not, but Goshen discloses the input data comprises an audio waveform (an audio recording of spoken words, [0054]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by replacing the image input data with an audio waveform in order to improve phone call security, as suggested by Goshen [0003-0004]). Doing so would have led to predictable results of better protecting personal information, as suggested by Goshen ([0004]). The references cited are analogous art in the same field of personal information security. Consider claim 9, Zhang does not, but Goshen discloses the audio waveform corresponds to human speech (an audio recording of spoken words, [0054]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the audio waveform corresponds to human speech for reasons similar to those for claim 8. Consider claim 18, Zhang does not, but Goshen discloses the input data comprises an audio waveform (an audio recording of spoken words, [0054]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang by replacing the image input data with an audio waveform for reasons similar to those for claim 8. Consider claim 19, Zhang does not, but Goshen discloses the audio waveform corresponds to human speech (an audio recording of spoken words, [0054]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang such that the audio waveform corresponds to human speech for reasons similar to those for claim 8. Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (“The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks”. arXiv:1911.07135v2 [cs.LG] 18 Apr 2020) in view of Goshen et al. (US 20210303724), in further view of Pal et al. ("Synthetic speech detection using fundamental frequency variation and spectral features". Computer Speech & Language 48 (2018) 31 50). Consider claim 10, Zhang and Goshen do not, but Pal discloses the audio waveform corresponds to synthesized speech (we evaluate using the ASVspoof 2015 database, which is developed using synthetic speech generation methods, page 40, Section 4.1.1, 4.1.2., the speech frames being the "audio waveforms"). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang and Goshen such that the audio waveform corresponds to synthesized speech in order to mitigate spoofing attacks, predictably reducing fraud, as suggested by Pal (pages 31-32, Section 1, Introduction). The references cited are analogous art in the same field of personal information security. Consider claim 20, Zhang and Goshen do not, but Pal discloses the audio waveform corresponds to synthesized speech (we evaluate using the ASVspoof 2015 database, which is developed using synthetic speech generation methods, page 40, Section 4.1.1, 4.1.2., the speech frames being the "audio waveforms"). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhang and Goshen such that the audio waveform corresponds to synthesized speech for reasons similar to those for claim 10. Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 20200311540 Chakraborty discloses a layer-wise distillation operation that obfuscates original training data used to train a deep learning model US 20210377035 Walheim discloses intelligent hashing of sensitive information US 20210064760 Sharma discloses protecting machine learning models from privacy attacks US 20210150269 Choudhury discloses anonymizing data for preserving privacy during use for federated machine learning Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jesse Pullias whose telephone number is 571/270-5135. The examiner can normally be reached on M-F 8:00 AM - 4:30 PM. The examiner’s fax number is 571/270-6135. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Andrew Flanders can be reached on 571/272-7516. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Jesse S Pullias/ Primary Examiner, Art Unit 2655 10/08/26