Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Acknowledgment is made of applicant's claim for domestic priority based on US provisional application 63/781246 filed on 03/31/2025.
Claim Objection
Claim 19 is objected to for improper dependency on claim 15. Claim 19 recites “The computer storage medium of claim 15, wherein the instructions upon execution by the processor cause the processor to convert the second audio segment in the audio signal into a voice of a single target virtual agent speaker, wherein the filler audio segment and the second audio segment in the de-identified audio signal are in the voice of the single target virtual agent speaker”.
However, claim 15 is a computerized method claim. Further, claims 7-15 do not require the second audio segment in the de-identified audio signal, which is present in claim 16. For claim interpretation purposes, claim 19 is dependent on claim 16.
Claim Rejections - 35 USC § 103
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 103 that form the basis for the rejections under this section made in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-7, 11-13, and 15 are rejected under 35 USC 103(a) as being unpatentable over Weinstein et al. (US 9990925 B2) in view of Bumberger et al. (US 12444407 B2) and Cline et al. (US 10747894 B1).
Regarding Claim 1, Weinstein discloses a system for audio de-identification (Fig. 1, Speech Processing System 106; see also Fig. 4, Col 7, Rows 43-48, computing device 400) comprising:
a processor (Col 7, Rows 57-58, processor 402); and
a memory storing instructions that upon execution by the processor cause the processor (Col 8, Rows 26-32, computer program product containing instructions embodied in memory 404, storage device 406, or memory on processor 402 to perform one or more methods when executed) to:
receive an audio signal comprising a first phrase representing privacy information and a second phrase representing non-privacy information (Col 3, Rows 35-38, receiving user utterance “This is Bob Smith. My credit card number is 01-234”; Col 3, Rows 44-47, generate speech recognition request 107 comprising an audio file encoding the utterance 108a and context data 108b);
generate, using a privacy tagged automatic speech recognition (ASR) model (Col 5, Rows 45-50, automated speech recognizer 120 uses a speech recognition model provided by training engine 118 to generate the transcript 110; Col 5, Rows 33-44, training engine 118 generates / updates a speech recognition model using received data from request 107 including likely sensitive data before any likely sensitive data is deleted; Col 7, Rows 17-23, continuously provide privacy sensitive data to model trainer 312 for training), a transcript associated with the audio signal (Col 3, Rows 63-65, speech processing system 106 receives the speech recognition request 107, analyzes the request 107, and generates a text transcription 110 of the utterance), the transcript is associated with a privacy tag encoding the first phrase representing the privacy information (Fig. 1, sensitive info identifier 116 outputs sensitive information signal 123 to sensitive information manager 122; Col 5, Row 61 – Col 6, Row 3, sensitive information signal 123 indicates which portions of the transcript 110 likely contains sensitive information and thus sensitive information manager 122 anonymizes a corresponding phrase; e.g., Col 6, Rows 1-3 and Col 7, Rows 8-9, anonymize “This is Bob Smith” to “This is <name>”) and a non-privacy tag encoding the second phrase representing the non-privacy information (Col 5, Rows 56-61 and Col 7, Rows 4-7, sensitive information signal 123 indicates which portions of the request 107 and the transcript 110 may not include sensitive information and that portion may be able to be logged; Col 5, Rows 61-66, based on the sensitive information signal 123, sensitive information manager 122 determines which portions of the request 107 and the transcript 110 are to be logged); and
remove a first audio segment corresponding to the privacy tag encoding the first phrase in the audio signal and the transcript to generate de-identified audio signal and transcript (Col 5, Rows 56-64, sensitive information manager 122 uses sensitive information signal 123 to determine which portion of the request 107 (comprising audio file data 108a per Col 3, Rows 44-47) and which portion of the transcription 110 are likely to contain sensitive information and thus not to be logged; Col 6, Rows 25-28, sensitive data is data that is deleted by the system without being stored in a log of data), the de-identified audio signal and transcript including a second audio segment corresponding to the non-privacy tag encoding the second phrase in the audio signal and the transcript (Col 7, Rows 1-8, a portion of the privacy sensitive data may not include sensitive information and that portion may be able to be logged; per Col 5, Rows 60-65, log the portion of the request 107 (which comprises audio file data 108a per Fig. 1) and the transcript 110).
Weinstein does not disclose the privacy tagged ASR model is an end-to-end automatic speech recognition (ASR) model.
Bumberger discloses using an end-to-end ASR model (Col 27, Rows 4-6) to generate interim ASR data (Col 65, Rows 10-11) to remove / obfuscate sensitive information (Col 70, Rows 18-20).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the privacy tagged ASR model as an end-to-end automatic speech recognition (ASR) model in order to remove / obfuscate sensitive information (Bumberger, Col 70, Rows 18-20).
Weinstein does not disclose the transcript includes the privacy tag encoding and the non-privacy tag encoding.
Cline discloses generating privacy tag encoding first phrase representing privacy information and non-privacy tag encoding second phrase representing non-privacy information to markup a corresponding text data (Col 13, Rows 38-58, generating SSML tags associated with various sensitivity levels indicating how sensitive a particular portion of responsive text data is; Col 13, Rows 58-63, SSML includes a markup language providing a standard way to mark up text).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the privacy tag encoding and the non-privacy tag encoding of Weinstein (Col 5, Rows 56-64, sensitive information signal 123 indicating portions of the transcript 110 are not to be logged for containing sensitive information and sensitive information signal 123 indicating portions of the transcript 110 are to be logged) using SSML tags indicating various sensitivity levels as a standard way for marking up the transcript (Cline, Col 13, Rows 59-61, SSML tags provide a standard way to mark up text; i.e., markup transcript 110 with SSML tags indicating sensitive portions that cannot be logged and indicating non-sensitive portions that may be logged) in order to provide differing sensitivity level designation for determining how to encrypt the transcript, how to store the transcript, and how to redact the transcript (Cline, Col 13, Rows 50-56).
Regarding Claim 5, Weinstein discloses wherein the privacy tag includes a plurality of types of tags for encoding the privacy information of different types in the audio signal (Weinstein, Col 5, Rows 10-25 and Rows 51-55, sensitive information identifier 116 sends sensitive information signal 123 indicating various types of sensitive information like user location, user credit card, user contact list, personal info, and others 118a-118e; as modified by Cline, Col 13, Rows 38-58, generating SSML tags associated with various sensitivity levels indicating how sensitive a particular portion of text data/ transcript is).
Regarding Claim 6, Weinstein discloses wherein the instructions upon execution by the processor cause the processor to expand boundary associated with the privacy tag in the audio signal by an amount based on background audio containing likely sensitive information (Col 4, Rows 42-48, sensitive information identifier 116 analyzes background audio present in the audio data 108a for likely sensitive information identifying speakers other than the user).
Weinstein does not disclose wherein the instructions upon execution by the processor cause the processor to expand boundary associated with the privacy tag in the audio signal by an amount using voice activity detection (VAD) in the audio signal.
Bumburger discloses expand boundary associated with the privacy tag in the audio signal by an amount using voice activity detection (VAD) in the audio signal (Col 60, Rows 6-33, implement VAD limited classifier configured to distinguish speech from background noise in order to determine whether speech is present in audio data).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to expand boundary associated with the privacy tag in the audio signal by an amount using voice activity detection (VAD) in the audio signal to distinguish sensitive information identifying speakers in the background audio (Weinstein, Col 4, Rows 42-48, identifying speakers in the background audio as sensitive information / privacy tags; compare Bumburger, Col 60, Rows 8-10 and Rows 23-24, configuring VAD classifier to distinguish human voice activity from background noise).
Regarding Claim 7, Weinstein discloses a computerized method for audio de-identification comprising:
receiving an audio signal (Col 3, Rows 35-38, receiving user utterance “This is Bob Smith. My credit card number is 01-234”; Col 3, Rows 44-47, generate speech recognition request 107 comprising an audio file encoding the utterance 108a and context data 108b);
generating, using a privacy tagged automatic speech recognition (ASR) model (Col 5, Rows 45-50, automated speech recognizer 120 uses a speech recognition model provided by training engine 118 to generate the transcript 110; Col 5, Rows 33-44, training engine 118 generates / updates a speech recognition model using received data from request 107 including likely sensitive data before any likely sensitive data is deleted; Col 7, Rows 17-23, continuously provide privacy sensitive data to model trainer 312 for training), a transcript associated with the audio signal (Col 3, Rows 63-65, speech processing system 106 receives the speech recognition request 107, analyzes the request 107, and generates a text transcription 110 of the utterance), the transcript is associated with privacy tags indicating privacy information in the audio signal (Fig. 1, sensitive info identifier 116 outputs sensitive information signal 123 to sensitive information manager 122; Col 5, Rows 56-61, sensitive information signal 123 indicates which portions of the request 107 and the transcript 110 are likely to contain sensitive information; Col 5, Rows 61-66, based on the sensitive information signal 123, sensitive information manager 122 determines which portions of the request 107 and the transcript 110 are not to be logged); and
removing audio segments corresponding to the privacy tags in the audio signal and the transcript to generate de-identified audio signal and transcript (Col 5, Rows 56-64, sensitive information manager 122 uses sensitive information signal 123 to determine which portion of the request 107 (comprising audio file data 108a) and which portion of the transcription 110 are likely to contain sensitive information and thus not to be logged; Col 6, Rows 25-28, sensitive data is data that is deleted by the system without being stored in a log of data).
Weinstein does not disclose the privacy tagged ASR model is an end-to-end automatic speech recognition (ASR) model.
Bumberger discloses using an end-to-end ASR model (Col 27, Rows 4-6) to generate interim ASR data (Col 65, Rows 10-11) to remove / obfuscate sensitive information (Col 70, Rows 18-20).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the privacy tagged ASR model as an end-to-end automatic speech recognition (ASR) model in order to remove / obfuscate sensitive information (Bumberger, Col 70, Rows 18-20).
Weinstein does not disclose the transcript includes the privacy tags.
Cline discloses generating privacy tags indicating privacy information to markup a corresponding text data (Col 13, Rows 38-58, generating SSML tags associated with various sensitivity levels indicating how sensitive a particular portion of responsive text data is; Col 13, Rows 58-63, SSML includes a markup language providing a standard way to mark up text).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the privacy tag encoding and the non-privacy tag encoding of Weinstein (Col 5, Rows 56-64, sensitive information signal 123 indicating portions of the transcript 110 are not to be logged for containing sensitive information and sensitive information signal 123 indicating portions of the transcript 110 are to be logged) using SSML tags indicating various sensitivity levels as a standard way for marking up the transcript (Cline, Col 13, Rows 59-61, SSML tags provide a standard way to mark up text; i.e., markup transcript 110 with SSML tags indicating sensitive portions that cannot be logged) in order to provide differing sensitivity level designation for determining how to encrypt the transcript, how to store the transcript, and how to redact the transcript (Cline, Col 13, Rows 50-56).
Regarding Claim 11, Weinstein discloses wherein the privacy tag includes a plurality of types of tags for indicating the privacy information of different types in the audio signal (Weinstein, Col 5, Rows 10-25 and Rows 51-55, sensitive information identifier 116 sends sensitive information signal 123 indicating various types of sensitive information like user location, user credit card, user contact list, personal info, and others 118a-118e; as modified by Cline, Col 13, Rows 38-58, generating SSML tags associated with various sensitivity levels indicating how sensitive a particular portion of text data/ transcript is).
Regarding Claim 12, Weinstein discloses expanding a boundary associated with the privacy tags in the audio signal by an amount (Col 4, Rows 42-48, sensitive information identifier 116 analyzes background audio present in the audio data 108a for likely sensitive information identifying speakers other than the user).
Regarding Claim 13, Weinstein does not disclose using voice activity detection (VAD) in the audio signal for expanding the boundary associated with the privacy tags in the audio signal.
Bumburger discloses expand boundary associated with the privacy tag in the audio signal by an amount using voice activity detection (VAD) in the audio signal (Col 60, Rows 6-33, implement VAD limited classifier configured to distinguish speech from background noise in order to determine whether speech is present in audio data).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to use voice activity detection (VAD) in the audio signal for expanding the boundary associated with the privacy tags in the audio signal to distinguish sensitive information identifying speakers in the background audio (Weinstein, Col 4, Rows 42-48, identifying speakers in the background audio as sensitive information / privacy tags; compare Bumburger, Col 60, Rows 8-10 and Rows 23-24, configuring VAD classifier to distinguish human voice activity from background noise).
Regarding Claim 15, Weinstein discloses wherein the de-identified audio signal and transcript includes the non-privacy information corresponding to the non-privacy tag in the audio signal and the transcript (Col 7, Rows 5-7, a portion of privacy-sensitive data may not include sensitive information and that portion may be able to be logged; e.g., Col 5, Rows 60-64, the portions of the request 107 (comprising speech data audio file per Col 3, Rows 45-47) and transcript 110 determined to be logged).
Weinstein does not disclose wherein the transcript further includes a non-privacy tag indicating non-privacy information in the audio signal.
Cline discloses generating privacy tag encoding first phrase representing privacy information and non-privacy tag encoding second phrase representing non-privacy information to markup a corresponding text data (Col 13, Rows 38-58, generating SSML tags associated with various sensitivity levels indicating how sensitive a particular portion of responsive text data is; Col 13, Rows 58-63, SSML includes a markup language providing a standard way to mark up text).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the non-privacy tag encoding of Weinstein (Col 5, Rows 60-64, sensitive information signal 123 indicating portions of the request 107 and transcript 110 to be logged) using SSML tags indicating various sensitivity levels as a standard way for marking up the transcript (Cline, Col 13, Rows 59-61, SSML tags provide a standard way to mark up text; i.e., markup transcript 110 with SSML tags indicating sensitive portions that cannot be logged and indicating non-sensitive portions that may be logged) to include the non-privacy tag indicating non-privacy information in the audio signal in order to provide differing sensitivity level designation for determining how to encrypt the transcript, how to store the transcript, and how to redact the transcript (Cline, Col 13, Rows 50-56).
Claims 2, 8, 14, 16-18 and 20 are rejected under 35 USC 103(a) as being unpatentable over Weinstein et al. (US 9990925 B2) in view of Bumberger et al. (US 12444407 B2), Cline et al. (US 10747894 B1) and Sharma et al. (US 2024/0404504 A1).
Regarding Claims 2 and 8, Weinstein discloses training the privacy tagged ASR model using a training transcript of a training audio signal (Col 6, Rows 10-16 and Rows 47-50, providing transcription data and audio file data as training data to model training engine that trains recognition models for the automated speech recognizer) and generating the transcript associated with the audio signal using the trained privacy tagged ASR model (Col 6, Rows 47-56, receive updated recognition model from the model training engine for use by the automated speech recognizer to recognize subsequently received audio data to generate a transcript per Col 5, Rows 45-50).
As modified by Bumburger, the combination implements the privacy tagged ASR model as a privacy tagged end-to-end ASR model (Bumburger, Col 27, Rows 4-5, implement an end-to-end ASR model).
The combination does not teach using privacy tags and non-privacy tags surrounding words in a training transcript of a training audio signal to train the privacy tagged end-to-end ASR model.
Sharma discloses training an ASR speech processing system / model using privacy tags surround words in a training transcript of a training audio signal (Sharma, ¶31, speech processing system 218 is an ASR system trained to ignore pseudo-speech representations, mark them as areas of sensitive information in the transcript; Sharma, ¶33, at training time, train speech processing ASR system 218 on data with pseudo-speech representations to ignore pseudo-speech representations).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to train the privacy tagged end-to-end ASR model using privacy tags and non-privacy tags surrounding words in a training transcript of a training audio signal (i.e., Cline, Col 13, Rows 59-61, SSML tags marking up text of the training transcript indicating various sensitivity levels indicating how sensitive a particular portion of text data is) in order to train the privacy tagged end-to-end ASR model to generate a transcript to mark areas of sensitive information in the transcript (Sharma, ¶31).
Regarding Claim 14, Weinstein discloses using a language model (LM) to extract transcriptions of speech (Col 6, Rows 19-24, speech recognizer uses neural network based acoustic model and language model to generate transcripts).
Weinstein does not disclose using the LM to extract one or more additional privacy tags from the transcript.
Sharma discloses using a language model to extract one or more additional privacy tags from a transcript (¶31 and Fig. 2, speech privacy process 10 uses a speech processing ASR system 218 to mark areas of sensitive information in a transcript);
combining, using the LM, the privacy tags in the transcript associated with the audio signal and the one or more additional privacy tags extracted from the transcript (¶31, speech privacy process 10 populates the transcript with pseudo-representations marked by the speech processing ASR system 218 as areas of sensitive information in the transcript; ¶38, speech privacy process 10 inserts pseudo-speech representations into the transcript); and
generating the de-identified audio signal and transcript based on the combination of the privacy tags in the audio signal and the one or more additional privacy tags extracted from the transcript (¶32, selectively insert pseudo-speech representations in for sensitive segments in the speech signal and the transcript).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to use a language model (Weinstein, Col 6, Rows 19-24, speech recognizer being a neural network based language model; compare Sharma, speech processing ASR system 218 in ¶31 being an equivalent structure to the neural network LM speech recognizer of Weinstein) to extract additional privacy tags / sensitive information marking from the transcript, combine the privacy tags, and generate the de-identified audio signal and transcript (Weinstein, Col 5, Rows 56-64, portions of the request 107 (comprising audio file data 108a) and portions of the transcription 110 corresponding to sensitive information are deleted per Col 6, Rows 25-28; compare Sharma, ¶32, insert pseudo-speech representations in sensitive segments in the speech signal and the transcript) based on the combination of the privacy tags in the audio signal and the one or more additional privacy tags extracted from the transcript in order to remove identity of the speaker and any personally identifiable information without diminishing accuracy of downstream speech processing (Sharma, ¶8).
Regarding Claim 16, Weinstein discloses a computer storage medium storing instructions that upon execution by a processor (Col 8, Rows 26-32, computer program product containing instructions embodied in memory 404, storage device 406, or memory on processor 402 to perform one or more methods when executed) cause the processor to1:
receive an audio signal comprising a first phrase representing privacy information and a second phrase representing non-privacy information (Col 3, Rows 35-38, receiving user utterance “This is Bob Smith. My credit card number is 01-234”; Col 3, Rows 44-47, generate speech recognition request 107 comprising an audio file encoding the utterance 108a and context data 108b);
generate, using a privacy tagged automatic speech recognition (ASR) model (Col 5, Rows 45-50, automated speech recognizer 120 uses a speech recognition model provided by training engine 118 to generate the transcript 110; Col 5, Rows 33-44, training engine 118 generates / updates a speech recognition model using received data from request 107 including likely sensitive data before any likely sensitive data is deleted; Col 7, Rows 17-23, continuously provide privacy sensitive data to model trainer 312 for training), a transcript associated with the audio signal (Col 3, Rows 63-65, speech processing system 106 receives the speech recognition request 107, analyzes the request 107, and generates a text transcription 110 of the utterance), the transcript is associated with a privacy tag encoding the first phrase representing the privacy information (Fig. 1, sensitive info identifier 116 outputs sensitive information signal 123 to sensitive information manager 122; Col 5, Row 61 – Col 6, Row 3, sensitive information signal 123 indicates which portions of the transcript 110 likely contains sensitive information and thus sensitive information manager 122 anonymizes a corresponding phrase; e.g., Col 6, Rows 1-3 and Col 7, Rows 8-9, anonymize “This is Bob Smith” to “This is <name>”) and a non-privacy tag encoding the second phrase representing the non-privacy information (Col 5, Rows 56-61 and Col 7, Rows 4-7, sensitive information signal 123 indicates which portions of the request 107 and the transcript 110 may not include sensitive information and that portion may be able to be logged; Col 5, Rows 61-66, based on the sensitive information signal 123, sensitive information manager 122 determines which portions of the request 107 and the transcript 110 are to be logged); and
replace a first audio segment corresponding to the privacy tag encoding the first phrase in the audio signal with an anonymized audio segment to generate de-identified audio signal (Col 5, Row 66 – Col 6, Row 1, sensitive information manager 122 may anonymize request data 107 (which comprises audio file data 108a per Col 3, Rows 44-47); see dependent claim 4: “anonymizing the at least the portion of the received audio data”), the de-identified audio signal including the anonymized audio segment and a second audio segment corresponding to the non-privacy tag encoding the second phrase in the audio signal (Col 5, Row 62 – Col 6, Row 1, sensitive information manager 122 determines portions of the request 107 (audio file data 108a) are to be logged and which portions are not to be logged such that when the sensitive information 122 anonymizes the request data 107 (audio file data 108a), the portions of request data 107 determined to be logged corresponds to non-sensitive audio while the portions of request data 107 determined not to be logged corresponds to sensitive audio that are anonymized).
Weinstein does not disclose the privacy tagged ASR model is an end-to-end automatic speech recognition (ASR) model.
Bumberger discloses using an end-to-end ASR model (Col 27, Rows 4-6) to generate interim ASR data (Col 65, Rows 10-11) to remove / obfuscate sensitive information (Col 70, Rows 18-20).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the privacy tagged ASR model as an end-to-end automatic speech recognition (ASR) model in order to remove / obfuscate sensitive information (Bumberger, Col 70, Rows 18-20).
Weinstein does not disclose the transcript includes the privacy tag encoding and the non-privacy tag encoding.
Cline discloses generating privacy tag encoding first phrase representing privacy information and non-privacy tag encoding second phrase representing non-privacy information to markup a corresponding text data (Col 13, Rows 38-58, generating SSML tags associated with various sensitivity levels indicating how sensitive a particular portion of responsive text data is; Col 13, Rows 58-63, SSML includes a markup language providing a standard way to mark up text).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the privacy tag encoding and the non-privacy tag encoding of Weinstein (Col 5, Rows 56-64, sensitive information signal 123 indicating portions of the transcript 110 are not to be logged for containing sensitive information and sensitive information signal 123 indicating portions of the transcript 110 are to be logged) using SSML tags indicating various sensitivity levels as a standard way for marking up the transcript (Cline, Col 13, Rows 59-61, SSML tags provide a standard way to mark up text; i.e., markup transcript 110 with SSML tags indicating sensitive portions that cannot be logged and indicating non-sensitive portions that may be logged) in order to provide differing sensitivity level designation for determining how to encrypt the transcript, how to store the transcript, and how to redact the transcript (Cline, Col 13, Rows 50-56).
Weinstein does not disclose the anonymized audio segment is a filler audio segment.
Sharma discloses replacing a first audio segment corresponding to a privacy tag encoding a first phrase in the audio signal with a filler audio segment to generate de-identified audio signal (¶8, a voice converter system processing a speech signal audio stream to insert pseudo-speech representations in regions that contain Personally Identifiable Information, the pseudo-speech representation is a portion of a speech signal with linguistic units of reduced acoustic quality such as a mumbling of word / phrase that is non-interpretable by a listener; ¶15, identify or flag sensitive speech portions).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to generate the de-identified audio signal by replacing the first audio segment corresponding to the privacy tag encoding the first phrase in the audio signal with an anonymized audio segment comprising a filler audio segment / pseudo-speech representation in order to mask or remove sensitive content without diminishing accuracy during downstream speech processing of the speech signal (Sharma, ¶8).
Regarding Claim 17, Weinstein discloses wherein the instructions upon execution by the processor cause the processor to generate a de-identified transcript by replacing the first phrase representing the privacy information with a filler phrase (Col 6, Rows 1-3, anonymize “This is Bob Smith” to “This is <name>”), the de-identified transcript including the filler phrase and the second phrase representing the non-privacy information (Col 7, Rows 4-7, log portions that does not include sensitive information; e.g., log “This is <name>” or “My credit card number is <> per Col 5, Rows 14-19”).
Regarding Claim 18, Weinstein discloses training the privacy tagged ASR model using a training transcript of a training audio signal (Col 6, Rows 10-16 and Rows 47-50, providing transcription data and audio file data as training data to model training engine that trains recognition models for the automated speech recognizer).
As modified by Bumburger, the combination implements the privacy tagged ASR model as a privacy tagged end-to-end ASR model (Bumburger, Col 27, Rows 4-5, implement an end-to-end ASR model).
The combination does not teach using privacy tags and non-privacy tags surrounding words in a training transcript of a training audio signal to train the privacy tagged end-to-end ASR model.
Sharma discloses training an ASR speech processing system / model using privacy tags surround words in a training transcript of a training audio signal (Sharma, ¶31, speech processing system 218 is an ASR system trained to ignore pseudo-speech representations, mark them as areas of sensitive information in the transcript; Sharma, ¶33, at training time, train speech processing ASR system 218 on data with pseudo-speech representations to ignore pseudo-speech representations).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to train the privacy tagged end-to-end ASR model using privacy tags and non-privacy tags surrounding words in a training transcript of a training audio signal (i.e., Cline, Col 13, Rows 59-61, SSML tags marking up text of the training transcript indicating various sensitivity levels indicating how sensitive a particular portion of text data is) in order to train the privacy tagged end-to-end ASR model to generate a transcript to mark areas of sensitive information in the transcript (Sharma, ¶31).
Regarding Claim 20, Weinstein discloses wherein the instructions upon execution by the processor cause the processor to expand boundary associated with the privacy tag in the audio signal by an amount based on background audio containing likely sensitive information (Col 4, Rows 42-48, sensitive information identifier 116 analyzes background audio present in the audio data 108a for likely sensitive information identifying speakers other than the user).
Weinstein does not disclose wherein the instructions upon execution by the processor cause the processor to expand boundary associated with the privacy tag in the audio signal by an amount using voice activity detection (VAD) in the audio signal.
Bumburger discloses expand boundary associated with the privacy tag in the audio signal by an amount using voice activity detection (VAD) in the audio signal (Col 60, Rows 6-33, implement VAD limited classifier configured to distinguish speech from background noise in order to determine whether speech is present in audio data).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to expand boundary associated with the privacy tag in the audio signal by an amount using voice activity detection (VAD) in the audio signal to distinguish sensitive information identifying speakers in the background audio (Weinstein, Col 4, Rows 42-48, identifying speakers in the background audio as sensitive information / privacy tags; compare Bumburger, Col 60, Rows 8-10 and Rows 23-24, configuring VAD classifier to distinguish human voice activity from background noise).
Claims 3-4 and 9-10 are rejected under 35 USC 103(a) as being unpatentable over Weinstein et al. (US 9990925 B2) in view of Bumberger et al. (US 12444407 B2) and Cline et al. (US 10747894 B1) as applied to claims 1 and 7, in further view of Henrotte et al. (US 2023/0410825 A1).
Regarding Claims 3 and 9, Weinstein does not teach wherein the instructions upon execution by the processor cause the processor to convert a voice of a speaker in the audio signal into a voice of a single target virtual agent speaker.
Henrotte discloses technique at anonymizing voice of a speaker in an audio signal (¶5) by converting the voice of the speaker into a voice of a single target virtual agent speaker (Abstract and ¶23, masking the voice of a speaker by altering the pitch and the timbre of the voice, combine the altered pitch and the altered timbre of the audio segment to form a single resulting altered audio segment; ¶84, form a single resulting altered audio segment).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to anonymize request data comprising voice of the speaker in the audio signal (Weinstein, Col 5, Row 67 – Col 6, Row 1, anonymizing request 107, which comprises speech audio file 108a per Col 3, Rows 45-47) by converting the voice of the speaker in the audio signal into a voice of a single target virtual agent speaker in order to anonymizing the voice of the speaker in the audio signal to protect identity and privacy (Henrotte, ¶5 and ¶¶14-15, masking the voice of the speaker in order to protect their identity and privacy).
Regarding Claims 4 and 10, Weinstein discloses wherein the de-identified audio signal includes the voice of the single target virtual agent speaker (Weinstein, Col 5, Row 67 – Col 6, Row 1, anonymize request data 107; as modified by Henrotte, ¶¶14-15 and ¶84, masking the voice of the speaker (i.e., audio file 108a in request data 107) by forming a single resulting altered audio segment).
Claim 19 are rejected under 35 USC 103(a) as being unpatentable over Weinstein et al. (US 9990925 B2) in view of Bumberger et al. (US 12444407 B2), Cline et al. (US 10747894 B1) and Sharma et al. (US 2024/0404504 A1) as applied to claim 16, in further view of Henrotte et al. (US 2023/0410825 A1).
Regarding Claim 19, Weinstein does not teach wherein the instructions upon execution by the processor cause the processor to convert the second audio segment in the audio signal into a voice of a single target virtual agent speaker, wherein the filler audio segment and the second audio segment in the de-identified audio signal are in the voice of the single target virtual agent speaker.
Henrotte discloses technique at anonymizing voice of a speaker in an audio signal (¶5) by converting the voice of the speaker into a voice of a single target virtual agent speaker (Abstract and ¶23, masking the voice of a speaker by altering the pitch and the timbre of the voice, combine the altered pitch and the altered timbre of the audio segment to form a single resulting altered audio segment; ¶84, form a single resulting altered audio segment).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to anonymize request data comprising voice of the speaker in the audio signal (Weinstein, Col 5, Row 67 – Col 6, Row 1, anonymizing request 107, which comprises speech audio file 108a per Col 3, Rows 45-47) by converting the filler audio segment and the second audio segment in the de-identified audio signal are in the voice of the single target virtual agent speaker in order to anonymizing the voice of the speaker in the audio signal to protect identity and privacy (Henrotte, ¶5 and ¶¶14-15, masking the voice of the speaker in order to protect their identity and privacy).
Conclusion
Prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2018/0330722 A1 discloses a digital assistant (¶209) for processing speech recognition result of user speech using language models to deduce user intent (¶212 and ¶218) and determining if the natural language input / recognition result corresponds to a privacy domain (¶249, determine “How many calories did I burn today?” as corresponding to a privacy domain / health domain).
US 2019/0371328 A1 discloses a speech processing system with a privacy mode associated with an identity of a user to detect user indication of activating a primary mode and generate a privacy mode flag indicating to any component of the speech processing system that any data relating to processing of the user’s utterances should not be sent to long term storage (Abstract, “delete data once the respective component is finished processing with respect to an utterance of the user”).
US 2025/0117516 A1 discloses training machine learning model with training data comprising keywords and audio frames corresponding to sensitive or confidential data (¶55) and applying the trained machine learning model to determine content element including sensitive information to perform encryption (Abstract; ¶78, apply machine learning model 117 to predict the presence of sensitive information in user natural language prompts).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to examiner Richard Z. Zhu whose telephone number is 571-270-1587 or examiner’s supervisor Hai Phan whose telephone number is 571-272-6338. Examiner Richard Zhu can normally be reached on M-Th, 0730:1700.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD Z ZHU/Primary Examiner, Art Unit 2654 09/30/2026
1 Specification at ¶50: “As defined herein, computer storage media does not include communication media. Therefore, a computer storage medium is not a propagating signal. Propagated signals are not examples of computer storage media”.
