DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 05/19/2026 have been fully considered but they are not persuasive. Regarding arguments filed on page 7, Examiner notes that in the referenced Office action from 9/18/2023, the limitation in question is not fully quoted. The limitation includes details about the training that are not taught by Dirac, but are also not claimed in the instant application. Since the limitation of the current claims remain broader than the referenced limitation, Dirac is still relied upon to teach them. Examiner further notes that regarding the use of machine learning, Dirac teaches that the accent translation models are updated and refined using machine learning by using new audio samples. Therefore, interpreting the accent translation models of Dirac as the machine-learning algorithm of the claims, Dirac teaches that the models are updated or trained using the new audio samples, the audio samples corresponding to each of the different accent sample sets corresponding to the different speakers. For example, Dirac col. 6 line 56 – col. 7 line 6 teaches different speakers with different accents, as well as multiple speakers for each accent.
Regarding arguments on pages 7-10 of the Remarks, Examiner notes that the Peng reference is not cited in the rejection of the independent claims. Moreover, the Peng reference in the cited Office action from 9/18/2023 relied upon Peng to teach limitations not found in the independent claims. Rather, Peng was relied upon to teach the limitations that are found in dependent claim 6 of the instant application, where details of the training are claimed. Since the independent claims only broadly state only that the algorithm is trained without details, the Dirac reference alone teaches the limitations.
Regarding arguments on page 11 of the Remarks, Examiner notes that while Peng teaches alignment and classification, that the Dirac reference teaches multiple speakers having the same accent in col. 6 line 56 – col. 7 line 6. Examiner further notes that classifying users as having a particular accent can be further extended to the users’ speech being classified as the same accent. For example, if user A is classified as having a Southern accent, then the speech from user A would also be classified as a Southern accent by extension. Further, Dirac also teaches the classifying of accents by separating the samples for the accent sample sets 131-134. Therefore, the limitations are taught by the combination of references.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-11, 13-17, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chang et al. (US 2014/0187210 A1), hereinafter referred to as Chang, in view of Dirac et al. (US 10,163,451 B2), hereinafter referred to as Dirac.
Regarding claim 1, Chang teaches:
A system, comprising an output audio device, a communication interface, memory having instructions stored thereon, and one or more processors coupled to the memory (Fig. 10 elements 1020, 1030, 1050, 1060, para [0069], where the components of the system are used) and configured to execute the instructions to:
receive first audio data via one or more networks and the communication interface (Fig. 8, para [0064], where the modified audio is transmitted to the user), wherein the first audio data comprises a synthesized version of first speech content associated with a first accent and generated using an output and a first machine-learning algorithm trained with second audio data associated with the first accent (Fig. 8, para [0040], [0054], [0064], where another speaker chooses to apply voice modifications to other speakers for synthesis in another accent), wherein the output is generated based on an application of a second machine-learning algorithm to the first speech content and the first speech content comprises a set of phonemes associated with a first pronunciation of the first speech content, wherein the second machine-learning algorithm is trained with second speech content from a first plurality of speakers having a second accent different than the first accent (Fig. 8, para [0063], where a speaker configures their voice to have an accent removed, and para [0031-32], where the phonemes are used to determine the dialect of the speaker);
store the first audio data in the memory (para [0070], where memory stores information); and
output the first audio data from the memory and via the output audio device (para [0030], where a filtered version of the input signal is output).
Chang does not explicitly teach machine learning algorithms and training data associated with different accents.
Dirac teaches:
a first machine-learning algorithm trained with second audio data associated with the first accent (Fig. 1 element 131-134, col. 3 lines 36-60, where sample sets for different accents are used, and col. 7 lines 52-60, where machine learning is used to update and refine the accent translation models);
wherein the second machine-learning algorithm is trained with third audio data comprising second speech content captured from a first plurality of speakers having a second accent different than the first accent (Fig. 1 element 131-134, col. 3 lines 36-60, where sample sets for different accents are used, and col. 7 lines 52-60, where machine learning is used to update and refine the accent translation models);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chang by using the machine learning of Dirac (Dirac col. 7 lines 52-60) in the accent modification of Chang (Chang para [0063-64]), in order to refine the accent translation models continually using new audio samples (Dirac col. 7 lines 52-60).
Regarding claim 2, Chang in view of Dirac teaches:
The system of claim 1, wherein the one or more processors are further configured to execute the instructions to output the first audio data via a digital communication application executed by the system (Chang para [0018], where the voice modification is implemented by an application installed in the user's smart phone).
Regarding claim 3, Chang in view of Dirac teaches:
The system of claim 1, wherein at least a first non-text linguistic representation of a first phoneme of the set of phonemes is mapped to a second non-text linguistic representation of a second phoneme associated with a second pronunciation of the first speech content (Chang para [0031], where phonemes are distinguished by a unique pattern or signature in a spectrogram, and para [0035], where the phoneme formant stream is modified to shape formant patterns to match phonemes in another dialect).
Regarding claim 4, Chang in view of Dirac teaches:
The system of claim 3, wherein one or more frames in the output are mapped to one or more corresponding frames in the first non-text linguistic representation (Chang para [0031], [0035], where a spectrogram represents the spectra of the frames, and where the locations of the phonemes are modified or shaped to form the modified spectrogram).
Regarding claim 5, Chang in view of Dirac teaches:
The system of claim 1, wherein the synthesized version of the first speech content retains a set of prosodic features included in the first speech content (Chang para [0035], where the synthesized voice removes the accent while still sounding like the speaker, and para [0046], where tone and tempo are options for modification).
Regarding claim 7, Chang in view of Dirac teaches:
The system of claim 1, wherein the synthesized version of first speech content is generated based on a continuous conversion of fourth audio data associated with the second accent (Chang para [0015], where the voice modification is performed in real time).
Regarding claim 8, Chang teaches:
One or more non-transitory computer-readable media having first audio data stored thereon (para [0073], where storage is used) comprising a synthesized version of first speech content associated with a first accent and generated using an output and a first machine-learning algorithm trained with second audio data associated with the first accent (Fig. 8, para [0040], [0054], [0064], where another speaker chooses to apply voice modifications to other speakers for synthesis in another accent), wherein the output is generated based on an application of a second machine-learning algorithm to the first speech content and the first speech content comprises a set of phonemes associated with a first pronunciation of the first speech content, wherein the second machine-learning algorithm is trained with third audio data comprising second speech content captured from a first plurality of speakers having a second accent different than the first accent (Fig. 8, para [0063], where a speaker configures their voice to have an accent removed, and para [0031-32], where the phonemes are used to determine the dialect of the speaker).
Chang does not explicitly teach machine learning algorithms and training data associated with different accents.
Dirac teaches:
a first machine-learning algorithm trained with second audio data associated with the first accent (Fig. 1 element 131-134, col. 3 lines 36-60, where sample sets for different accents are used, and col. 7 lines 52-60, where machine learning is used to update and refine the accent translation models);
wherein the second machine-learning algorithm is trained with second speech content from a first plurality of speakers having a second accent different than the first accent (Fig. 1 element 131-134, col. 3 lines 36-60, where sample sets for different accents are used, and col. 7 lines 52-60, where machine learning is used to update and refine the accent translation models);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chang by using the machine learning of Dirac (Dirac col. 7 lines 52-60) in the accent modification of Chang (Chang para [0063-64]), in order to refine the accent translation models continually using new audio samples (Dirac col. 7 lines 52-60).
Regarding claim 9, Chang in view of Dirac teaches:
The one or more non-transitory computer-readable media of claim 8, wherein the synthesized version of the first speech content comprises a first phoneme of the set of phonemes, wherein the first phoneme has a first non-text linguistic representation and is associated with a first pronunciation of the first speech content, wherein a second non-text linguistic representation of a second phoneme of the set of phonemes is mapped to the first non-text linguistic representation (Chang para [0031], where phonemes are distinguished by a unique pattern or signature in a spectrogram, and para [0035], where the phoneme formant stream is modified to shape formant patterns to match phonemes in another dialect).
Regarding claim 10, Chang in view of Dirac teaches:
The one or more non-transitory computer-readable media of claim 9, wherein one or more frames in the output are mapped to one or more corresponding frames in the first non-text linguistic representation (Chang para [0031], [0035], where a spectrogram represents the spectra of the frames, and where the locations of the phonemes are modified or shaped to form the modified spectrogram).
Regarding claim 11, Chang in view of Dirac teaches:
The one or more non-transitory computer-readable media of claim 8, wherein the synthesized version of the first speech content retains a set of prosodic features included in the first speech content (Chang para [0035], where the synthesized voice removes the accent while still sounding like the speaker, and para [0046], where tone and tempo are options for modification).
Regarding claim 13, Chang in view of Dirac teaches:
The one or more non-transitory computer-readable media of claim 8, wherein the synthesized version of first speech content is generated based on a continuous conversion of fourth audio data associated with the second accent (Chang para [0015], where the voice modification is performed in real time).
Regarding claim 14, Chang teaches:
A method, comprising:
receive first audio data via one or more networks (Fig. 8, para [0064], where the modified audio is transmitted to the user), wherein the first audio data comprises a synthesized version of first speech content associated with a first accent and generated using an output and a first machine-learning algorithm trained with second audio data associated with the first accent (Fig. 8, para [0040], [0054], [0064], where another speaker chooses to apply voice modifications to other speakers for synthesis in another accent), wherein the output is generated based on an application of a second machine-learning algorithm to the first speech content and the first speech content comprises a set of phonemes associated with a first pronunciation of the first speech content, wherein the second machine-learning algorithm is trained with third audio data comprising second speech content captured from a first plurality of speakers having a second accent different than the first accent (Fig. 8, para [0063], where a speaker configures their voice to have an accent removed, and para [0031-32], where the phonemes are used to determine the dialect of the speaker); and
output the first audio data via an output audio device, wherein the first audio data represents an accent-converted version of fourth audio data corresponding to the first speech content (para [0030], where a filtered version of the input signal is output).
Chang does not explicitly teach machine learning algorithms and training data associated with different accents.
Dirac teaches:
a first machine-learning algorithm trained with second audio data associated with the first accent (Fig. 1 element 131-134, col. 3 lines 36-60, where sample sets for different accents are used, and col. 7 lines 52-60, where machine learning is used to update and refine the accent translation models);
wherein the second machine-learning algorithm is trained with second speech content from a first plurality of speakers having a second accent different than the first accent (Fig. 1 element 131-134, col. 3 lines 36-60, where sample sets for different accents are used, and col. 7 lines 52-60, where machine learning is used to update and refine the accent translation models);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chang by using the machine learning of Dirac (Dirac col. 7 lines 52-60) in the accent modification of Chang (Chang para [0063-64]), in order to refine the accent translation models continually using new audio samples (Dirac col. 7 lines 52-60).
Regarding claim 15, Chang in view of Dirac teaches:
The method of claim 14, further comprising outputting the first audio data via a digital communication application (Chang para [0018], where the voice modification is implemented by an application installed in the user's smart phone).
Regarding claim 16, Chang in view of Dirac teaches:
The method of claim 14, wherein the synthesized version of the first speech content comprises a first phoneme of the set of phonemes, the first phoneme has a first non-text linguistic representation and is associated with a first pronunciation of the first speech content, and one or more frames in the output are mapped to one or more corresponding frames in the first non-text linguistic representation (Chang para [0031], where phonemes are distinguished by a unique pattern or signature in a spectrogram, and para [0035], where the phoneme formant stream is modified to shape formant patterns to match phonemes in another dialect).
Regarding claim 17, Chang in view of Dirac teaches:
The method of claim 14, wherein the first audio data corresponds to a single speaker having the first accent (Chang para [0032], where the dialect of a speaker is known ahead of time).
Regarding claim 19, Chang in view of Dirac teaches:
The method of claim 14, wherein one or more frames in the output are mapped to one or more corresponding frames in the first non-text linguistic representation (Chang para [0031], [0035], where a spectrogram represents the spectra of the frames, and where the locations of the phonemes are modified or shaped to form the modified spectrogram).
Regarding claim 20, Chang in view of Dirac teaches:
The method of claim 14, wherein the synthesized version of the first speech content comprises a first phoneme of the set of phonemes, wherein the first phoneme has a first non-text linguistic representation and is associated with a first pronunciation of the first speech content, wherein a second non-text linguistic representation of a second phoneme of the set of phonemes is mapped to the first non-text linguistic representation (Chang para [0031], where phonemes are distinguished by a unique pattern or signature in a spectrogram, and para [0035], where the phoneme formant stream is modified to shape formant patterns to match phonemes in another dialect).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chang, in view of Dirac, and further in view of Peng et al. (US 2015/0170642 A1), hereinafter referred to as Peng.
Regarding claim 6, Chang in view of Dirac teaches:
The system of claim 1,
Chang in view of Dirac does not teach:
wherein the second machine-learning algorithm is trained based on an alignment and classification of each of a plurality of frames of the first speech content corresponding to respective ones of the first plurality of speakers having the second accent.
Peng teaches:
wherein the second machine-learning algorithm is trained based on an alignment and classification of each of a plurality of frames of the first speech content corresponding to respective ones of the first plurality of speakers having the second accent (para [0025-26], where alignment of frames is performed, and para [0030], where the accent group identifier is determined).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chang in view of Dirac by using the training of Peng (Peng para [0018]) on the system of Chang in view of Dirac (Chang para [0063-64]) to associate actual pronunciations to an expected pronunciation and perform replacements of substitutions before processing an utterance (Peng para [0003]).
Claim(s) 12, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chang, in view of Dirac, and further in view of Peng, and Ganapathiraju et al. (US 2014/0025379 A1), hereinafter referred to as Ganapathiraju.
Regarding claim 12, Chang in view of Dirac teaches:
The one or more non-transitory computer-readable media of claim 8, wherein the second machine-learning algorithm comprises a non-text learned linguistic representation for the second accent (Chang para [0031], [0035], where spectrograms are used, containing formants representing the phonemes),
Chang in view of Dirac does not teach:
wherein the second machine-learning algorithm is trained based on an alignment and classification of a plurality of frames of captured speech content according to monophone and triphone sounds of the captured speech content.
Peng teaches:
wherein the second machine-learning algorithm is trained based on an alignment and classification of a plurality of frames of captured speech content according to phoneme sounds of the captured speech content (para [0025-26], where alignment of frames is performed, and para [0030], where the accent group identifier is determined).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chang in view of Dirac by using the training of Peng (Peng para [0018]) on the system of Chang in view of Dirac (Chang para [0063-64]) to associate actual pronunciations to an expected pronunciation and perform replacements of substitutions before processing an utterance (Peng para [0003]).
Ganapathiraju teaches:
monophone and triphone sounds of the captured speech content (para [0023], where monophones and triphones are both used)
Chang in view of Dirac and Peng teaches processing using phonemes (Chang para [0031]). However, claim 1 recites that monophones and triphones are used. Ganapathiraju teaches using monophones and triphones (Ganapathiraju para [0023]). Para [0023] recognizes that phonemes can be modeled in isolation or in context of other phonemes, both being withing the level of ordinary skill in the art, and predictable in usage. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have substituted the monophones and triphones of Ganapathiraju in place of the phonemes of Chang in view of Dirac and Peng, where the result of the substitution would predictably allow for processing of individual phonemes or phonemes in context of other phonemes.
Regarding claim 18, Chang in view of Dirac teaches:
The method of claim 14, wherein the second machine-learning algorithm comprises a non-text learned linguistic representation for the second accent (Chang para [0031], [0035], where spectrograms are used, containing formants representing the phonemes),
Chang in view of Dirac does not teach:
wherein the second machine-learning algorithm is trained based on an alignment and classification of a plurality of frames of captured speech content according to monophone and triphone sounds of the captured speech content.
Peng teaches:
wherein the second machine-learning algorithm is trained based on an alignment and classification of a plurality of frames of captured speech content according to phoneme sounds of the captured speech content (para [0025-26], where alignment of frames is performed, and para [0030], where the accent group identifier is determined).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chang in view of Dirac by using the training of Peng (Peng para [0018]) on the system of Chang in view of Dirac (Chang para [0063-64]) to associate actual pronunciations to an expected pronunciation and perform replacements of substitutions before processing an utterance (Peng para [0003]).
Ganapathiraju teaches:
monophone and triphone sounds of the captured speech content (para [0023], where monophones and triphones are both used)
Chang in view of Dirac and Peng teaches processing using phonemes (Chang para [0031]). However, claim 1 recites that monophones and triphones are used. Ganapathiraju teaches using monophones and triphones (Ganapathiraju para [0023]). Para [0023] recognizes that phonemes can be modeled in isolation or in context of other phonemes, both being withing the level of ordinary skill in the art, and predictable in usage. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have substituted the monophones and triphones of Ganapathiraju in place of the phonemes of Chang in view of Dirac and Peng, where the result of the substitution would predictably allow for processing of individual phonemes or phonemes in context of other phonemes.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 2015/0371633 A1 para [0040] teaches alignment between speech frames using the pronunciations.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRYAN S BLANKENAGEL whose telephone number is (571)270-0685. The examiner can normally be reached 8:00am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRYAN S BLANKENAGEL/Primary Examiner, Art Unit 2658
Prosecution Insights