DETAILED ACTION
This communication is in response to the Application filed on October 23, 2024.
Claims 1 - 14 are pending and have been examined.
Claims 1 and 14 are independent.
Foreign priority: May 31, 2022.
PCT/KR2023/006781 was filed on May 18, 2023.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on October 23, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
The drawings filed on October 23, 2024 have been accepted and considered by the Examiner.
Double Patenting Note
The Examiner notes that previously published patent application U.S. Patent Application Publication 2023/0076239 was analyzed for Double Patenting. However, based on the current claim scope no Double patenting was found.
35 U.S.C. 112(f) Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are: “speaker embedding generation unit” and “speaker embedding prediction unit” in Claim 14. Note the varied definition of this phrase in the supporting Specification which indicates that the “unit” was intended as a generic placeholder. Based on the Specification, this refers to a large number of options: [66] A processor of the speaker adaptation system 100 may include a speaker embedding generation unit 710 and a speaker embedding prediction unit 720. Such components of the processor may be expressions of different functions that are performed by the processor based on a control command that is provided by a program code stored in the speaker adaptation system. The processor and the components of the processor may control the speaker adaptation system so that the speaker adaptation system performs steps (S810 to S820) included in the speaker adaptation method of FIG. 8. In this case, the processor and the components of the processor may be implemented to execute instructions according to a code of an operating system that is included in memory and a code of at least one program.”
These limitations are generic in the context of the art and don’t refer to any specific structure and only serve as placeholders for the structure that performs the associated function(s) without providing any information about what that structure is. MPEP 2181 I A says:
For a term to be considered a substitute for "means," and lack sufficient structure for performing the function, it must serve as a generic placeholder and thus not limit the scope of the claim to any specific manner or structure for performing the claimed function. It is important to remember that there are no absolutes in the determination of terms used as a substitute for "means" that serve as generic placeholders. The examiner must carefully consider the term in light of the specification and the commonly accepted meaning in the technological art. Every application will turn on its own facts.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1, 13 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Li et al., (U.S. Patent Application Publication 2025/0349282), hereinafter referred to as Li, in view of Khoury et al., (U.S. Patent Application Publication 2021/0326421), hereinafter referred to as Khoury.
Regarding Claims 1 and 14, Li teaches:
1. A speaker adaptation method performed by a speaker adaptation system, the speaker adaptation method comprising steps of:
generating a plurality of speaker embeddings that represents a tone of a speaker from a speaker embedding by using a voice conversion model comprising a global style token (GST) mechanism; and [Li, “The speaker embeddings 528 (i.e., the claimed “plurality of speaker embeddings”), for example, comprise the embeddings that are generated by the speaker encoder 522 in response to speaker input samples.” Par. 0075; “global style token 512, global prosodic features 514, and speaker embedding 528) to generate a combination of embeddings that are provided to the variance adaptor 516.” Par. 0074; “It also predicts the phone-level fundamental frequency, which is the relative highness or lowness of a tone as perceived by human.” Par. 0079]
predicting a final speaker embedding that represents a new speaker based on a comparison of similarity between a new speaker embedding that is predicted by using a prediction model that predicts a speaker embedding and the plurality of generated speaker embeddings. [Li, “Additionally, during training, the predicted embedding 524 is aligned with speaker embedding 528 (i.e., the claimed “predicting a final speaker embedding”) using a cycle loss training method (e.g., cycle loss 525), such that the predicted speaker embedding is more accurately aligned with the target speaker's natural speaking voice.” Par. 0086]
Li fails to explicitly teach a comparison of similarity between a new speaker embedding that is predicted by using a prediction model that predicts a speaker embedding and the plurality of generated speaker embeddings.
However, Khoury teaches:
predicting a final speaker embedding that represents a new speaker based on a comparison of similarity between a new speaker embedding that is predicted by using a prediction model that predicts a speaker embedding and the plurality of generated speaker embeddings. [Khoury, “compares an speaker embedding extracted from an inbound audio signal against an enrolled embedding (voiceprint) and computes a similarity or prediction score.” Par. 0051; “similarity score based upon a distance between the inbound embedding (i.e., the claimed “new speaker embedding”) and a voiceprint stored in speaker profile in a speaker profile database (i.e., the claimed “plurality of generated speaker embeddings)” Par. 0013; “Components of the analytics system (i.e., the claimed “prediction model”) 101, such as the analytics server 102, generate voiceprints, update voiceprints, predict a similarity score,” Par. 0085]
Li and Khoury pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury in order to automatically “differentiate between speakers” (Khoury, Par. 0010).
Regarding Claim 13, Li in view of Khoury has been discussed above. The combination further teaches:
13. A computer program which is stored in a non-transitory computer-readable recording medium in order to execute the speaker adaptation method according to claim 1 in the speaker adaptation system. [Li, “Physical computer-readable storage media/devices (i.e., the claimed “computer readable recording medium”) are hardware and include RAM, ROM, EEPROM, CD-ROM or other optical disk storage (such as CDs, DVDs, etc.), magnetic disk storage or other magnetic storage devices, or any other hardware which can be used to store desired program (i.e., the claimed “computer program”) code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer.” Par. 0117]
Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Li in view of Khoury as applied in claim 1 above, and in further view of Pan et al., (U.S. Patent Application Publication 2023/0081659), hereinafter referred to as Pan, and Chen (CN113763924A).
Regarding Claim 2, Li in view of Khoury has been discussed above. The combination further teaches:
wherein the step of generating comprises steps of:
constructing the voice conversion model comprising the global style token mechanism, [Li, see mapping applied to claim 1; Khoury, see mapping applied to claim 1]
extracting the speaker embedding corresponding to a speaker ID through a speaker embedding table by using the constructed voice conversion model, [Li, see mapping applied to claim 1; Khoury, see mapping applied to claim 1; Li, “In some instances, the language embedding is accessed from a look up table (i.e., the claimed “speaker embedding table”).” Par. 0087; “The speaker encoder module then takes the acoustic features as input, and output speaker embedding, which represents speaker identity (i.e., the claimed “extracting the speaker embedding corresponding to a speaker ID”) of target speaker.” Par. 0063]
predicting a variance of a Gaussian distribution for the extracted speaker embedding through the global style token mechanism, and [Li, see mapping applied to claim 1; Khoury, see mapping applied to claim 1; “Additionally, during training, the predicted embedding 524 is aligned with speaker embedding 528 (i.e., the claimed “predicting a final speaker embedding”) using a cycle loss training method (e.g., cycle loss 525), such that the predicted speaker embedding is more accurately aligned with the target speaker's natural speaking voice.” Par. 0086; “global style token 512, global prosodic features 514, and speaker embedding 528) to generate a combination of embeddings that are provided to the variance adaptor 516.” Par. 0074]
the extracted speaker embedding is a potential vector that represents a tone of each speaker. [Li, see mapping applied to claim 1; Khoury, see mapping applied to claim 1; Li, “It also predicts the phone-level fundamental frequency, which is the relative highness or lowness of a tone as perceived by human.” Par. 0079; “The speaker encoder module then takes the acoustic features as input, and output speaker embedding (i.e., the claimed “extracted speaker embedding”), which represents speaker identity of target speaker.” Par. 0063]
The combination fails to explicitly teach speaker ID, variance of a Gaussian distribution, and vector.
However, Pan teaches:
extracting the speaker embedding corresponding to a speaker ID through a speaker embedding table by using the constructed voice conversion model, [Pan, “Predicted acoustic features may be generated based at least on a state sequence corresponding to the text, a speaker embedding vector corresponding to the speaker ID, and the style embedding vector.” Par. 0004]
the extracted speaker embedding is a potential vector that represents a tone of each speaker. [Pan, “Predicted (i.e., the claimed “potential”) acoustic features may be generated based at least on a state sequence corresponding to the text, a speaker embedding vector (i.e., the claimed “speaker embedding is a potential vector”) corresponding to the speaker ID, and the style embedding vector.” Par. 0004]
The combination fails to explicitly teach variance of a Gaussian distribution.
However, Chen teaches:
predicting a variance of a Gaussian distribution for the extracted speaker embedding through the global style token mechanism, and [Chen, “speaker embedding,” Par. n0049; “variance of a multivariate Gaussian distribution,” Par. n0010; “prediction module,” Par. n0013]
Li, Khoury, Pan and Chen pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury, the teachings of “speaker embedding vector corresponding to the speaker ID” (Pan, Par. 0004) taught by Pan, and the teachings of “variance of a multivariate Gaussian distribution” (Chen, Par. n0010) taught by Chen in order to automatically “differentiate between speakers” (Khoury, Par. 0010), “synthesize speech in a specific speaking style” (Pan, 0021), and overcome problems of “high requirements for noise, the speaker’s accent and fluency” (Chen, Par. n0003).
Claims 5 - 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Li in view of Khoury as applied in claim 1 above, and in further view of Gabrys et al., (U.S. Patent Application Publication 2023/0260502), hereinafter referred to as Gabrys.
Regarding Claim 5, Li in view of Khoury has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises steps of: [Li, see mapping applied to claim 1; Khoury, see mapping applied to claim 1]
constructing the prediction model that predicts the speaker embedding, and [Li, see mapping applied to claim 1; Khoury, see mapping applied to claim 1]
receiving a selected speaker embedding, among a plurality of speaker embeddings generated in the constructed prediction model, and a fundamental frequency of the new speaker. [Li, see mapping applied to claim 1; Khoury, see mapping applied to claim 1]
The combination fails to teach fundamental frequency of the new speaker.
However, Gabrys teaches:
receiving a selected speaker embedding, among a plurality of speaker embeddings generated in the constructed prediction model, and a fundamental frequency of the new speaker. [Gabrys, “The voice characteristic data may include speaker embedding data 166 and/or frequency data (“f.sub.0”) 168 of the target speech (i.e., the claimed “fundamental frequency of the new speaker”).” Par. 0032; “The frequency data 168 may include pitch and/or pitch contour information that indicates a fundamental frequency of the speech and/or any modulation of pitch (e.g., upward or downward inflections). For example, the frequency data 168 may include a fundamental frequency “f.sub.0” representing a pitch of the speech.” Par. 0032]
Li, Khoury and Gabrys pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury and “fundamental frequency” (Gabrys, Par. 0032) taught by Gabrys in order to automatically “differentiate between speakers” (Khoury, Par. 0010) and “modify the audio data to approximate different voice characteristics” (Gabrys, Par. 0020).
Regarding Claim 6, Li in view of Khoury and Gabrys has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises a step of selecting a speaker having a pitch contour of the new speaker, among trained speakers, through the voice conversion model. [Li, see mapping applied to claims 1, 5; Khoury, see mapping applied to claims 1, 5; Gabrys, see mapping applied to claim 5; Gabrys, “The frequency data 168 may include pitch and/or pitch contour information that indicates a fundamental frequency of the speech and/or any modulation of pitch (e.g., upward or downward inflections). For example, the frequency data 168 may include a fundamental frequency “f.sub.0” representing a pitch of the speech.” Par. 0032]
Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Li in view of Khoury and Gabrys as applied in claim 5 above, and in further view of Battenberg et al., (U.S. Patent Application Publication 2020/0372897), hereinafter referred to as Battenberg.
Regarding Claim 7, Li in view of Khoury and Gabrys has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises a step of selecting a speaker having a low value of Kullback-Leibler (KL) divergence as the speaker embedding based on a comparison of similarity using the KL divergence between the pitch contour of the new speaker and pitch contours of the trained speakers. [Li, see mapping applied to claims 1, 5-6; Khoury, see mapping applied to claims 1, 5-6; Gabrys, see mapping applied to claim 5-6; Gabrys, “The frequency data 168 may include pitch and/or pitch contour information that indicates a fundamental frequency of the speech and/or any modulation of pitch (e.g., upward or downward inflections). For example, the frequency data 168 may include a fundamental frequency “f.sub.0” representing a pitch of the speech.” Par. 0032]
The combination fails to teach Kullback-Leibler (KL) divergence.
However, Battenberg teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises a step of selecting a speaker having a low value of Kullback-Leibler (KL) divergence as the speaker embedding based on a comparison of similarity using the KL divergence between the pitch contour of the new speaker and pitch contours of the trained speakers. [Battenberg, “In these examples, the adjustable variational bound may include an adjustable KL (i.e., the claimed “Kullback-Leibler (KL) divergence”) term that provides an upper bound on the variational embedding (i.e., the claimed “speaker embedding”). Optionally, the adjustable variational bound may include a tunable KL weight that provides an upper bound on the variational embedding. Increasing the adjustable variational bound may increase the specified capacity of the variational embedding while decreasing the adjustable variational bound may decrease the specified capacity of the variational embedding.” Par. 0007; “The bound in Equation 8 follows from Equation 7 and the non-negativity of the KL divergence, wherein Equation 7 shows that the slack on the bound is the KL divergence between the aggregated posterior, q(z), and the prior, p(z).” Par. 0077; “For instance, when a reference speaker has a different pitch range (i.e., the claimed “pitch contour”) than the target speaker, the synthesized speech 450 may still sound like the target speaker since a suitable variational embedding 420 can be sampled by the reference encoder 410 when the reference speaker y.sub.s is provided.” Par. 0060; “Speaker information is represented as learned speaker-wise embedding (i.e., the claimed “speaker embedding”) vectors,” Par. 0085]
Li, Khoury, Gabrys and Battenberg pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury, “fundamental frequency” (Gabrys, Par. 0032) taught by Gabrys and “KL divergence” (Battenberg, Par. 0084) taught by Battenberg in order to automatically “differentiate between speakers” (Khoury, Par. 0010), “modify the audio data to approximate different voice characteristics” (Gabrys, Par. 0020), and enable “variational embeddings”/adaptable speaker embeddings (Battenberg, Par. 0007).
Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Li in view of Khoury, Gabrys and Battenberg as applied in claim 7 above, and in further view of Kim et al., (KR102495455B1), hereinafter referred to as Kim, Finkelstein et al., (U.S. Patent Application Publication 2022/0051654), hereinafter referred to as Finkelstein, and Garbacea et al., (U.S. Patent Application Publication 2020/0234725), hereinafter referred to as Garbacea.
Regarding Claim 8, Li in view of Khoury, Gabrys and Battenberg has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises steps of: [Li, see mapping applied to claims 1, 5-7; Khoury, see mapping applied to claims 1, 5-7; Gabrys, see mapping applied to claims 5-7; Battenberg, see mapping applied to claim 7]
extracting a pitch embedding by inputting a pitch contour of the new speaker to a pitch embedding table, [Li, see mapping applied to claims 1, 5-7; Khoury, see mapping applied to claims 1, 5-7; Gabrys, see mapping applied to claims 5-7; Battenberg, see mapping applied to claim 7]
generating a global pitch embedding through a convolutional neural network (CNN) and mean pooling for the extracted pitch embedding, and [Li, see mapping applied to claims 1, 5-7; Khoury, see mapping applied to claims 1, 5-7; Gabrys, see mapping applied to claims 5-7; Battenberg, see mapping applied to claim 7; Li, “convolutional neural network,” Par. 0057]
generating a new speaker embedding that represents a tone of the new speaker by combining the global pitch embedding and the selected speaker embedding through the prediction model. [Li, see mapping applied to claims 1, 5-7; Khoury, see mapping applied to claims 1, 5-7; Gabrys, see mapping applied to claims 5-7; Battenberg, see mapping applied to claim 7]
The combination fails to teach pitch embedding table, global pitch embedding and mean pooling.
However, Kim teaches:
extracting a pitch embedding by inputting a pitch contour of the new speaker to a pitch embedding table, [Kim, “While FastSpeech2's pitch (energy) predictor outputs a pitch (energy) embedding selected from an embedding table (i.e., the claimed “pitch embedding table”),” Par. 0037]
The combination fails to teach global pitch embedding and mean pooling.
However, Finkelstein teaches:
generating a global pitch embedding through a convolutional neural network (CNN) and mean pooling for the extracted pitch embedding, and [Finkelstein, “Accordingly, a given text input that is associated with a high degree of prosodic variation can produce synthesized speech with local changes in pitch and speaking duration to convey different semantic meanings, and also with global changes in the overall pitch trajectory to convey different moods and emotions.” Par. 0028; “In the example shown, the first TTS system 210 receives, as input, a text utterance 320 and optional other inputs 325, that may include speaker characteristics (e.g., speaker embedding Z) of the target voice (i.e., the claimed “global pitch embedding”).” Par. 0043]
The combination fails to teach mean pooling.
However, Garbacea teaches:
generating a global pitch embedding through a convolutional neural network (CNN) and mean pooling for the extracted pitch embedding, and [Garbacea, “For example in some implementations generating the speaker vector comprises applying mean pooling over the encoder vectors.” Par. 0013]
Li, Khoury, Gabrys, Battenberg, Kim, Finkelstein and Garbacea pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury, “fundamental frequency” (Gabrys, Par. 0032) taught by Gabrys, “KL divergence” (Battenberg, Par. 0084) taught by Battenberg, “pitch (energy) embedding selected from an embedding table (i.e., the claimed “pitch embedding table”)” (Kim, Par. 0037) taught by Kim, “global changes in the overall pitch trajectory”/global pitch embedding (Finkelstein, Par. 0028) taught by Finkelstein, and “mean pooling” (Garbacea, Par. 0013) taught by Garbacea in order to automatically “differentiate between speakers” (Khoury, Par. 0010), “modify the audio data to approximate different voice characteristics” (Gabrys, Par. 0020), enable “variational embeddings”/adaptable speaker embeddings (Battenberg, Par. 0007), overcome problems of “degradation of sound quality or expressiveness” (Kim, Par. 0002), and synthesize “expressive speech” (Finkelstein, Par. 0004), and “provide high quality audio, e.g., speech,” (Garbacea, Par. 0007).
Claims 9 - 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over Li in view of Khoury, Gabrys, Battenberg, Kim, Finkelstein and Garbacea as applied in claim 8 above, and in further view of Chen (CN113763924A).
Regarding Claim 9, Li in view of Khoury, Gabrys, Battenberg, Kim, Finkelstein and Garbacea has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises steps of: [Li, see mapping applied to claims 1-2, 5-8; Khoury, see mapping applied to claims 1-2, 5-8; Gabrys, see mapping applied to claims 5-8; Battenberg, see mapping applied to claims 7-8; Kim, see mapping applied to claim 8; Finkelstein, see mapping applied to claim 8; Garbacea, see mapping applied to claim 8]
predicting a Gaussian distribution of the new speaker by inputting the generated new speaker embedding to the global style token as a query, and [Li, see mapping applied to claims 1-2, 5-8; Khoury, see mapping applied to claims 1-2, 5-8; Gabrys, see mapping applied to claims 5-8; Battenberg, see mapping applied to claims 7-8; Kim, see mapping applied to claim 8; Finkelstein, see mapping applied to claim 8; Garbacea, see mapping applied to claim 8]
extracting a plurality of new speaker embeddings from the Gaussian distribution. [Li, see mapping applied to claims 1-2, 5-8; Khoury, see mapping applied to claims 1-2, 5-8; Gabrys, see mapping applied to claims 5-8; Battenberg, see mapping applied to claims 7-8; Kim, see mapping applied to claim 8; Finkelstein, see mapping applied to claim 8; Garbacea, see mapping applied to claim 8]
The combination fails to teach Gaussian distribution.
However, Chen teaches:
predicting a Gaussian distribution of the new speaker by inputting the generated new speaker embedding to the global style token as a query, and [Chen, “speaker embedding,” Par. n0049; “variance of a multivariate Gaussian distribution,” Par. n0010; “prediction module,” Par. n0013]
extracting a plurality of new speaker embeddings from the Gaussian distribution. [Chen, “speaker embedding,” Par. n0049; “variance of a multivariate Gaussian distribution,” Par. n0010; “prediction module,” Par. n0013]
Li, Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea and Chen pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury, “fundamental frequency” (Gabrys, Par. 0032) taught by Gabrys, “KL divergence” (Battenberg, Par. 0084) taught by Battenberg, “pitch (energy) embedding selected from an embedding table (i.e., the claimed “pitch embedding table”)” (Kim, Par. 0037) taught by Kim, “global changes in the overall pitch trajectory”/global pitch embedding (Finkelstein, Par. 0028) taught by Finkelstein, “mean pooling” (Garbacea, Par. 0013) taught by Garbacea, and teachings of “Gaussian distribution” (Chen, Par. n0010) taught by Chen in order to automatically “differentiate between speakers” (Khoury, Par. 0010), “modify the audio data to approximate different voice characteristics” (Gabrys, Par. 0020), enable “variational embeddings”/adaptable speaker embeddings (Battenberg, Par. 0007), overcome problems of “degradation of sound quality or expressiveness” (Kim, Par. 0002), and synthesize “expressive speech” (Finkelstein, Par. 0004), “provide high quality audio, e.g., speech,” (Garbacea, Par. 0007), and overcome problems of “high requirements for noise, the speaker’s accent and fluency” (Chen, Par. n0003).
Regarding Claim 10, Li in view of Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea and Chen has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises a step of selecting one new speaker embedding capable of most similarly representing an actual voice of the new speaker, among the plurality of extracted new speaker embeddings. [Li, see mapping applied to claims 1-2, 5-9; Khoury, see mapping applied to claims 1-2, 5-9; Gabrys, see mapping applied to claims 5-9; Battenberg, see mapping applied to claims 7-9; Kim, see mapping applied to claims 8-9; Finkelstein, see mapping applied to claims 8-9; Garbacea, see mapping applied to claims 8-9; Chen, see mapping applied to claim 9]
Claims 11 are rejected under 35 U.S.C. 103(a) as being unpatentable over Li in view of Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea, and Chen as applied in claim 10 above, and in further view of Moreno (WO2021112840A1), hereinafter referred to as Moreno.
Regarding Claim 11, Li in view of Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea and Chen has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises steps of: [Li, see mapping applied to claims 1-2, 5-10; Khoury, see mapping applied to claims 1-2, 5-10; Gabrys, see mapping applied to claims 5-10; Battenberg, see mapping applied to claims 7-10; Kim, see mapping applied to claims 8-10; Finkelstein, see mapping applied to claims 8-10; Garbacea, see mapping applied to claims 8-10; Chen, see mapping applied to claims 9-10]
selecting noise having a smallest difference from an actual voice from the Gaussian distribution of the new speaker, and [Li, see mapping applied to claims 1-2, 5-10; Khoury, see mapping applied to claims 1-2, 5-10; Gabrys, see mapping applied to claims 5-10; Battenberg, see mapping applied to claims 7-10; Kim, see mapping applied to claims 8-10; Finkelstein, see mapping applied to claims 8-10; Garbacea, see mapping applied to claims 8-10; Chen, see mapping applied to claims 9-10]
The combination fails to teach obtaining a speaker embedding that represents the new speaker by adding the selected noise to the new speaker embedding.
However, Moreno teaches:
obtaining a speaker embedding that represents the new speaker by adding the selected noise to the new speaker embedding. [Moreno, “speaker embedding for the target user”, Par. 0003; “Noisy audio data 156 can be generated by combining (i.e., the claimed “adding”) the utterance(s) spoken by the target user with additional noise (i.e., the claimed “selected noise”),” Par. 0027]
Li, Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea, Chen and Moreno pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury, “fundamental frequency” (Gabrys, Par. 0032) taught by Gabrys, “KL divergence” (Battenberg, Par. 0084) taught by Battenberg, “pitch (energy) embedding selected from an embedding table (i.e., the claimed “pitch embedding table”)” (Kim, Par. 0037) taught by Kim, “global changes in the overall pitch trajectory”/global pitch embedding (Finkelstein, Par. 0028) taught by Finkelstein, “mean pooling” (Garbacea, Par. 0013) taught by Garbacea, teachings of “Gaussian distribution” (Chen, Par. n0010) taught by Chen, and the teachings of “combining (i.e., the claimed “adding”) the utterance(s) spoken by the target user with additional noise (i.e., the claimed “selected noise”)” (Moreno, Par. 0027) taught by Moreno in order to automatically “differentiate between speakers” (Khoury, Par. 0010), “modify the audio data to approximate different voice characteristics” (Gabrys, Par. 0020), enable “variational embeddings”/adaptable speaker embeddings (Battenberg, Par. 0007), overcome problems of “degradation of sound quality or expressiveness” (Kim, Par. 0002), and synthesize “expressive speech” (Finkelstein, Par. 0004), “provide high quality audio, e.g., speech,” (Garbacea, Par. 0007), overcome problems of “high requirements for noise, the speaker’s accent and fluency” (Chen, Par. n0003), and “generate different output that is personalized” (Moreno, Par. 0003).
Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Li in view of Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea, Chen and Moreno as applied in claim 11 above, and in further view of Liu (CN116601702A), hereinafter referred to as Liu.
Regarding Claim 12, Li in view of Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea, Chen and Moreno has been discussed above. The combination further teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises a step of generating the final speaker embedding that represents the new speaker by fine-tuning a speaker embedding that represents the obtained new speaker as data of the new speaker. [Li, see mapping applied to claims 1-2, 5-11; Khoury, see mapping applied to claims 1-2, 5-11; Gabrys, see mapping applied to claims 5-11; Battenberg, see mapping applied to claims 7-11; Kim, see mapping applied to claims 8-11; Finkelstein, see mapping applied to claims 8-11; Garbacea, see mapping applied to claims 8-11; Chen, see mapping applied to claims 9-11]
The combination fails to teach fine-tuning.
However, Liu teaches:
wherein the step of predicting the final speaker embedding that represents the new speaker comprises a step of generating the final speaker embedding that represents the new speaker by fine-tuning a speaker embedding that represents the obtained new speaker as data of the new speaker. [Liu, “Such processes and information can be used to fine-tune pre-trained multi-speaker and multilingual models using data from one or more different target speakers and for one or more different target languages.” Par. n0084]
Li, Khoury, Gabrys, Battenberg, Kim, Finkelstein, Garbacea, Chen, Moreno and Liu pertain to voice analytical systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the voice analytical systems art to modify Li’s teachings of “global style tokens” and “speaker embeddings” (Li, Par. 0074) with the teachings of “computing similarity” to “compare a speaker embedding”/ new speaker embedding with database/plurality of generated speaker embeddings” (Khoury, Par. 0051, Par. 0013) taught by Khoury, “fundamental frequency” (Gabrys, Par. 0032) taught by Gabrys, “KL divergence” (Battenberg, Par. 0084) taught by Battenberg, “pitch (energy) embedding selected from an embedding table (i.e., the claimed “pitch embedding table”)” (Kim, Par. 0037) taught by Kim, “global changes in the overall pitch trajectory”/global pitch embedding (Finkelstein, Par. 0028) taught by Finkelstein, “mean pooling” (Garbacea, Par. 0013) taught by Garbacea, teachings of “Gaussian distribution” (Chen, Par. n0010) taught by Chen, the teachings of “combining (i.e., the claimed “adding”) the utterance(s) spoken by the target user with additional noise (i.e., the claimed “selected noise”)” (Moreno, Par. 0027) taught by Moreno, and the teachings of “fine-tuning” (Liu, Par. n0084) taught by Liu in order to automatically “differentiate between speakers” (Khoury, Par. 0010), “modify the audio data to approximate different voice characteristics” (Gabrys, Par. 0020), enable “variational embeddings”/adaptable speaker embeddings (Battenberg, Par. 0007), overcome problems of “degradation of sound quality or expressiveness” (Kim, Par. 0002), and synthesize “expressive speech” (Finkelstein, Par. 0004), “provide high quality audio, e.g., speech,” (Garbacea, Par. 0007), overcome problems of “high requirements for noise, the speaker’s accent and fluency” (Chen, Par. n0003), “generate different output that is personalized” (Moreno, Par. 0003), and “closely mimics the target speaker's register” (Liu, Par. n0007).
Subject Matter to be Novel and Nonobvious
Claims 3 and 4 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding Claim 3, Applicant discloses speaker adaptation by speaker embeddings, global style token and prediction model based on “constructing the voice conversion model comprising the global style token mechanism, extracting the speaker embedding corresponding to a speaker ID through a speaker embedding table by using the constructed voice conversion model, predicting a variance of a Gaussian distribution for the extracted speaker embedding through the global style token mechanism, extracting a variance of each speaker by using the extracted speaker embedding in attention of the global style token mechanism as a query, and obtaining a Gaussian noise vector having the extracted variance by multiplying noise sampled from the Gaussian distribution by the extracted variance.”
Closest Prior Art
Li et al. (U.S. Patent Application Publication 2025/0349282), hereinafter referred to as Li, discloses speaker adapted speech synthesis based on “predicted embeddings aligned with speaker embeddings (Li, Par. 0086) using “global style tokens” (Li, Par. 0079).
Khoury et al., (U.S. Patent Application Publication 2021/0326421), hereinafter referred to as Khoury, discloses “similarity score based upon a distance between the inbound embedding (i.e., the claimed “new speaker embedding”) and a voiceprint stored in speaker profile in a speaker profile database (i.e., the claimed “plurality of generated speaker embeddings)” (Khoury, Par. 0013),
Chen (CN113763924A) discloses “variance of a multivariate Gaussian distribution” (Chen, Par. n0010) in order to overcome problems of “high requirements for noise, the speaker’s accent and fluency” (Chen, Par. n0003).
However, the prior art of record, fails to teach, alone or in a combination, among other things, “constructing the voice conversion model comprising the global style token mechanism, extracting the speaker embedding corresponding to a speaker ID through a speaker embedding table by using the constructed voice conversion model, predicting a variance of a Gaussian distribution for the extracted speaker embedding through the global style token mechanism, extracting a variance of each speaker by using the extracted speaker embedding in attention of the global style token mechanism as a query, and obtaining a Gaussian noise vector having the extracted variance by multiplying noise sampled from the Gaussian distribution by the extracted variance” as recited in claim 3.
Regarding Claim 4 disclose subject matter to be novel and nonobvious by virtue of dependency.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Semenov et al., (U.S. Patent Application Publication 2020/0402497) teaches speech synthesis using global style tokens and attention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EUNICE LEE whose telephone number is 571-272-1886. The examiner can normally be reached M-F 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EUNICE LEE/Examiner, Art Unit 2656
/BHAVESH M MEHTA/ Supervisory Patent Examiner, Art Unit 2656
Prosecution Insights