CTNF 18/954,928 CTNF 98840 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 12-151 AIA 26-51 12-51 Status of Claims This action is in reply to the claims filed on 21 November 2024. Claims 1-22 are currently pending and have been examined. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more. Independent claims 1 and 12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims regard a process that, as drafted under its broadest reasonable interpretation, covers performance of the limitations as a mental process and mathematics, but for the recitation of generic computer hardware (e.g., data processing hardware, a content encoder of a voice conversion model, and a conditioned decoder of a voice conversion model in claims 1 and 12, and a memory hardware in communication with the data processing hardware , the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations in claim 12). In regards to the processing of independent claims 1 and 12, the claimed functionality could be practiced as a mental process in the following manner: receiving a sequence of acoustic frames characterizing a source speech utterance comprising semantic information and source speech characteristics; (a human can receive data mentally or with pen and paper and can audibly hear acoustics) obtaining a latent speaker embedding representing target speech characteristics; (a human can translate speech into a vector with pen and paper using their mind and mathematics given data) generating, at each of a plurality of output steps, a soft speech representation for a corresponding acoustic frame from the sequence of acoustic frames; (a human can use their mind, mathematics, and a pen and paper to determine a representation of audio given data) determining, at each of the plurality of output steps, an acoustic estimation for the corresponding acoustic frame from the sequence of acoustic frames; (a human can use their mind and mathematics to turn data into an estimation of acoustics) and generating, at each of the plurality of output steps, a synthetic speech representation for the corresponding acoustic frame from the sequence of acoustic frames based on the soft speech representation and the acoustic estimation, the synthetic speech representation comprising the semantic information of the source speech utterance and the target speech characteristics of the latent speaker embedding (a human can mentally generate a mathematical representation of speech using pen and paper given data) This judicial exception is not integrated into a practical application. Outside of the identified abstract idea, the claimed invention only includes data processing hardware, a content encoder of a voice conversion model, and a conditioned decoder of a voice conversion model in claims 1 and 12, and a memory hardware in communication with the data processing hardware , the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations in claim 12, which amount to no more than mere instructions to implement an otherwise abstract idea using generic components. Note that the computing components here are being used for their ordinary purpose of executing a program to carry out a process (i.e., being used as a tool) instead of being improved as a tool. Independent claims 1 and 12 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the above additional element(s) merely use a computer as a tool to perform an abstract idea, which does not render a claim as being significantly more than the judicial exception. Therefore, claims 1 and 12 are not eligible subject matter under 35 USC 101. The remaining dependent claims fail to add patent eligible subject matter to their respective parent claims: Claims 2 and 13 regard making a mathematical determination based on data that can be mentally understood and evaluated by a human. Claims 3 and 14 recites a first encoder convolution layer, a stack of encoder blocks, and a second encoder convolution layer. This/these additional element(s) alone or in ordered combination do no more than generally link the use of a judicial exception to a particular technological environment or field of use (i.e., encoders) (see MPEP 2106.05(h)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 4 and 15 recite a first decoder convolution layer, a stack of decoder blocks, and a second decoder convolution layer. This/these additional element(s) alone or in ordered combination do no more than generally link the use of a judicial exception to a particular technological environment or field of use (i.e., decoders) (see MPEP 2106.05(h)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 5 and 16 recite one or more residual units, one or more respective FiLM layers, and a strided convolution layer. This/these additional element(s) alone or in ordered combination do no more than generally link the use of a judicial exception to a particular technological environment or field of use (i.e., decoders) (see MPEP 2106.05(h)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 6 and 17 regarding analyzing frames to create encoding to create embeddings in a way that can be mentally understood and evaluated by a human. Claims 6 and 17 further recites a speaker encoder. This/these additional element(s) alone or in ordered combination does no more than merely use a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 7 and 18 recites training a voice conversion model using a training process based on training data comprising a plurality of training source speech utterances each paired with a corresponding target speech utterance. This/these additional element(s) alone or in ordered combination does no more than merely use a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)), because the training is done at a high level, which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 8 and 19 regard mathematically determining loss using data in a way that can be mentally understood by a human. Claims 8 and 19 further recite using a Hidden-Unit BERT model and training the content encoder based on loss. This/these additional element(s) alone or in ordered combination do no more than generally link the use of a judicial exception to a particular technological environment or field of use (i.e., machine learning training techniques) (see MPEP 2106.05(h)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 9 and 20 regards making a classification of data to determine if it is authentic of synthetic in a way that can be mentally understood by a human. Claims 9 and 20 further recite using a multi-scale STFT discriminator and training the voice conversion model or multi-scale STFT discriminator based on loss. This/these additional element(s) alone or in ordered combination do no more than generally link the use of a judicial exception to a particular technological environment or field of use (i.e., machine learning training techniques) (see MPEP 2106.05(h)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 10 and 21 regard mathematically determining loss using data in a way that can be mentally understood by a human. Claims 10 and 21 further recite training the voice conversion model based on loss. This/these additional element(s) alone or in ordered combination do no more than generally link the use of a judicial exception to a particular technological environment or field of use (i.e., machine learning training techniques) (see MPEP 2106.05(h)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claims 11 and 22 regard mathematically determining loss using data in a way that can be mentally understood by a human. Claims 11 and 22 further recite training the voice conversion model based on loss. This/these additional element(s) alone or in ordered combination do no more than generally link the use of a judicial exception to a particular technological environment or field of use (i.e., machine learning training techniques) (see MPEP 2106.05(h)), which does not integrate the claim(s) into a practical application nor does it render a claim as being significantly more than the abstract idea. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claim (s) 1-2, 6-7, 12-13, and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 20250061908 A1) in view of Zhu (see attached NPL) in further view of Wu (US 20220165249 A1) . Regarding claim 1, Huang teaches a computer-implemented method that when executed on data processing hardware causes the data processing hardware to perform operations (see at least Paragraph [0202] “data processing device”) comprising: receiving a sequence of acoustic frames characterizing a source speech utterance comprising semantic information and source speech characteristics; (see at least Paragraph [0133] “In S301, source audio data and a tone feature of a target speaker are acquired.”; Paragraph [0138] “the user inputs a synthesis request on an intelligent device, such that the intelligent device is controlled by the synthesis request to synthesize audio data of a certain speaking content from the target speaker. […] the synthesis request is input by inputting voice information”; Paragraph [0139] “The source audio data refers to audio data configured to provide semantic information”; Paragraph [0111] “audio frames contained in each piece of sample audio data”; Paragraph [0111] explains that audio data exists in frames; step 301 of Fig. 3) obtaining a latent speaker embedding representing target speech characteristics; (see at least Paragraph [0145] “the electronic device performing the tone conversion determines a tone feature of the target speaker based on the information of the target speaker.”; Paragraph [0095] explains that the tone feature can be represented as a tone vector; step 301 of Fig. 3) generating, at each of a plurality of output steps, using a content encoder [posterior encoder of Fig. 4] of a voice conversion model, a soft speech representation for a corresponding acoustic frame from the sequence of acoustic frames; (Paragraph [0165] “By a posteriori encoder included in the tone network in the tone conversion model, such as the posterior encoder shown in FIG. 4, a hidden vector (such as z.sub.sq) of semantic information in the source audio data is acquired based on the tone feature corresponding to the source audio data and a linear spectrogram (such as the linear spectrogram shown in FIG. 4) corresponding to the source audio data.”; Fig. 4) determining, at each of the plurality of output steps, an acoustic estimation for the corresponding acoustic frame from the sequence of acoustic frames; and generating, at each of the plurality of output steps, using a decoder of the voice conversion model, a synthetic speech representation for the corresponding acoustic frame from the sequence of acoustic frames based on the soft speech representation generated by the content encoder and the acoustic estimation , the synthetic speech representation comprising the semantic information of the source speech utterance and the target speech characteristics of the latent speaker embedding, wherein the decoder is conditioned on the latent speaker embedding. (Paragraph [0167] “The synthesized audio data is acquired by the vocoder (decoder) shown in FIG. 4 based on that semantic feature and the tone feature of the target speaker (such as the speaker inner embedding shown in FIG. 4). That is, the original waveform shown in FIG. 4 is acquired.”; Fig. 4) Huang does not teach: generating, at each of a plurality of output steps, using a content encoder of a voice conversion model, a soft speech representation for a corresponding acoustic frame from the sequence of acoustic frames; determining, at each of the plurality of output steps, an acoustic estimation for the corresponding acoustic frame from the sequence of acoustic frames; and generating, at each of the plurality of output steps, using a decoder of the voice conversion model, a synthetic speech representation for the corresponding acoustic frame from the sequence of acoustic frames based on the soft speech representation generated by the content encoder and the acoustic estimation , the synthetic speech representation comprising the semantic information of the source speech utterance and the target speech characteristics of the latent speaker embedding, wherein the decoder is conditioned on the latent speaker embedding. However, Zhu teaches: a soft speech representation for a corresponding acoustic frame from the sequence of acoustic frames ; (see at least Page 3 “the MFCC parameters of each frame of normal speech and whispered speech were extracted, and then the reference normal speech and whispered speech were processed with dynamic time warping (DTW). The normal speech-whispered speech joint MFCC feature distribution model was then constructed using GMM.” Of Zhu) a synthetic speech representation for the corresponding acoustic frame from the sequence of acoustic frames based on the soft speech representation. (see at least Page 3 “the MFCC features of whispered speech were converted to the MFCC features of normal speech using the established GMM from the first stage, and the normal speech was then synthesized using the MFCC features of normal speech.” of Zhu) This step of Zhu is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to synthesizing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the method of Huang to incorporate the soft speech representation as taught by Zhu. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to convert whispered speech into normal speech (see Page 1: Introduction of Zhu). Further, Zhu is directly applicable is directed Huang given that Huang’s acoustic features are detailed using cestrum coefficients [0070], and Zhu is directed to MFCC feature parameters of normal speech. Inclusion of Zhu therefor would also provide the beneficial features found in Zhu, namely distortion reduction and achieving improvements in intelligibility and sound quality (see abstract and conclusion of Zhu). Huang in view of Zhu does not teach: determining, at each of the plurality of output steps, an acoustic estimation for the corresponding acoustic frame from the sequence of acoustic frames. a speech representation for the corresponding acoustic frame from the sequence of acoustic frames based on the acoustic estimation. However, Wu teaches: determining, at each of the plurality of output steps, an acoustic estimation for the corresponding acoustic frame from the sequence of acoustic frames. (see at least Paragraph [0070] “In step S302, the speech samples corresponding to each training text are divided into different frames according to a preset frequency, and the acoustic characteristic parameters are extracted for each frame” of Wu) a synthetic speech representation for the corresponding acoustic frame from the sequence of acoustic frames based on the acoustic estimation. (see at least Paragraph [0061] “After the acoustic characteristic parameters of the input phoneme sequence are obtained by prediction, the acoustic characteristic parameters (for example, Mel spectrum parameters) are input into the vocoder parameter conversion model to be converted into the vocoder characteristic parameter, and then speech synthesis may be performed by the vocoder.” of Wu) This step of Wu is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to synthesizing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the method of Huang to incorporate determining and using an acoustic estimation as taught by Wu. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to allow a more accurate, smooth and natural synthesized speech (see paragraph [0029] of Wu). Regarding claim 2, Huang in view of Zhu in further view of Wu teaches the computer-implemented method of claim 1. Huang does not teach: wherein the soft speech representation comprises a probability distribution of discrete speech units. However, Zhu teaches: wherein the soft speech representation comprises a probability distribution of discrete speech units. (see at least Page 3 “the MFCC parameters of each frame of normal speech and whispered speech were extracted, and then the reference normal speech and whispered speech were processed with dynamic time warping (DTW). The normal speech-whispered speech joint MFCC feature distribution model was then constructed using GMM.” Of Zhu) The motivation for making this modification to the teachings of Huang is the same as that set forth above, in the rejection of claim 1. Regarding claim 6, Huang in view of Zhu in further view of Wu teaches the computer-implemented method of claim 1. Huang further teaches wherein the operations further comprise: receiving a sequence of acoustic frame characterizing a target speech utterance comprising target speech characteristics; (see at least Paragraph [0133] “In S301, source audio data and a tone feature of a target speaker are acquired.”; Paragraph [0111] “audio frames contained in each piece of sample audio data”; Paragraph [0111] explains that audio data exists in frames; step 301 of Fig. 3) for each respective acoustic frame from the sequence of acoustic frames, generating, using a speaker encoder, a corresponding speaker encoding for the respective acoustic frame; and aggregating the speaker encodings generated for the sequence of acoustic frames to generate the latent speaker embedding. (see at least Paragraph [0145] “the electronic device performing the tone conversion determines a tone feature of the target speaker based on the information of the target speaker.”; Paragraph [0095] explains that the tone feature can be represented as a tone vector; step 301 of Fig. 3) Huang does not teach: for each respective acoustic frame from the sequence of acoustic frames, generating, using a speaker encoder, a corresponding speaker encoding for the respective acoustic frame; and aggregating the speaker encodings generated for the sequence of acoustic frames to generate the latent speaker embedding. However, Zhu teaches: for each respective acoustic frame from the sequence of acoustic frames, generating, using a speaker encoder, a corresponding speaker encoding for the respective acoustic frame; (see at least Page 4 “the MFCC parameters of each frame of normal speech and whispered speech were extracted”; Page 6 “The frame segmentation was performed on the signals, with a frame length of 512 sampling points” of Zhu) and aggregating the speaker encodings generated for the sequence of acoustic frames to generate the latent speaker embedding. (see at least Page 6 “three frames of whispered speech and one frame of normal speech constituted a joint feature vector.” Of Zhu) This step of Zhu is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to synthesizing speech with audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the method of Huang to incorporate encoding the audio frame by frame to generate the latent speaker embedding as taught by Zhu. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to achieve better conversion effectiveness (see Page 6 of Zhu). Regarding claim 7, Huang in view of Zhu in further view of Wu teaches the computer-implemented method of claim 1. Huang further teaches: wherein the voice conversion model is trained by a training process based on training data comprising a plurality of training source speech utterances each paired with a corresponding target speech utterance. (see at least Paragraph [0042] “In S101, a sample set is acquired, wherein the sample set contains sample audio data of different speakers, each piece of the sample audio data corresponds to a piece of target audio data”; Fig. 1 showing the training process of Huang) Claim 12: Claim(s) 12 is/are directed to a system. Claim(s) 12 recite limitations parallel in nature as those addressed above for claim(s) 1, which are directed towards a method. Claim(s) 12 is/are therefore rejected for the same reasons as set above for claim(s) 1, respectively. Claim 12 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) Claim 13: Claim(s) 13 is/are directed to a system. Claim(s) 13 recite limitations parallel in nature as those addressed above for claim(s) 2, which are directed towards a method. Claim(s) 13 is/are therefore rejected for the same reasons as set above for claim(s) 2, respectively. Claim 13 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) Claim 17: Claim(s) 17 is/are directed to a system. Claim(s) 17 recite limitations parallel in nature as those addressed above for claim(s) 6, which are directed towards a method. Claim(s) 17 is/are therefore rejected for the same reasons as set above for claim(s) 6, respectively. Claim 17 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) Claim 18: Claim(s) 18 is/are directed to a system. Claim(s) 18 recite limitations parallel in nature as those addressed above for claim(s) 7, which are directed towards a method. Claim(s) 18 is/are therefore rejected for the same reasons as set above for claim(s) 7, respectively. Claim 18 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) 07-21-aia AIA Claim (s) 3 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 20250061908 A1) in view of Zhu (see attached NPL) in further view of Wu (US 20220165249 A1) in further view of Sun (US 20230401429 A1) . Regarding claim 3, Huang in view of Zhu in further view of Wu teaches the computer-implemented method of claim 1.Huang in view of Zhu in further view of Wu does not teach: wherein the content encoder comprises: a first encoder convolution layer; a stack of encoder blocks; and a second encoder convolution layer. However, Sun teaches: wherein the content encoder comprises: a first encoder convolution layer; (Paragraph [0050] “each encoding layer 11, 12, 13, of the repeated application of convolutions”; Fig. 1 of Sun) a stack of encoder blocks; (Paragraph [0050] “The U-net architecture builds upon a stack of convolutional layers organized in a contracting path including encoding layers 11, 12, 13”) and a second encoder convolution layer. (Paragraph [0050] “each encoding layer 11, 12, 13, of the repeated application of convolutions”; Fig. 1 of Sun) This step of Sun is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the method of Huang to incorporate the structure of the content encoder as taught by Sun. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to handle various audio processing problems including vocal separation, speech enhancement and speech source separation (see paragraph [0007] of Sun). Claim 14: Claim(s) 14 is/are directed to a system. Claim(s) 14 recite limitations parallel in nature as those addressed above for claim(s) 3, which are directed towards a method. Claim(s) 14 is/are therefore rejected for the same reasons as set above for claim(s) 3, respectively. Claim 14 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) 07-21-aia AIA Claim (s) 4-5 and 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 20250061908 A1) in view of Zhu (see attached NPL) in further view of Wu (US 20220165249 A1) in further view of Liu (see attached NPL) . Regarding claim 4, Huang in view of Zhu in further view of Wu teaches the computer-implemented method of claim 1. Huang further teaches: wherein the decoder (vocoder/decoder of Huang; Fig. 2 of Huang) comprises: a first decoder convolution layer; (Paragraph [0080] “The vocoder is a high efficiency (HiFiGAN) vocoder”; Examiner notes that one of ordinary skill in the art knows HiFiGAN vocoders use multiple convolution layers.) a stack of decoder blocks; and a second decoder convolution layer. (Paragraph [0080] “The vocoder is a high efficiency (HiFiGAN) vocoder”; Examiner notes that one of ordinary skill in the art knows HiFiGAN vocoders use multiple convolution layers.) Huang in view of Zhu in further view of Wu does not teach: wherein the decoder comprises: a stack of decoder blocks. However, Liu teaches: wherein the decoder comprises: a stack of decoder blocks. (Page 3 “Fig. 3, […] up-sample blocks and downsample blocks”; Fig. 3) This step of Liu is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to voice conversion. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the decoder of Huang to incorporate a stack of decoder blocks as taught by Liu. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to improve inference speed (see Page 1: Abstract of Liu). Regarding claim 5, Huang in view of Zhu in further view of Wu in further view of Liu teaches the computer-implemented method of claim 4. Huang does not teach: wherein each decoder block comprises: one or more residual units; one or more respective Feature-wise Linear Modulation (FiLM) layers; and a strided convolution layer. However, Liu teaches: wherein each decoder block (Page 3 “Fig. 3, […] up-sample blocks and downsample blocks”; Fig. 3) comprises: one or more residual units; (Page 3 “up-sample blocks and downsample blocks […] The intuitions behind this using two U-shape branches is to fully fuse information from the sine-excitation signals and loudness features into the waveform generation process in different time scales.”; Fig. 3; Examiner notes the U-shape structure is in line with the explanation in Paragraph [0028] of the instant specification “residual units using dilated convolutions followed by a down-sampling layer”) one or more respective Feature-wise Linear Modulation (FiLM) layers; (See at least Page 2 “A feature-wise linear modulation (FiLM) based generator is used to synthesize waveform directly from linguistic features”) and a strided convolution layer. (Page 2 “The down-sample layer adopts a strided 2D convolutional structure to down-sample the input mel spectrograms”) The motivation for making this modification to the teachings of Huang is the same as that set forth above, in the rejection of claim 4. Claim 15: Claim(s) 15 is/are directed to a system. Claim(s) 15 recite limitations parallel in nature as those addressed above for claim(s) 4, which are directed towards a method. Claim(s) 15 is/are therefore rejected for the same reasons as set above for claim(s) 4, respectively. Claim 15 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) Claim 16: Claim(s) 16 is/are directed to a system. Claim(s) 16 recite limitations parallel in nature as those addressed above for claim(s) 5, which are directed towards a method. Claim(s) 16 is/are therefore rejected for the same reasons as set above for claim(s) 5, respectively. Claim 16 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) 07-21-aia AIA Claim (s) 8 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 20250061908 A1) in view of Zhu (see attached NPL) in further view of Wu (US 20220165249 A1) in further view of Qian (US 20240170007 A1) . Regarding claim 8, Huang in view of Zhu in further view of Wu teaches the computer-implemented method of claim 7. Huang further teaches: wherein, for each respective training source speech utterance, the training process trains the voice conversion model (see at least paragraph [0041] “FIG. 1 is a schematic diagram of a training process of a tone conversion model”; paragraph [0042] “In S101, a sample set is acquired, wherein the sample set contains sample audio data of different speakers, each piece of the sample audio data corresponds to a piece of target audio data”; Fig. 1) by: predicting, using the content encoder, a soft speech representation for the respective training source speech utterance; (see at least Paragraph [0064] “any sample audio data in the sample set is acquired, and the sample audio data is input into the original tone conversion model. By the original tone conversion model, based on the sample audio data and the tone feature (denoted as the second tone feature) of the target audio data corresponding to the sample audio data, the synthesized audio data corresponding to the sample audio data is acquired.”; Fig. 1) generating , using a Hidden-Unit BERT model , a target soft speech representation for the respective training source speech utterance; (see at least Paragraph [0060] “for each piece of sample audio data in the sample set, target audio data of a different speaker with the same semantic information as the sample audio data is determined.”; Fig. 1) determining a cross-entropy loss based on the predicted soft speech representation and the target soft speech representation; (see at least Paragraph [0085] “based on the target audio data corresponding to each piece of sample audio data and the corresponding synthesized audio data, a reconstruction loss value is determined”; Fig. 1) and training the content encoder based on the cross-entropy loss. (see at least Paragraph [0085] “based on the reconstruction loss value, the value of the parameter in the original tone conversion model is adjusted, and thus the trained tone conversion model is acquired.”; Fig. 1) Huang does not teach: predicting, using the content encoder, a soft speech representation for the respective training source speech utterance; generating, using a Hidden-Unit BERT model , a target soft speech representation for the respective training source speech utterance; and determining a cross-entropy loss based on the predicted soft speech representation and the target soft speech representation; and training the content encoder based on the cross-entropy loss. However, Zhu teaches the known technique of the representation being a soft speech representation (see at least Page 3 “the MFCC parameters of each frame of normal speech and whispered speech were extracted, and then the reference normal speech and whispered speech were processed with dynamic time warping (DTW). The normal speech-whispered speech joint MFCC feature distribution model was then constructed using GMM.” Of Zhu) This step of Zhu is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to synthesizing speech. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the representation of Huang to be a soft speech representation as taught by Zhu. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to convert whispered speech into normal speech (see Page 1: Introduction of Zhu). Huang in view of Zhu in further view of Wu does not teach: generating, using a Hidden-Unit BERT model , a target soft speech representation for the respective training source speech utterance; and determining a cross-entropy loss based on the predicted soft speech representation and the target soft speech representation; and training the content encoder based on the cross-entropy loss. However Qian teaches: generating, using a Hidden-Unit BERT model , a target speech representation for the respective training source speech utterance; (see at least Paragraph [0056] “the converted utterances may be passed through a pre-trained unsupervised speech representation network, (in this scenario HUBERT), to generate a set of speech representations”; Fig. 4 of Qian) and determining a cross-entropy loss based on the predicted speech representation and the target speech representation; (see at least Paragraph [0036] “a cross-entropy loss between the generated label sequence and the predicted label sequence.”; Fig. 2 of Qian) training the content encoder based on the cross-entropy loss. (see at least Paragraph [0057] “contrastive loss may be introduced to penalize dissimilarity between R.sup.(1) and R.sup.(2)”; Paragraph [0054] “cross-entropy loss”; Fig. 4 of Qian) This step of Qian is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to determining speech representations from audio. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the generation of the target speech representation of Huang to use HuBERT and for the loss of Huang to be cross-entropy loss as taught by Qian. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to provide self-supervised speech representations (see paragraph [0002] of Qian). Claim 19: Claim(s) 19 is/are directed to a system. Claim(s) 19 recite limitations parallel in nature as those addressed above for claim(s) 8, which are directed towards a method. Claim(s) 19 is/are therefore rejected for the same reasons as set above for claim(s) 8, respectively. Claim 19 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) 07-21-aia AIA Claim (s) 9-11 and 20-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 20250061908 A1) in view of Zhu (see attached NPL) in further view of Wu (US 20220165249 A1) in further view of Qian (US 20240170007 A1) in further view of Yu (US 20260141908 A1) . Regarding claim 9, Huang in view of Zhu in further view of Wu in further view of Qian teaches the computer-implemented method of claim 8. Huang further teaches: wherein the training process further trains the voice conversion model (Fig. 2) or a multi-scale Short-Time Fourier Transform (STFT) discriminator (Paragraph [0115] “train a multi-scale discriminator”) by: generating, using the decoder, a synthetic speech representation for the predicted soft speech representation; (see at least Paragraph [0095] “by the decoder shown in FIG. 2, the synthesized audio data (raw waveform) is acquired based on the first semantic feature and the second tone feature (speaker inner embedding) of the target audio data corresponding to the sample audio data.”; Fig. 2) receiving, as input to the multi-scale STFT discriminator, a respective one of the synthetic speech representation generated by the decoder or the respective training source speech utterance; (Paragraph [0115] “The HiFiGAN vocoder includes […] a multi-scale discriminator”) determining, using the multi-scale STFT discriminator, a classification for the received respective one of the synthetic speech representation generated by the decoder or the respective training target speech utterance, the classification comprising a synthetic speech classification or a non-synthetic speech classification; (see at least Paragraph [0118] “it is expected that the generator in the HiFiGAN vocoder is capable of synthesizing synthesized audio data that is close to the target audio data, such that the discriminator in the HiFiGAN vocoder fails to determine whether the audio data is the target audio data or the synthesized audio data.”) determining an adversarial loss based on the classification; (see at least Paragraph [0118] “the discriminator in the HiFiGAN vocoder fails to determine whether the audio data is the target audio data or the synthesized audio data. Based on this, the loss value of the HiFiGAN vocoder further includes a generative adversarial loss value.”) and training the voice conversion model or the multi-scale STFT discriminator based on the adversarial loss. (see at least Paragraph [0121] “Upon acquisition of the loss value of the vocoder based on the above embodiments, the value of each parameter in the original tone conversion model is adjusted based on the loss value of the vocoder”) Huang does not teach: generating, using the decoder, a synthetic speech representation for the predicted soft speech representation; However, Zhu teaches: generating, using the decoder, a synthetic speech representation for the predicted soft speech representation; (see at least Page 3 “the MFCC features of whispered speech were converted to the MFCC features of normal speech using the established GMM from the first stage, and the normal speech was then synthesized using the MFCC features of normal speech.” of Zhu) The motivation for making this modification to the teachings of Huang is the same as that set forth above, in the rejection of claim 1. Huang in view of Zhu in further view of Wu in further view of Qian does not teach: the multi-scale discriminator being a the multi-scale STFT discriminator. However, Yu teaches: the multi-scale discriminator being a the multi-scale STFT discriminator. (Paragraph [0224] “a multi-scale STFT discriminator is used” of Yu) This step of Yu is applicable to the method of Huang as they both share characteristics and capabilities, namely, they are directed to voice signal decoding. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have modified the multi-scale discriminator of Huang to make it a multi-scale STFT discriminator as taught by Yu. One of ordinary skill in the art before the effective filling date of the claimed invention would have been motivated to modify Huang in order to train a voice coding system (see paragraph [0224] of Yu). Regarding claim 10, Huang in view of Zhu in view of Wu in further view of Qian in further view of Yu teaches the computer-implemented method of claim 9. Huang further teaches: wherein the training process further trains the voice conversion model by: determining a feature loss based on an output of the multi-scale STFT discriminator and the corresponding target speech utterance; (see at least Paragraph [0115] “The HiFiGAN vocoder […] a multi-scale discriminator […] The HiFiGAN vocoder uses a feature matching loss as an additional loss for training the generator, and stabilizes the GAN by extracting each intermediate feature of the discriminator and calculating a distance L1 between the target audio data and the synthesized audio data in each feature space.”) and training the voice conversion model based on the feature loss. (see at least Paragraph [0121] “Upon acquisition of the loss value of the vocoder based on the above embodiments, the value of each parameter in the original tone conversion model is adjusted based on the loss value of the vocoder”) Huang in view of Zhu in further view of Wu in further view of Qian does not teach: the multi-scale discriminator being a the multi-scale STFT discriminator. However, Yu teaches: the multi-scale discriminator being a the multi-scale STFT discriminator. (Paragraph [0224] “a multi-scale STFT discriminator is used” of Yu) The motivation for making this modification to the teachings of Huang is the same as that set forth above, in the rejection of claim 9. Regarding claim 11, Huang in view of Zhu in further view of Wu in further view of Qian in further view of Yu teaches the computer-implemented method of claim 9. Huang further teaches: wherein the training process further trains the voice conversion model by: determining a reconstruction loss based on the synthetic speech representation generated by the decoder and the corresponding target speech utterance; (see at least Paragraph [0122] “based on the target audio data and synthesized audio data corresponding to each piece of sample audio data, the reconstruction loss value is determined”; Fig. 2 shows the synthetic speech being generated by the decoder.) and training the voice conversion model based on the reconstruction loss. (see at least Paragraph [0122] “Based on the reconstruction loss value and the loss value of the vocoder, the composite loss value is determined. Based on the composite loss value, the value of each parameter in the original tone conversion model is adjusted.”) Claim 20: Claim(s) 20 is/are directed to a system. Claim(s) 20 recite limitations parallel in nature as those addressed above for claim(s) 9, which are directed towards a method. Claim(s) 20 is/are therefore rejected for the same reasons as set above for claim(s) 9, respectively. Claim 20 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) Claim 21: Claim(s) 21 is/are directed to a system. Claim(s) 21 recite limitations parallel in nature as those addressed above for claim(s) 10, which are directed towards a method. Claim(s) 21 is/are therefore rejected for the same reasons as set above for claim(s) 10, respectively. Claim 21 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) Claim 22: Claim(s) 22 is/are directed to a system. Claim(s) 22 recite limitations parallel in nature as those addressed above for claim(s) 11, which are directed towards a method. Claim(s) 22 is/are therefore rejected for the same reasons as set above for claim(s) 11, respectively. Claim 22 further recites memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations (see at least Paragraph [0202] “a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a particular manner” of Huang) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIELLE ELIZABETH ZEVITZ whose telephone number is (703)756-1070. The examiner can normally be reached Mo-Th 10am-6pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lynda Jasmin can be reached at (571) 272-6782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /DANIELLE ELIZABETH ZEVITZ/Examiner, Art Unit 3629 /ANDREW C FLANDERS/Supervisory Patent Examiner, Art Unit 2655 Application/Control Number: 18/954,928 Page 2 Art Unit: 3629 Application/Control Number: 18/954,928 Page 3 Art Unit: 3629 Application/Control Number: 18/954,928 Page 4 Art Unit: 3629 Application/Control Number: 18/954,928 Page 5 Art Unit: 3629 Application/Control Number: 18/954,928 Page 6 Art Unit: 3629 Application/Control Number: 18/954,928 Page 7 Art Unit: 3629 Application/Control Number: 18/954,928 Page 8 Art Unit: 3629 Application/Control Number: 18/954,928 Page 9 Art Unit: 3629 Application/Control Number: 18/954,928 Page 10 Art Unit: 3629 Application/Control Number: 18/954,928 Page 11 Art Unit: 3629 Application/Control Number: 18/954,928 Page 12 Art Unit: 3629 Application/Control Number: 18/954,928 Page 13 Art Unit: 3629 Application/Control Number: 18/954,928 Page 14 Art Unit: 3629 Application/Control Number: 18/954,928 Page 15 Art Unit: 3629 Application/Control Number: 18/954,928 Page 16 Art Unit: 3629 Application/Control Number: 18/954,928 Page 17 Art Unit: 3629 Application/Control Number: 18/954,928 Page 18 Art Unit: 3629 Application/Control Number: 18/954,928 Page 19 Art Unit: 3629 Application/Control Number: 18/954,928 Page 20 Art Unit: 3629 Application/Control Number: 18/954,928 Page 21 Art Unit: 3629 Application/Control Number: 18/954,928 Page 22 Art Unit: 3629 Application/Control Number: 18/954,928 Page 23 Art Unit: 3629 Application/Control Number: 18/954,928 Page 24 Art Unit: 3629 Application/Control Number: 18/954,928 Page 25 Art Unit: 3629 Application/Control Number: 18/954,928 Page 26 Art Unit: 3629 Application/Control Number: 18/954,928 Page 27 Art Unit: 3629 Application/Control Number: 18/954,928 Page 28 Art Unit: 3629 Application/Control Number: 18/954,928 Page 29 Art Unit: 3629 Application/Control Number: 18/954,928 Page 30 Art Unit: 3629 Application/Control Number: 18/954,928 Page 31 Art Unit: 3629 Application/Control Number: 18/954,928 Page 32 Art Unit: 3629 Application/Control Number: 18/954,928 Page 33 Art Unit: 3629
Prosecution Insights