DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
All objections/rejections not mentioned in this Office Action have been withdrawn by the Examiner.
Status of the Claims
Prior to entry of the amendment(s) and/or consideration of the argument(s), the status of the claims is as follows.
Claim(s) 1-20 is/are pending.
Claims 13, 15, and 18 are objected to because of the following informalities.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 5 and 13 of U.S. Pat. No. 12,100,383 in view of combinations of Zhang (CN 113436608 A, hereinafter Zhang), Aggarwal (U.S. Pat. No. 11017763, hereinafter Aggarwal), Minnen (U.S. Pat. App. Pub. No. 2020/0027247, hereinafter Minnen), and Golman (U.S. Pat. App. Pub. No. 2023/0223011, hereinafter Golman).
Claim(s) 1 and 11 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang.
Claims 2 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang as applied to claim(s) 1 and 11 above, and further in view of Aggarwal.
Claims 3-9 and 13-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang as applied to claim(s) 1 and 11 above, and further in view of Minnen.
Claims 10 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang as applied to claim 1 and 11 above, and further in view of Golman.
Response to Amendments
Applicant’s amendment filed on 03 August 2026 has been entered.
Applicant has submitted a terminal disclaimer in compliance with 37 C.F.R. 1.321 in response to the obviousness-type double patenting rejection of claims 1-20. Therefore, the rejection is withdrawn.
In view of the amendment to the claim(s), the amendment of claim(s) 1, 3, 7-8, 11, 13, 15, 17, 19; the cancellation of claim(s) 8 and 18; and the addition of claim(s) 21 and 22 have been acknowledged and entered.
After entry of the amendments, claims 1-7, 9-17, and 19-22 remain pending.
In view of the amendment to claim(s) 13, 15, and 18, the objection to claim(s) 13, 15, and 18 is withdrawn.
In view of the amendment to claim(s) 1 and 11, the rejection of claims 1 and 11 under 35 U.S.C. §102 is withdrawn.
In light of the amended and newly added claims, new grounds for rejection under 35 U.S.C. §103 are provided in the action below.
Response to Arguments
Applicant’s arguments regarding the prior art rejections under 35 U.S.C. §102/103, see pages 7-9 of the Response to Non-Final Office Action dated 14 May 2026, which was received on 03 August 2026 (hereinafter Response and Office Action, respectively), have been fully considered.
With respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. §102 as being anticipated by Zhang, applicant asserts that Zhang fails to teach or suggest all limitations of the claims as amended. Applicant’s arguments with regards to the rejection as previously presented are persuasive. As such, the rejections of claims 1 and 11 under 35 U.S.C. §102/ 35 U.S.C. §103 are withdrawn.
Claims 8 and 18 are cancelled in this response. Therefore, the rejection of claims 8 and 18 are withdrawn.
Applicant further argues that the rejection(s) of dependent claims 2-7, 9-10, 12-17, and 19-20 should be withdrawn for at least the same reasons as independent claims 1 and 11. Applicant’s arguments in light of the amended claims are persuasive. As such, the rejections of claims 2-7, 9-10, 12-17, and 19-20 under 35 U.S.C. §102/ 35 U.S.C. §103 are withdrawn.
However, upon further consideration, new ground(s) of rejection under 35 U.S.C. §103 are made in light of combinations of Zhang, Aggarwal, Minnen, and Golman, and newly cited reference Non-Patent Literature to Prenger (Prenger, R., Valle, R. and Catanzaro, B., 2019, May. Waveglow: A flow-based generative network for speech synthesis. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 3617-3621). IEEE.(2019), hereinafter Prenger).
The Applicant has not provided any further statement and therefore, the Examiner directs the Applicant to the below rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-7, 9, 11, 13-17, 19 and 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Minnen, with further evidence from Prenger.
Regarding claim 1, Zhang discloses A computer-implemented method comprising (Discloses systems and methods described with reference to a “dual-stream speech conversion device, comprising: a memory and at least a processor, in which instructions are stored; At least one processor calls the instruction in the memory to enable the dual-stream speech conversion device to execute the dual-stream speech conversion method described above”; Zhang, ¶ [0059]): receiving first audio data representing first speech (The system receives the “voice signal includes the time domain signal of the voice data and the frequency domain signal of the voice data” and where “The discrete Fourier transform and logarithmic spectral conversion are performed on the time-domain signals in the preprocessed speech signals to obtain the first spectral information {first audio data}.”; Zhang, ¶ [0072]); receiving first data representing first voice characteristics of the first speech (Discloses “extracting the features of the speech signal to obtain the... target voiceprint features, target emotional features {...representing the first voice characteristics}” where the above described target “features of the speech signal” are the first data.; Zhang, ¶ [0091]); processing the first audio data and the first data to generate second data representing the first speech (“The server calls the vector compression layer in the vector processing layer to divide the voice signal features into the product of the low-dimensional subspace to obtain the voice signal segmentation vector {processing the first audio data}, and the target audio feature into the product of the low-dimensional subspace to obtain the audio segmentation vector {processing the first data},” and “The audio compression vector {second data} is obtained by mapping the voice signal segmentation vector and the audio segmentation vector into the preset set.”; Zhang, ¶ [0092]); receiving third data representing second voice characteristics (The system relies on voice parameters (also referred to as “preset speech features” or “the value of the target learning variable”) which is used to “indicate the weight ratio for adaptive addition,” where “two-way mapping learning based on the preset speech features is carried out to obtain the candidate dual-flow speech synthesis model” and “the preset speech features are used to indicate the converted speech,” thereby defining the new voice the audio will be converted into.; Zhang, ¶ [0081], [0087]-[0088]), different from the first voice characteristics (Discloses using “the preset speech features” to “obtain the converted voice feature vector,” as part of a voice conversion process. As such, the “preset speech features{second voice characteristics}” are understood to be different from the “target…features {first voice characteristics}”; Zhang, ¶ [0001]-[0002], [0077], [0088], [0091]); processing the decompressed data and the third data to generate second audio data (The system uses the target features and variables within “the affine coupling layer” and “the dual-flow affine transform” based on the “preset speech features” is performed on “the treated vector to obtain the converted voice feature vector,”; Zhang, ¶ [0077]); and generating, using the second audio data, audio representing synthesized speech corresponding to the second voice characteristics (“Obtain the value of the target learning variable, and use the normalization layer to weighted normalize the converted voice feature vector based on the value of the target learning variable to obtain the target speech.”; Zhang, ¶ [0080]). However, Zhang fails to expressly recite compressing the second data to determine compressed data; decompressing the compressed data to determine decompressed data representing the first speech.
Minnen teaches systems and methods for compression and decompression of data and data representations. (Minnen, ¶ [0059]-[0061]). Regarding claim 1, Minnen teaches compressing the second data to determine compressed data (Discloses “compression of the latent representation 116 of the input data” which is noted as being structurally identical to Zhang’s audio compression vector, “using entropy encoding techniques, the compression system 100 quantizes the latent representation 116 of the input data using a quantizer Q 118 to generate an ordered collection of code symbols 120 (ŷ).” in “accordance with the conditional entropy model.”; Minnen, ¶ [0066], [0076]-[0077]); decompressing the compressed data to determine decompressed data representing the first speech (“The entropy decoding engine 208 is configured to entropy decode the compressed code symbols 134 included in the compressed data 104 in accordance with the conditional entropy model to recover the code symbols 120.”; Minnen, ¶ [0082]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the flow-based voice conversion systems of Zhang to incorporate the teachings of Minnen to include compressing the second data to determine compressed data; decompressing the compressed data to determine decompressed data representing the first speech. Zhang teaches systems and methods for flow-based voice conversion, which mathematically fuse target voice print features and phoneme features into a single latent vector. However, though Zhang considers remote implementation at a server, Zhang fails to expressly contemplate the details of the distribution of this process to devices and/or between devices across a network. In distributed speech processing, it is well known in the art that transmitting raw audio requires significant bandwidth, and transmitting uncompressed, high dimensional neural network vectors introduces unacceptable latency. Minnen specifically addresses the technical problem of transmitting high dimensional neural network data across networks, including a specialized architecture designed to generate a compressed representation from a latent representation, and transmit that compressed representation to a second device for further use and processing of the underlying data. A PHOSITA would be motivated to modify the flow-based voice conversion process of Zhang with the transmission architecture of Minnen to further enable the server embodiments described in Zhang, while avoiding known latency difficulties and increasing transmission efficiency across a distributed network, as recognized by Minnen. (Minnen, ¶ [0046]-[0048]).
Regarding claim 3, the rejection of claim 1 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. However, Zhang fails to expressly recite further comprising: causing the second data to be sent from a first device to a second device.
The relevance of Minnen is described above with relation to claim 1. Regarding claim 3, Minnen teaches further comprising: causing the second data to be sent from a first device to a second device (“the compression and decompression systems may be co-located or remotely located, and compressed data generated by the compression system can be provided to the decompression system in any of a variety of ways. For example, the compressed data may be stored (e.g., in a physical data storage device or logical data storage area), and then subsequently retrieved from storage and provided to the decompression system. As another example, the compressed data may be transmitted over a communications network (e.g., the Internet) to a destination, where it is subsequently retrieved and provided to the decompression system.”; Minnen, ¶ [0060]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the flow-based voice conversion systems of Zhang to incorporate the teachings of Minnen to include further comprising: causing the second data to be sent from a first device to a second device. Zhang teaches systems and methods for flow-based voice conversion, which mathematically fuse target voice print features and phoneme features into a single latent vector. However, though Zhang considers remote implementation at a server, Zhang fails to expressly contemplate the details of the distribution of this process to devices and/or between devices across a network. In distributed speech processing, it is well known in the art that transmitting raw audio requires significant bandwidth, and transmitting uncompressed, high dimensional neural network vectors introduces unacceptable latency. Minnen specifically addresses the technical problem of transmitting high dimensional neural network data across networks, including a specialized architecture designed to generate a compressed representation from a latent representation, and transmit that compressed representation to a second device for further use and processing of the underlying data. A PHOSITA would be motivated to modify the flow-based voice conversion process of Zhang with the transmission architecture of Minnen to further enable the server embodiments described in Zhang, while avoiding known latency difficulties and increasing transmission efficiency across a distributed network, as recognized by Minnen. (Minnen, ¶ [0046]-[0048]).
Regarding claim 4, the rejection of claim 3 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. Zhang further discloses wherein the first data is not sent to the second device (Zhang in light of Minnen teaches the transmission of the compressed data of Zhang to a second device. Zhang teaches “extracting the features of the speech signal to obtain the… target voiceprint features, [and] target emotional features” which are the sampled data, where said sampled data is generated using the Mel spectrum information (which is mapped to the first data) and the speech signal. The sampled data including “semantic feature vector, voiceprint feature vector, emotional feature vector and phonemic feature vector” is then “compressed through the vector compression layer in the vector processing layer to obtain the audio compression vector,” where the audio compression vector is the first compressed data which is transmitted to the second device in light of the combination with Minnen. As the first compressed data does not include the Mel spectrum information or the speech signal, the first data is not part of the data which is sent to the second device.; Zhang, ¶ [0089], [0091]).
Regarding claim 5, the rejection of claim 1 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. Zhang further discloses further comprising: processing, by a first device, the first data representing first voice characteristics of the first speech to determine modified voice characteristics (Further discloses “extracting the features of the speech signal to obtain the target semantic features… and target phoneme features {...representing the first voice characteristics}” where the “features of the speech signal” are further voice characteristic which will be modified {the modified voice characteristics}, and the sampled data comprising the “semantic feature vector... and phonemic feature vector” are “compressed through the vector compression layer in the vector processing layer” with the “voiceprint feature vector” and “emotional feature vector” to “obtain the audio compression vector”; Zhang, ¶ [0089], [0091]). However, Zhang fails to expressly recite sending, from the first device to a second device, data representing the modified voice characteristics.
The relevance of Minnen is described above with relation to claim 1. Regarding claim 5, Minnen teaches ; and sending, from the first device to a second device, data representing the modified voice characteristics (“the compression and decompression systems may be co-located or remotely located, and compressed data generated by the compression system can be provided to the decompression system in any of a variety of ways. For example, the compressed data may be stored (e.g., in a physical data storage device or logical data storage area), and then subsequently retrieved from storage and provided to the decompression system. As another example, the compressed data may be transmitted over a communications network (e.g., the Internet) to a destination, where it is subsequently retrieved and provided to the decompression system.”; Minnen, ¶ [0060]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the flow-based voice conversion systems of Zhang to incorporate the teachings of Minnen to include sending, from the first device to a second device, data representing the modified voice characteristics. Zhang teaches systems and methods for flow-based voice conversion, which mathematically fuse target voice print features and phoneme features into a single latent vector. However, though Zhang considers remote implementation at a server, Zhang fails to expressly contemplate the details of the distribution of this process to devices and/or between devices across a network. In distributed speech processing, it is well known in the art that transmitting raw audio requires significant bandwidth, and transmitting uncompressed, high dimensional neural network vectors introduces unacceptable latency. Minnen specifically addresses the technical problem of transmitting high dimensional neural network data across networks, including a specialized architecture designed to generate a compressed representation from a latent representation, and transmit that compressed representation to a second device for further use and processing of the underlying data. A PHOSITA would be motivated to modify the flow-based voice conversion process of Zhang with the transmission architecture of Minnen to further enable the server embodiments described in Zhang, while avoiding known latency difficulties and increasing transmission efficiency across a distributed network, as recognized by Minnen. (Minnen, ¶ [0046]-[0048]).
Regarding claim 6, the rejection of claim 5 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. Zhang further discloses wherein the data representing the modified voice characteristics comprises the third data (As previously indicated, the sampled data includes both “voiceprint feature vector, [and] emotional feature vector {third data}” and “semantic feature vector... and phonemic feature vector {data representing the modified third voice}” which are “compressed through the vector compression layer in the vector processing layer to obtain the audio compression vector.” As such, the data representing the modified voice characteristics comprises the third data.; Zhang, ¶ [0089], [0091]).
Regarding claim 7, the rejection of claim 1 is incorporated. Zhang discloses all of the elements of the current invention as stated above. Zhang further discloses wherein: generation of the second data uses a first machine learning model (The sampled data is “compressed through the vector compression layer in the vector processing layer to obtain the audio compression vector” where the vector compression layer is a machine learning model, which is part of “an improved version of the stream-based generative model Waveglow” where the WaveGlow model is operating in the forward direction to map the audio in the latent space.; Zhang, ¶ [0073], [0089]); and generation of the second audio data uses a second machine learning model corresponding to an inverse of the first machine learning model (The system uses the target features and variables within “the affine coupling layer” and “the dual-flow affine transform” based on the “preset speech features” is performed on “the treated vector to obtain the converted voice feature vector,” which is inherently performed by the Dual WaveGlow model operating in the reverse (generative) direction.; Zhang, ¶ [0073], [0077]).
As explained Prenger, WaveGlow, as a flow-based generative model, is by mathematical definition, a bijective function. Executing the forward pass maps the complex data (audio) to a latent distribution (constituting the claimed first machine learning model). Because WaveGlow is a normalizing flow, speech synthesis in Zhang, as performed using the WaveGlow model, is performed by passing latent distributions back through the exact same network using the mathematical inverse of the forward operations (the second machine learning model corresponding to the inverse of the first machine learning model), as shown mathematically in equations 5-8 and explained with reference to “invertibility”, at least at Prenger, page 2, col. 2, para. 1).
Regarding claim 9, the rejection of claim 8 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. However, Zhang fails to expressly recite wherein decompressing the second data to determine the decompressed data comprises decompressing the second data to generate a latent representation of the first speech.
The relevance of Minnen is described above with relation to claim 1. Regarding claim 9, Minnen teaches wherein decompressing the second data to determine the decompressed data comprises decompressing the second data to generate a latent representation of the first speech (“The decompression system 200 processes the compressed data 104 generated by the compression system to generate a reconstruction 202 that approximates the original input data” including “using the entropy decoding engine 206 to entropy decode the compressed representation 126 of the quantized hyper-prior 124 that is included in the compressed data 104. In this example, the entropy decoding engine 206 may entropy decode the compressed representation 126 of the quantized hyper-prior 124 using the same (e.g., predetermined) entropy model that was used to entropy encode it,” which is performed to determine the latent representation, such that the original data can be reconstructed.; Minnen, ¶ [0080]-[0081]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the flow-based voice conversion systems of Zhang to incorporate the teachings of Minnen to include wherein decompressing the second data to determine the decompressed data comprises decompressing the second data to generate a latent representation of the first speech. Zhang teaches systems and methods for flow-based voice conversion, which mathematically fuse target voice print features and phoneme features into a single latent vector. However, though Zhang considers remote implementation at a server, Zhang fails to expressly contemplate the details of the distribution of this process to devices and/or between devices across a network. In distributed speech processing, it is well known in the art that transmitting raw audio requires significant bandwidth, and transmitting uncompressed, high dimensional neural network vectors introduces unacceptable latency. Minnen specifically addresses the technical problem of transmitting high dimensional neural network data across networks, including a specialized architecture designed to generate a compressed representation from a latent representation, and transmit that compressed representation to a second device for further use and processing of the underlying data. A PHOSITA would be motivated to modify the flow-based voice conversion process of Zhang with the transmission architecture of Minnen to further enable the server embodiments described in Zhang, while avoiding known latency difficulties and increasing transmission efficiency across a distributed network, as recognized by Minnen. (Minnen, ¶ [0046]-[0048]).
Regarding claim 11, Zhang discloses A system comprising: at least one processor; and at least one memory comprising instructions that, when executed by the at least one processor, cause the system (Discloses systems and methods described with reference to a “dual-stream speech conversion device, comprising: a memory and at least a processor, in which instructions are stored; At least one processor calls the instruction in the memory to enable the dual-stream speech conversion device to execute the dual-stream speech conversion method described above”; Zhang, ¶ [0059]) to: receive first audio data representing first speech (The system receives the “voice signal includes the time domain signal of the voice data and the frequency domain signal of the voice data” and where “The discrete Fourier transform and logarithmic spectral conversion are performed on the time-domain signals in the preprocessed speech signals to obtain the first spectral information {first audio data}.”; Zhang, ¶ [0072]); receive first data representing first voice characteristics of the first speech (Discloses “extracting the features of the speech signal to obtain the... target voiceprint features, target emotional features {...representing the first voice characteristics}” where the above described target “features of the speech signal” are the first data.; Zhang, ¶ [0091]); process the first audio data and the first data to generate second data representing the first speech (“The server calls the vector compression layer in the vector processing layer to divide the voice signal features into the product of the low-dimensional subspace to obtain the voice signal segmentation vector {processing the first audio data}, and the target audio feature into the product of the low-dimensional subspace to obtain the audio segmentation vector {processing the first data},” and “The audio compression vector {second data} is obtained by mapping the voice signal segmentation vector and the audio segmentation vector into the preset set.”; Zhang, ¶ [0092]); receive third data representing second voice characteristics different from the first voice characteristics (The system relies on voice parameters (also referred to as “preset speech features” or “the value of the target learning variable”) which is used to “indicate the weight ratio for adaptive addition,” where “two-way mapping learning based on the preset speech features is carried out to obtain the candidate dual-flow speech synthesis model” and “the preset speech features are used to indicate the converted speech,” thereby defining the new voice the audio will be converted into.; Zhang, ¶ [0081], [0087]-[0088]) different from the first voice characteristics (Discloses using “the preset speech features” to “obtain the converted voice feature vector,” as part of a voice conversion process. As such, the “preset speech features{second voice characteristics}” are understood to be different from the “target…features {first voice characteristics}”; Zhang, ¶ [0001]-[0002], [0077], [0088], [0091]); process the decompressed data and the third data to generate second audio data (The system uses the target features and variables within “the affine coupling layer” and “the dual-flow affine transform” based on the “preset speech features” is performed on “the treated vector to obtain the converted voice feature vector,”; Zhang, ¶ [0077]); and generate, using the second audio data, audio representing synthesized speech corresponding to the second voice characteristics (“Obtain the value of the target learning variable, and use the normalization layer to weighted normalize the converted voice feature vector based on the value of the target learning variable to obtain the target speech.”; Zhang, ¶ [0080]). However, Zhang fails to expressly recite compress the second data to determine compressed data; decompress the compressed data to determine decompressed data representing the first speech.
The relevance of Minnen is described above with relation to claim 1. Regarding claim 11, Minnen teaches compress the second data to determine compressed data (Discloses “compression of the latent representation 116 of the input data” which is noted as being structurally identical to Zhang’s audio compression vector, “using entropy encoding techniques, the compression system 100 quantizes the latent representation 116 of the input data using a quantizer Q 118 to generate an ordered collection of code symbols 120 (ŷ).” in “accordance with the conditional entropy model.”; Minnen, ¶ [0066], [0076]-[0077]); decompress the compressed data to determine decompressed data representing the first speech (“The entropy decoding engine 208 is configured to entropy decode the compressed code symbols 134 included in the compressed data 104 in accordance with the conditional entropy model to recover the code symbols 120.”; Minnen, ¶ [0082]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the flow-based voice conversion systems of Zhang to incorporate the teachings of Minnen to include compress the second data to determine compressed data; decompress the compressed data to determine decompressed data representing the first speech. Zhang teaches systems and methods for flow-based voice conversion, which mathematically fuse target voice print features and phoneme features into a single latent vector. However, though Zhang considers remote implementation at a server, Zhang fails to expressly contemplate the details of the distribution of this process to devices and/or between devices across a network. In distributed speech processing, it is well known in the art that transmitting raw audio requires significant bandwidth, and transmitting uncompressed, high dimensional neural network vectors introduces unacceptable latency. Minnen specifically addresses the technical problem of transmitting high dimensional neural network data across networks, including a specialized architecture designed to generate a compressed representation from a latent representation, and transmit that compressed representation to a second device for further use and processing of the underlying data. A PHOSITA would be motivated to modify the flow-based voice conversion process of Zhang with the transmission architecture of Minnen to further enable the server embodiments described in Zhang, while avoiding known latency difficulties and increasing transmission efficiency across a distributed network, as recognized by Minnen. (Minnen, ¶ [0046]-[0048]).
Regarding claim 13, the rejection of claim 11 is incorporated. Claim 13 is substantially the same as claim 3 and is therefore rejected under the same rationale as above.
Regarding claim 14, the rejection of claim 13 is incorporated. Claim 14 is substantially the same as claim 4 and is therefore rejected under the same rationale as above.
Regarding claim 15, the rejection of claim 11 is incorporated. Claim 15 is substantially the same as claim 5 and is therefore rejected under the same rationale as above.
Regarding claim 16, the rejection of claim 15 is incorporated. Claim 16 is substantially the same as claim 6 and is therefore rejected under the same rationale as above.
Regarding claim 17, the rejection of claim 11 is incorporated. Claim 17 is substantially the same as claim 7 and is therefore rejected under the same rationale as above.
Regarding claim 19, the rejection of claim 18 is incorporated. Claim 19 is substantially the same as claim 9 and is therefore rejected under the same rationale as above.
Regarding claim 21, the rejection of claim 1 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. Zhang further discloses wherein processing the first audio data and the first data to generate the second data comprises generating the second data to correspond to a predetermined distribution (The sampled data is “compressed through the vector compression layer in the vector processing layer to obtain the audio compression vector” where the recited flow model (relying on the exemplary WaveGlow model) is inherently operating in the forward direction to map the audio in the latent space based on a predetermined distribution (specifically, as explained in Prenger, at pg. 2, Col. 1, para. 1, “WaveGlow is a generative model that generates audio by sampling from... a simple [predetermined] distribution, in our case, a zero mean spherical Gaussian”); Zhang, ¶ [0073], [0089]).
Regarding claim 22, the rejection of claim 11 is incorporated. Claim 22 is substantially the same as claim 21 and is therefore rejected under the same rationale as above.
Claims 2 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Minnen as applied to claim(s) 1 and 11 above, and further in view of Aggarwal (U.S. Pat. No. 11017763, hereinafter Aggarwal).
Regarding claim 2, the rejection of claim 1 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. However, Zhang fails to expressly recite wherein the first speech is captured by a first device and the audio is output by a second device different from the first device.
Aggarwal teaches systems and methods for speech synthesis using normalizing flow. (Aggarwal, ¶ Col. 1, lines 5-10). Regarding claim 2, Aggarwal teaches wherein the first speech is captured by a first device and the audio is output by a second device different from the first device (Discloses a system for “transform text and/or other audio into synthesized speech” comprising “multiple devices... employed in a single system” where each of the “multiple devices may include overlapping components,” and each of the disclosed components may be “disposed on either the user device 110 and/or the remote system 120.” Thus, the system can include two user devices 110, where the first “user device 110... receives” the “text data 14 (and/or audio data) for transformation into audio data that includes a representation of synthesized speech” and the second “user device 110” then “processes (132) the text data using a trained sequence-to-sequence model (and/or other trained model)” to “output a series of power spectrograms, such as Mel-spectrograms, that each correspond to a certain duration of output audio” which are processed “to determine the audio data.”; Aggarwal, ¶ Col. 1, lines 53-65; Col. 2, lines 45-56; Col. 3, lines 1-15, and 35-38; Col. 16, lines 28-36).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the flow-based voice conversion systems of Zhang, as modified by the latent representation transmission systems of Minnen, to incorporate the teachings of Aggarwal to include wherein the first speech is captured by a first device and the audio is output by a second device different from the first device. Zhang teaches systems and methods for flow-based voice conversion, which mathematically fuse target voice print features and phoneme features into a single latent vector. However, though Zhang considers remote implementation at a server for interaction with a terminal, Zhang fails to expressly contemplate the details of specific hardware topology and the distribution of this process to devices and/or between devices. It is well known that normalizing flow networks are computationally expensive, and a person having ordinary skill in the art (PHOSITA) would recognize that forcing a local device, such as a smart phone, to perform the entire conversion pipeline would result in unacceptable battery drain, overheating, and latency. Aggarwal explicitly addresses the physical hardware topology of distributed speech processing. Specifically, Aggarwal teaches a “synthetic speech processing” system that divides computational components between any combination of user devices and remote systems communicating over a network. As such, a PHOSITA would be motivated to modify the flow-based voice conversion process of Zhang with the distributed multi-device processing scheme of Aggarwal to optimize computational resource allocation and minimize latency in a voice conversion system, as recognized by Aggarwal. (Aggarwal, ¶ Col. 15, lines 10-36).
Regarding claim 12, the rejection of claim 11 is incorporated. Claim 12 is substantially the same as claim 2 and is therefore rejected under the same rationale as above.
Claims 10 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Minnen as applied to claim 1 and 11 above, and further in view of Golman (U.S. Pat. App. Pub. No. 2023/0223011, hereinafter Golman).
Regarding claim 10, the rejection of claim 1 is incorporated. Zhang and Minnen disclose all of the elements of the current invention as stated above. However, Zhang fails to expressly recite wherein receiving the third data comprises: receiving an input to a user interface component, the input corresponding to at least one voice characteristic; and based at least in part on the input, determining the third data.
Golman teaches “systems and methods for real-time correction of accent in speech audio signals.” (Golman, ¶ [0002]). Regarding claim 10, Golman teaches wherein receiving the third data comprises: receiving an input to a user interface component (“Input devices 1208, in some examples, may be configured to receive input from a user through tactile, audio, video, or biometric channels” including any “device capable of detecting an input from a user... and relaying the input to the computer system 1200, or components thereof.”; Golman, ¶ [0110]), the input corresponding to at least one voice characteristic (“The user 102 may be provided with an option to select speaker embedding 808 form a list of pretrained speaker embeddings corresponding to different speakers” and the selected speaker embedding corresponds to “voice style {at least one voice characteristic}”; Golman, ¶ [0070]-[0071]); and based at least in part on the input, determining the third data (“The speaker embedding 808 of a target speaker and embeddings of the discretized values of energy 208 and pitch 206 (f0) are further added to the output of the encoder 802 to form input for the decoder 804.”; Golman, ¶ [0070]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the flow-based voice conversion systems of Zhang, as modified by the latent representation transmission systems of Minnen, to incorporate the teachings of Golman to include wherein receiving the third data comprises: receiving an input to a user interface component, the input corresponding to at least one voice characteristic; and based at least in part on the input, determining the third data. Zhang teaches systems and methods for flow-based voice conversion, which mathematically fuse target voiceprint features and phoneme features into a single latent vector based on a preset. However, Zhang relies on preset voice characteristics to dictate the target voice, Zhang fails to expressly contemplate user interaction in selecting the one or more voice characteristics which are integrated into the target voice. Golman teaches an accent correction system including a user interface component which presents the user with the option to select a desired speaker embedding from a list of available speaker embeddings. A PHOSITA would be motivated to modify the static flow-based voice conversion process of Zhang with the speaker characteristic selection described in Minnen to transform Zhang’s preset voice customization into an interactive and customizable voice conversion application based on recognizing the limited functionality for preset voices, thus allowing a broader number of use cases for the system (e.g., obfuscating voice for safety, allowing third party markets for voice profiles, etc.) while maintaining flexibility to meet a wide variety of user needs, as recognized in light of the disclosure of Golman. (Golman, ¶ [0063], [0071]).
Regarding claim 20, the rejection of claim 11 is incorporated. Claim 20 is substantially the same as claim 10 and is therefore rejected under the same rationale as above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zeghidour (U.S. Pat. App. Pub. No. 2023/0019128) teaches compression and decompression systems that can be trained to simultaneously enhance an audio waveform as part of compressing and decompressing the waveform.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sean E. Serraguard whose telephone number is (313)446-6627. The examiner can normally be reached 07:00-17:00 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel C. Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Sean E Serraguard/Primary Examiner, Art Unit 2657
