DETAILED ACTION
This communication is in response to the Application filed on November 27, 2024.
Claims 1 - 20 are pending and have been examined.
Claims 1, 8 and 15 are independent.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 26, 2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
The drawings filed on November 27, 2024 have been accepted and considered by the Examiner.
Double Patenting Note
The Examiner notes that previously published U.S. Patent Application Publication 2026/0057265 was analyzed for Double Patenting. However, based on the current claim scope no Double patenting was found.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1 - 2, 5 - 6, 8 - 9, 12 - 13 and 15 - 16, 19 - 20 rejected under 35 U.S.C. 103(a) as being unpatentable over Wang et al., (U.S. Patent Application Publication 2026/0080251), hereinafter referred to as Wang, in view of Hu et al., (“Robust zero-shot text-to-speech synthesis with reverse inference optimization,” arXiv:2407.02243, 2024), hereinafter referred to as Hu.
Regarding Claims 1, 8 and 15, Wang teaches:
1. A method performed by at least one processor, the method comprising:
inputting an input text sequence into a language model to generate a plurality of speech samples, 8. An apparatus comprising, and 15. A non-transitory computer readable medium having instructions stored therein, which when executed by a processor cause the processor to execute a method comprising:
at least one memory configured to store program code; and [Wang, “In some examples, memory 420 may include non-transitory, tangible, machine readable media that includes executable code (i.e., the claimed “store program code”) that when run by one or more processors (e.g., processor 410) may cause the one or more processors to perform the methods described in further detail herein.” Par. 0057]
at least one processor configured to read the program code and operate as instructed by the program code, the program code including: [Wang, “In some examples, memory 420 may include non-transitory, tangible, machine readable media that includes executable code that when run by one or more processors (e.g., processor 410) may cause the one or more processors to perform the methods described in further detail herein.” Par. 0057]
applying one or more preference models to the plurality of speech samples to generate a set of preferred samples and a set of non-preferred samples; [Wang, “In some embodiments, in the CoT critiques, only a few important tokens may determine the final judgement while the remaining tokens improve flow of speech and coherence, as exemplified in the following example evaluation,” Par. 0045; “FIG. 3 is a simplified diagram illustrating a data generation 300 and model training 340 framework, according to some embodiments. Data generation 300 may include a first teacher language model 310 and a second teacher language model 320 (i.e., the claimed “one or more preference models”). In some embodiments, first teacher language model 310 and second teacher language model 320 may be large language models, including the same language model or different language models. In some embodiments, first teacher language model 310 and second teacher language model 320 may be used to generate standard judgement dataset D.sub.Std 330, CoT dataset D.sub.CoT 332, and response deduction dataset D.sub.Ded 334. Each dataset may include positive (i.e., the claimed “set of preferred samples”) and negative samples (i.e., the claimed “set of non-preferred samples”), y.sup.w and y.sup.l, respectively.” Par. 0039]
applying direct performance improvement optimization to preference data associated with the set of preferred samples and the set of non-preferred samples to generate optimized samples; and [Wang, “n some embodiments, preference pairs 250, including both positive and negative evaluations may be utilized with direct preference optimization (“DPO”) to enhance the evaluation capabilities of generative judge models.” Par. 0036; “Positive (i.e., the claimed “set of preferred samples”) and negative examples (i.e., the claimed “set of non-preferred samples”) may be used for training a generative judge model via preference optimization (i.e., the claimed “performance improvement optimization”). In some embodiments, three types of positive and negative examples to improve the capability of generative judges from different perspectives, e.g., preference pairs 250 as shown in FIG. 1.” Par. 0037; “In some embodiments, CoT preference pair 252 may be denoted by y.sup.w={critique, judgement} for a positive sample and y.sup.l={critique′, judgement′} for a negative sample. In some embodiments, standard judgement preference pair 254 may be denoted by y.sup.w={judgement} for a positive sample and y.sup.l={judgement′} for a negative sample. In some embodiments, response deduction preference pair 256 may be denoted y.sup.w={response} for a positive sample and y.sup.l={response′}.” Par. 0038; “Through preference optimization, the judge model may learn to increase the probability of good reasoning traces (i.e., the claimed “optimized samples”) while decreasing that of bad reasoning traces.” Par. 0044; “In this way, the judge model 350 may serve as back-end or front-end processing of texts, ranking, rating, classifying, and/or selecting results (i.e., the claimed “optimized samples’) according to prompted criteria.” Par. 0051; “At step 610, construct a preference training dataset (i.e., the claimed “optimized samples”) for a third neural network based language model (e.g., an LLM as described herein) is constructed based on judgment data generated from the trained second neural network based language model (e.g., judge model 350).” Par. 0109]
training the language model based on the optimized samples. [Wang, “At step 612, the third neural network based language model (e.g., an LLM as described herein) is trained using the constructed preference training dataset (i.e., the claimed “optimized samples”).” Par. 0110; “Through preference optimization, the judge model may learn to increase the probability of good reasoning traces (i.e., the claimed “optimized samples”) while decreasing that of bad reasoning traces.” Par. 0044; “In this way, the judge model 350 may serve as back-end or front-end processing of texts, ranking, rating, classifying, and/or selecting results (i.e., the claimed “optimized samples’) according to prompted criteria.” Par. 0051; “Using three types of preference data D.sub.train= D.sub.CoT∪D.sub.Std∪D.sub.Ded, a DPO training objective for fine-tuning a judge model M.sub.s 350. In some embodiment, parameters of M, are initialized from an instruction-tuned LLM (i.e., the claimed “training the language model”) (e.g. Llama-3.1-8B-Instruct) and are learnable during training (i.e., the claimed “training the language model”). However, the positive examples y.sup.w could be considered as nearly-gold completions (e.g., an evaluation with the judgement matching the ground-truth).” Par. 0050]
Wang fails to explicitly teach speech samples.
However, Hu teaches:
applying one or more preference models to the plurality of speech samples to generate a set of preferred samples and a set of non-preferred samples; [Hu, “In this paper, we introduce reverse inference optimization (RIO) (i.e., the claimed “preference model”), an RLHF related method tailored to improve the robustness of zero-shot TTS,” Pg. 2; “selecting a specific speech sample as a positive exemplar (i.e., the claimed “preferred sample”), Pg. 2; “Conversely, J bad cases are also selected as negative examples to inhibit generating such undesirable samples (i.e., the claimed “set of non-preferred samples”). Pg. 4; “positive pool Ppos (i.e., the claimed “set of preferred samples”) and a negative pool Pneg (i.e., the claimed “set of non-preferred samples”),” Pg. 4]
applying direct performance improvement optimization to preference data associated with the set of preferred samples and the set of non-preferred samples to generate optimized samples; and [Hu, “direct preference optimization (DPO),” Pg. 3; “selecting a specific speech sample as a positive exemplar (i.e., the claimed “preferred sample”), Pg. 2; “Conversely, J bad cases are also selected as negative examples to inhibit generating such undesirable samples (i.e., the claimed “set of non-preferred samples”). Pg. 4; “positive pool Ppos (i.e., the claimed “set of preferred samples”) and a negative pool Pneg (i.e., the claimed “set of non-preferred samples”),” Pg. 4; “satisfactory sample generated (i.e., the claimed “generate optimized sample”) by a robust zero-shot TTS system,” Pg. 2]
training the language model based on the optimized samples. [Hu, “We suggest that his pipeline offers a viable strategy to address the discrepancy in history sequence between TTS training and test, as it exposes the model (i.e., the claimed “language model”) with the samples (i.e., the claimed “optimized samples”) with self-regenerated history during training (i.e., the claimed “training the language model based on the optimized samples”).” Pg. 2; “satisfactory sample generated (i.e., the claimed “generate optimized sample”) by a robust zero-shot TTS system,” Pg. 2]
Wang and Hu pertain to machine learning systems for natural language processing and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the machine learning systems for natural language processing art to modify Wang’s teachings of “direct preference optimization (“DPO”)” (Wang Par. 0036) with the explicit teachings of “speech samples” (Hu, Pg. 2) taught by Hu in order to overcome the problem of “sub-optimal robustness that results in unstable speech synthesis performance” (Hu, Pg. 1).
Regarding Claims 2, 9 and 16, Wang in view of Hu has been discussed above. The combination further teaches:
wherein the one or more preference models score each sample from the generated samples, [Wang, see mapping applied to claim 1; Hu, see mapping applied to claim 1; “We selected 200 positive samples and 200 negative samples according to RIO policy for RLHF optimization. Specifically, we first calculate the average MOS of zero-shot inference and reverse inference results, where the samples with top average MOS (i.e., the claimed “score”) are set as positive samples (i.e., the claimed “generated samples”) and those with bottom average MOS (i.e., the claimed “score”) are set as negative samples (i.e., the claimed “generated samples”). Then, we use the WER metric to further refine the quality of exemplars, where only the positive samples with WER lower than 10% and the negative samples with WER higher than 10% are maintained.” Pg. 5]
wherein each sample above a threshold is assigned to the set of preferred samples, and [Wang, see mapping applied to claim 1; Hu, see mapping applied to claim 1; “We selected 200 positive samples and 200 negative samples according to RIO policy for RLHF optimization. Specifically, we first calculate the average MOS of zero-shot inference and reverse inference results, where the samples with top average MOS are set as positive samples and those with bottom average MOS are set as negative samples. Then, we use the WER metric to further refine the quality of exemplars, where only the positive samples (i.e., the claimed “preferred samples”) with WER lower than 10% (i.e., the claimed “above a threshold”) and the negative samples with WER higher than 10% are maintained.” Pg. 5; “In this process, MOSNet is employed as the discriminator according to a threshold,” Pg. 4]
each sample below or equal to the threshold is assigned to the set of non-preferred models. [Wang, see mapping applied to claim 1; Hu, see mapping applied to claim 1; Hu, “We selected 200 positive samples and 200 negative samples according to RIO policy for RLHF optimization. Specifically, we first calculate the average MOS of zero-shot inference and reverse inference results, where the samples with top average MOS are set as positive samples and those with bottom average MOS are set as negative samples. Then, we use the WER metric to further refine the quality of exemplars, where only the positive samples with WER lower than 10% and the negative samples (i.e., the claimed “non-preferred samples”) with WER higher than 10% (i.e., the claimed “threshold”) are maintained.” Pg. 5; “In this process, MOSNet is employed as the discriminator according to a threshold,” Pg. 4]
Regarding Claims 5, 12 and 19, Wang in view of Hu has been discussed above. The combination further teaches:
wherein the one or more preference models are pre-trained in accordance with one or more alignment metrics comprising intelligibility, speaker similarity, and evaluation scores. [Wang, see mapping applied to claim 1; Hu, see mapping applied to claim 1; Wang, “For example, initial training (e.g., pre-training) may be performed on one set of training data, and then an additional training stage (e.g., fine-tuning) may be performed using a different set of training data.” Par.0082; “At step 608, a second neural network based language model (e.g., judge model 350 of FIG. 3) is trained using the preference training sample to judge whether a model-generated response to the user query aligns with user preference.” Par. 0108; Hu, “pre-trained models,” Pg. 5; Hu, “Our experimental results verify that RIO can effectively improve both subjective and objective metrics (i.e., the claimed “alignment metrics”), including mean opinion scores (i.e., the claimed “evaluation scores”), word error rates, and speaker similarity.” Pg. 1]
Regarding Claims 6, 13 and 20, Wang in view of Hu has been discussed above. The combination further teaches:
wherein the language model is a large language model. [Wang, see mapping applied to claim 1; Hu, see mapping applied to claim 1; Wang “In some embodiments, first teacher language model 310 and second teacher language model 320 may be large language models, including the same language model or different language models.” Par. 0039; Hu, “Large language models,” Pg. 1]
Claims 3 - 4, 10 - 11 and 17 - 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over Wang in view of Hu as applied in claim 1 above, and in further view of Zhang et al., (CN119128070A), hereinafter referred to as Zhang.
Regarding Claims 3, 10 and 17, Wang in view of Hu has been discussed above. The combination further teaches:
wherein each speech sample comprises a condition sequence that includes a tokenized input sequence and a corresponding tokenized target sequence. [Wang, see mapping applied to claim 1; Hu, see mapping applied to claim 1; Hu, “Under this setup, the speech signal is firstly tokenized into sequences (i.e., the claimed “tokenized input sequence”) of discrete units based on vector quantization,” Pg 3; Hu, “target transcripts containing over 6 tokens (i.e., the claimed “tokenized target sequence”), ; Wang, “First, the input sequence may be tokenized and converted into embeddings, which are dense numerical representations, e.g., vectors of values. Positional encodings are added to these embeddings to provide information about the order of tokens.” Par. 0068]
The combination fails to teach a condition sequence.
However, Zhang teaches:
wherein each speech sample comprises a condition sequence that includes a tokenized input sequence and a corresponding tokenized target sequence. [Zhang, “It returns two tensors, the input sequence x and the target sequence y (i.e., the claimed “condition sequence”)”. Par. n0134]
Wang, Hu and Zhang pertain to machine learning systems for natural language processing and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in the machine learning systems for natural language processing art to modify Wang’s teachings of “direct preference optimization (“DPO”)” (Wang Par. 0036) with the explicit teachings of “speech samples” (Hu, Pg. 2) taught by Hu and the teachings of returning condition sequences of “input sequence x and the target sequence y” (Zhang, Par. n0134) taught by Zhang in order to overcome the problem of “sub-optimal robustness that results in unstable speech synthesis performance” (Hu, Pg. 1) and “customize the combination of basic models” (Zhang, Par. n0004).
Regarding Claims 4, 11 and 18, Wang in view of Hu has been discussed above. The combination further teaches:
wherein the preference data is a data triplet is formed based on the condition sequence, at least one preferred sample from the set of preferred samples, and at least one non-preferred sample from the set of non-preferred samples. [Wang, see mapping applied to claims 1, 3; Hu, see mapping applied to claims 1, 3; Zhang, see mapping applied to claim 3; Wang, “Using three types of preference data (i.e., the claimed “data triplet”) D.sub.train=D.sub.CoT∪D.sub.Std∪D.sub.Ded, a DPO training objective for fine-tuning a judge model M.sub.s 350.” Par. 0050; Chen, “It returns two tensors, the input sequence x and the target sequence y (i.e., the claimed “condition sequence”)”. Par. n0134; “Positive (i.e., the claimed “set of preferred samples”), negative examples (i.e., the claimed “set of non-preferred samples”), Par. 0037]
Claims 7 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Wang in view of Hu as applied in claim 1 above, and in further view of Lu et al., (CN122073634A), hereinafter referred to as Lu.
Regarding Claims 7 and 14, Wang in view of Hu has been discussed above. The combination further teaches:
wherein the plurality of speech samples are generated by inputting the input text sequence into the language model N different times, [Wang, see mapping applied to claim 1; Hu, see mapping applied to claim 1]
The combination fails to teach N different times and N is an integer greater than 0.
However, Lu teaches:
wherein the plurality of speech samples are generated by inputting the input text sequence into the language model N different times, [Lu, “generated by iteratively running a generation model N times (i.e., the claimed “N different times”), where N is an integer greater than 1.” Par. n0004]
wherein N is an integer greater than 0. [Lu teaches N is an integer greater than 1. Thus, N is greater than 0: “generated by iteratively running a generation model N times, where N is an integer greater than 1.” Par. n0004]
Wang, Hu and Lu pertain to generation systems and are analogous to the instant application. Accordingly, it would have been obvious to one of ordinary skill in generation systems art to modify Wang’s teachings of “direct preference optimization (“DPO”)” (Wang Par. 0036) with the explicit teachings of “speech samples” (Hu, Pg. 2) taught by Hu and the teachings of “N times”/ N different times (Lu, Par. n0004) taught by Lu in order to overcome the problem of “sub-optimal robustness that results in unstable speech synthesis performance” (Hu, Pg. 1) and “ensure generation quality” (Lu, Par. n0003).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Crabtree et al., (U.S. Patent Application Publication 2026/0154553) teaches direct preference optimization (“DPO”).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EUNICE LEE whose telephone number is 571-272-1886. The examiner can normally be reached M-F 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EUNICE LEE/Examiner, Art Unit 2656
/BHAVESH M MEHTA/ Supervisory Patent Examiner, Art Unit 2656
Prosecution Insights