DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Group I (Claims 1-12) in the reply filed on May 20th, 2026 is acknowledged. The application has pending claims 1-15 (withdrawn claims 13-15 are withdrawn from further consideration).
Claim Objections
Claim 1 is objected to because of the following informalities: “processing a second set of training item (410)” is missing an s after training item. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-12 are rejected under 35 U.S.C. 103 as being unpatentable over Kochanski et al. (hereinafter Kochanski) (US 20030078780 A1) in view of Alvarez et al. (hereinafter Alvarez) (GB 2591245 A) in further view of Raitio et al. (hereinafter Raitio) (Controllable neural text-to-speech synthesis using intuitive prosodic features).
Regarding claim 1, Kochanski discloses: A method for configuring a speaking style for a voice synthesis system (Kochanski, P[0005]: "a novel method and apparatus for synthesizing speech from text is provided, whereby the speech may be generated in a manner so as to effectively convey a particular, selectable style. " (teaches speech synthesis method configured to generated selected a speaking/prosodic style)):
and the outputs of the summarization unit to determine a style basis (340) (Kochanski, P[0052]: "the prosody evaluation module may be used to transform the approximations of psychological features into actual prosodic features. It may be advantageously assumed, for example, that a linear, matrix transformation exists between the approximate psychological and the prosodic features", P[0053]: "a single approximate psychological feature--namely, emphasis--is used to control, via a matrix multiplication, pitch, amplitude, spectral tilt, and duration." (teaches basis-like matrix transformation between style variables and prosodic/style features));
transforming the quality targets to a target style characterization using the style basis (Kochanski, P[0052]: "the prosody evaluation module may be used to transform the approximations of psychological features into actual prosodic features", P[0053]: "a single approximate psychological feature--namely, emphasis--is used to control, via a matrix multiplication, pitch, amplitude, spectral tilt, and duration." (teaches transforming style/quality control values through a matrix/basis into prosodic/style characterization values)); and
configuring the voice synthesis system according to the target style characterization (Kochanski, P[0006]: "selecting one or more prosody control templates based on the particular prosodic style which has been selected for the voice signal synthesis; applying the one or more selected prosody control templates to the one or more identified portions of the predetermined voice control information stream, thereby generating a stylized voice control information stream; and synthesizing the voice signal based on this stylized voice control information stream so that the synthesized voice signal advantageously has the particular desired prosodic style." (teaches configuring and synthesizing speech according to the selected target prosodic/style characterization)).
Kochanski does not explicitly disclose:
configuring a summarizing unit (122) and a synthesizing unit (140) according to values of a plurality of configurable parameters;
processing a second set of training item (410) to determine a style summary for each item as an output of the summarizing unit (130) for an audio representation of the training item, and to determine a plurality of measurements of the training item as outputs of a measurement unit (440), each measurement being a function of at least one of a text representation of the item and an audio representation of said item
using relationships between the measurements
accepting a plurality of quality targets for the speaking style;
However, Alvarez discloses:
configuring a summarizing unit (122) and a synthesizing unit (140) according to values of a plurality of configurable parameters (Alvarez, P[60]: "generate, using an expressivity characterisation module 104, a plurality of expression vectors, where each expression vector is a representation of prosodic information in a reference audio style file; and synthesise expressive speech from the input text, using an expressive acoustic model 106 comprising a deep convolutional neural network that is conditioned by at least one of the plurality of generated expression vectors." (expressivity characterization module reads on summarizing unit, expressive acoustic model/vocoder reads on synthesizing unit, and the generated expression vectors/configuration values condition the synthesizing system));
processing a second set of training item (410) to determine a style summary for each item as an output of the summarizing unit (130) for an audio representation of the training item (Alvarez, P[015]: "an interface for receiving a reference audio style file; and a reference encoder sub-module for compressing prosodic information of the received reference audio style file into a fixed-length vector." (the audio style file is processed to produce a fixed length prosody/style summary output))
It would have been prima facie obvious to a person of ordinary skill in the art, before the earliest filing date of the claimed invention, to have modified Kochanski in view of Alvarez. Doing so would have provided the TTS style control system of Kochanski (Kochanski, P[0005], P[0052]-P[0053]) with the fixed length expression vector/reference encoder architecture and expressive acoustic model of Alvarez (Alvarez, P[015], P[061]), thus, leading to improving implementation of controllable speaking styles by using expression vectors generated from reference audio style files to condition an expressive acoustic model (Alvarez, P[061]).
The combination of Kochanski and Alvarez does not explicitly disclose:
and to determine a plurality of measurements of the training item as outputs of a measurement unit (440), each measurement being a function of at least one of a text representation of the item and an audio representation of said item
using relationships between the measurements
accepting a plurality of quality targets for the speaking style However, Raitio discloses:
and to determine a plurality of measurements of the training item as outputs of a measurement unit (440), each measurement being a function of at least one of a text representation of the item and an audio representation of said item (Raitio, Page 2: "we use acoustic features extracted from the original speech to condition the model. We use fundamental frequency (pitch), phone duration, speech energy, and spectral tilt to model the prosodic space.", "We use automatic speech recognition to force-align the text and audio to obtain the phone durations" (Pitch, energy, and spectral tilt are audio derived measurements. Phone duration is determined from text/audio alignment.));
using relationships between the measurements (Raitio, Page 3: "Then we compared how well the output speech reflects the given target prosodic bias by measuring the corresponding acoustic features from the synthetic utterances." (teaches using measured acoustic feature relationships))
accepting a plurality of quality targets for the speaking style (Raitio, Page 3: "For the prosody control models, we used pitch, pitch range, phone duration, speech energy, and spectral tilt as the conditioning features.", "we synthesized speech at different points in the [−1,1] scale." (teaches plural target/conditioning values for speaking style qualities));
It would have been prima facie obvious to a person of ordinary skill in the art, before the earliest filing date of the claimed invention, to have modified Kochanski in view of Alvarez and Raitio. Doing so would have provided the TTS style control system of Kochanski (Kochanski, P[0005], P[0052]-P[0053]) with the fixed length expression vector/reference encoder architecture and expressive acoustic model of Alvarez (Alvarez, P[015], P[061]) and with the measurable, independtly controllable prosodic conditioning features including pitch, pitch range, phone duration, speech energy and spectral tilt of Raitio (Raitio, Page 2, Sections 3.2-3.3), thus, leading to improved user control of speaking style by enabling independent adjustment of objective prosodic dimensions and verifying the target prosodic bias is reflected in measured synthetic utterances (Raitio, Page 2, section 3.3).
Regarding claim 2, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 1.
Raitio further discloses:
wherein each quality target corresponds to a distinct quality of synthesized speech (Raitio, Page 2, Section 2.2: "We use fundamental frequency (pitch), phone duration, speech energy, and spectral tilt to model the prosodic space." (pitch, duration, energy, and spectral tilt are distinct quality targets corresponding to distinct synthesized-speech qualities)).
Regarding claim 3, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 2.
Raitio further discloses:
wherein the quality targets include at least one quality from a group consisting of pitch, pitch variation, power, and speed (Raitio, Page 3, Section 3.2: "For the prosody control models, we used pitch, pitch range, phone duration, speech energy, and spectral tilt as the conditioning features." (pitch reads on pitch, pitch range reads on pitch variation, speech energy reads on power/loudness, phone duration reads on speed/rate)).
Regarding claim 4, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 2.
Raitio further discloses:
wherein the style basis is selected such that, with variation of a first quality target, variation of qualities of synthesized speech corresponding to other of the quality targets is minimized (Raitio, Page 3, section 3.3: "Each dimension was varied independently", "To measure how well the model can control each prosodic dimension, we synthesized speech at different points in the [−1,1] scale." (Indepedently varying each prosodic dimension teaches reducing undesired variation of other speech qualities when varying a first quality target)).
Regarding claim 5, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 1.
Raitio further discloses:
wherein a range of quality targets that is accepted is limited to correspond to a range in the second training set (Raitio, Page 3: "normalized to a range [−1,1] by first calculating the median (M) and the standard deviation (σ) of each feature, and then projecting the data in the range [M −3σ, M +3σ] into [−1,1]. Finally, we clip values |x| > 1 so that all data is in the range [−1,1]." (accepted feature/target values are limited to a normalized range derived from training data feature statistics)).
Regarding claim 6, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 1.
Raitio further discloses:
further comprising determining the configurable parameters from a first set of training items (110), (Raitio, Page 3: "We trained the following three models", "trained with the 36-hour dataset" (teaches determining model/configurable parameters from training items))
Raitio does not explicitly disclose:
each item comprising a text representation and a corresponding audio representation
However, Alvarez further discloses:
each item comprising a text representation and a corresponding audio representation (Alvarez, P[22]: "the expressive acoustic model may be trained using input text and pre-recorded speech (or pre-synthesized speech) corresponding to the input text. " (teaches each training item having input text and corresponding audio/speech)).
Regarding claim 7, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 1.
Alvarez further discloses:
wherein the summarization unit (130) is configured to accept an audio input and to produce a fixed-length representation of said input as a style summary (Alvarez, P[015]: "The expressivity characterisation module may comprise: an interface for receiving a reference audio style file; and a reference encoder sub-module for compressing prosodic information of the received reference audio style file into a fixed-length vector." (teaches accepting an audio input/reference audio style file and producing a fixed length prosodic/style representation)).
Regarding claim 8, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 1.
Alvarez further discloses:
further comprising:
using the configured voice synthesis system to compute a synthesized utterance (Alvarez, P[60]: "The system 100 comprises an interface 102 for receiving an input text for conversion to speech, and an interface 110 for outputting the synthesised speech." (teaches using system to synthesize/output speech));
causing presentation of the synthesized utterance to a user (Alvarez, P[60]: "an interface 110 for outputting the synthesised speech." (outputting synthesized speech teaches presentation to a user));
receiving in response to the presentation modification of the quality targets from the user (Alvarez, P[60]: "The system 100 may comprise other interfaces (not shown) that enable the system to receive inputs and/or generate outputs (e.g. user selections of expression vectors, etc.)" (teaches receiving user selections/inputs for expression vector/style modification));
Alvarez does not explicitly disclose:
and repeating the steps of computing the synthesized utterance and the causing its presentation and the receiving of the modifications of the quality targets;
However, Raitio further discloses:
and repeating the steps of computing the synthesized utterance and the causing its presentation and the receiving of the modifications of the quality targets (Raitio, Page 3: "synthesized speech at different points in the [−1,1] scale.", "Then we compared how well the output speech reflects the given target prosodic bias by measuring the corresponding acoustic features from the synthetic utterances." (teaches repeated synthesis at different target values, supporting iterative modification/recomputation/presentation));
Regarding claim 9, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 1.
Kochanski further discloses:
wherein using relationships between the measurements and the outputs of the summarization unit to determine a style basis comprises determining the style basis for use in a computational mapping from quality targets to the style characterizations (Kochanski, P[0052]: "the prosody evaluation module may be used to transform the approximations of psychological features into actual prosodic features. It may be advantageously assumed, for example, that a linear, matrix transformation exists between the approximate psychological and the prosodic features" (teaches determining a style-basis/mapping from target style or quality control values to prosodic/style characterizations)).
Regarding claim 10, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 9.
Kochanski further discloses:
wherein determining the style basis comprises computing a linear mapping from a vector representation of quality targets (Kochanski, P[0052]-P[0053]: "a linear, matrix transformation exists between the approximate psychological and the prosodic features", "a single approximate psychological feature--namely, emphasis--is used to control, via a matrix multiplication, pitch, amplitude, spectral tilt, and duration." (teaches linear mapping/matrix multiplication from target/control variables to prosodic features))
Kochanski does not explicitly disclose:
to a vector representation of a style characterization
However, Alvarez discloses:
to a vector representation of a style characterization (Alvarez, P[34]: "perform a linear interpolation or extrapolation between the first expression vector and the second expression vector, using a user-defined scaler value; and generate the user-defined expression vector." (teaches a vector representation of style characterization as an expression vector and linear manipulation)).
Regarding claim 11, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 9.
Alvarez further discloses:
further comprising using correlations of the measurements and the style characterizations to determine the mapping (Alvarez, P[104]: "Then, the features -i.e., single component of an expression vector -which exhibit a linear behaviour in relation to the pitch or the speaking rate variation are detected. ", P[105]: "The offsets of linear features are computed as the difference between the medians of the distributions generated by the aforesaid groups." (teaches determining a mapping in expression vector/style characterization space using measured relationships for pitch/speaking rate variation)).
Regarding claim 12, the combination of Kochanski, Alvarez, and Raitio discloses the method of claim 1.
Alvarez further discloses:
wherein transforming the quality targets to a target style characterization using the style basis comprises using a reference style characterization corresponding to a reference style (Alvarez, P[103]: "the expression vectors of five groups of reference files are extracted and saved: fast speaking rate, slow speaking rate, higher average pitch, lower average pitch, and normal average pitch and speaking rate." (teaches reference expression vectors/style characterizations, including normal/reference pitch and speaking rate style)), and wherein the quality targets represent deviations from a reference style (Alvarez, P[101]: "This method consists of the application of an offset to the expression vector of normal pitch and speaking rate reference audio file to: keep the style, and modify the average pitch and/or speaking rate.", "by means of beta, the user can control the degree of change in the output pitch or speaking rate" (teaches quality targets as offsets/deviations from the normal/reference style)).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHASHIDHAR S MANOHARAN whose telephone number is (571)272-6772. The examiner can normally be reached M-F 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at 571-272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHASHIDHAR SHANKAR MANOHARAN/Examiner, Art Unit 2655
/ANDREW C FLANDERS/Supervisory Patent Examiner, Art Unit 2655
Prosecution Insights