DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments with respect to 35 U.S.C. 101 Abstract Idea in regards to claims 1-20 (and the newly added claims 21-26) have been considered, however are not found to be persuasive due to the following reasons. Examiner respectfully disagrees with Applicant’s arguments because the claims are clearly directed to the abstract idea of organizing and processing information, which is essentially mimicking a human metal process. The claims describe the basic concept of reading a text script, identifying natural places to take a breath or pause, and speaking the words out loud in chunks. Under the current U.S. patent law, automating something a human naturally does in their mind, like evaluating text to decide when to pause while speaking or simply organizing data into chucks based on labels, is considered an abstract idea that cannot be patented on its own.
Furthermore, the claims fail to add an inventive concept that transforms this abstract idea into a specific, technical invention. While the text mentions computer components like “processors”, “buffers,” “language models” and “text-to-speech models”, it relies on them purely for their standard, generic functions. Storing data in a buffer, scanning that data until a specific label is found, and sending it to an output module are basic, conventional computer operations. The claims do not provide any technical improvement to how a computer or an AI model operates; it merely uses off-the-shelf technology to automate a known process. Newly added claims 21-26 are further rejected.
Applicant's arguments with respect to 35 U.S.C. 103 in regards to claims 1, 8 and 15 have been considered, however are not found to be persuasive due to the following reasons. Examiner respectfully disagrees with Applicant’s arguments with regards to stating Yang in Hassid combination do not teach the limitations of claims 1, 8 and 15 because Yang et al. in view of Hassid et al. in view of Fleizach et al. clearly teach all the limitations. The newly added limitations are taught by Fleizach. See detailed reason for allowance below. Fleizach teaches a text-to-speech markup in which pause instructions, including long and short pauses, are include in text prepared for a voice synthesizer, and the voice synthesizer inserts the pauses at designated locations. Therefore, the claims are still rejected.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-6, 8, 10-13, 15, 17-19 and 21-26 are rejected under 35 U.S.C. 101.
Claims 1, 8 and 15 are directed to an abstract idea because it primarily recites organizing and presenting information (i.e. generating a textual response (from a language model), labeling portions of that text with pause indicators, buffering the text, and conditionally outputting portions for speech presentation. At a high level, the claim describes managing the flow of content (tokens/words) and annotating it with timing cues (pause labels) to control when parts of the response are provided for speech output, which is a form of information processing and output formatting that can be performed mentally (deciding where to pause) or with pen-and-paper (marking pauses), and then conveyed via a generic delivery channel (speech). The “digital human” is functionally just a speaker/presenter, and the claim’s focus remains on the rule-based selection and sequencing of content for output rather than a specific improvement to computer functionality.
The claim recites a method of collecting, analyzing, and outputting information: obtaining a response, identifying pause locations, storing it, processing tokens until a label/indicator is detected, and providing a portion of the response for TTS presentation. This aligns with groupings such as (i) mental processes (evaluating where a speaker would pause; deciding when to stop reading and speak a chunk), (ii) methods of organizing human activity / managing interactions (controlling how/when an agent presents a response), and (iii) certain methods of organizing information (annotating text with pause labels; segmenting output). The claim does not clearly recite an improvement to the functioning of a computer or another technology; instead, it uses generic components (processor, memory, buffer, language model, TTS model) to implement the abstract idea of segmenting and presenting a response with pauses. The buffering and token processing are described functionally and at a high level, without a particular technical mechanism (e.g., a specific data structure, timing model, scheduling algorithm, or low-level speech synthesis improvement) that would indicate a technological improvement rather than automation of a presentation concept.
The additional elements, “processor coupled to a memory,” “buffer,” “language model,” “text-to-speech model,” and “digital human” do not amount to significantly more than the abstract idea, because they are recited as conventional computing components performing their ordinary functions (storing data, processing tokens, generating text, synthesizing speech, and outputting audio). The claim does not require a particular architecture, specialized hardware, or a nonconventional technical solution that transforms the abstract idea into a patent-eligible application; rather, it broadly claims the result of producing spoken output with pauses based on labeled text. Accordingly, Claim 1 is ineligible under 101 as being directed to an abstract idea without an inventive concept sufficient to transform it into patent-eligible subject matter.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claims are (i) mere instructions to implement the idea on a computer, and/or (ii) recitation of generic computer structure that serves to perform generic computer functions that are well-understood, routine, and conventional activities previously known to the pertinent industry. Viewed as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter. There is further no improvement to the computing device.
Dependent claims 3-6, 10-13, 17-19 and 21-26 further recite an abstract idea performable by a human and do not amount to significantly more than the abstract idea as they do not provide steps other than what is conventionally known in audio/text processing.
Claims 3, 10 and 17: prompt-based control is a generic way of configuring software behavior and does not add a technological improvement or “significantly more” than the abstract idea.
Claims 4 and 11: executing an abstract idea on an edge device is a generic computer implementation that does not confer eligibility.
Claims 5, 12 and 18: the use of message passing does not add an inventive concept; it is routine networking/software plumbing applied to the same abstract information-processing workflow.
Claims 6, 13 and 19: adding a session identifier is merely organizing and tracking information (metadata management) for a communication session.
Claims 21, 23 and 25: it merely adds the abstract idea of preparing dialogue responsive text with pause marks where a person would naturally pause, which is a mental process, and it does not add any specific technological improvement.
Claims 22, 24 and 26: it merely adds the generic data-handling step of collecting tokens in a buffer until a pause mark is found, which is routine computer processing used to carry out the abstract idea and does not add significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 8, 10-12, 15, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (Duration-Aware Pause Insertion… For Multi-Speaker Text-to-Speech”; IEEE 2023) in view of Hassid et al. (WO 2022093192) in view of Fleizach et al. (US 8,996,376).
Claims 1, 8 and 15,
Yang teaches a method, comprising: obtaining at least one response, generated by at least one language model, to be delivered by at least one processor-based digital human in a spoken format ([Abstract] we propose more powerful pause insertion frameworks based on a pre-trained language model; uses BERT pre-trained on a large-scale text corpus; essential part of making computers and robots speak as fluently as human speakers),
wherein one or more of the at least one response comprises a plurality of words and one or more predicted pause labels ([Abstract] performs duration-aware pause insertion, predicting both RPs and punctuation-indicated pauses (PIPs) that are categorized by duration),
wherein the one or more predicted pause labels identify a respective portion of the corresponding response where a human speaker is expected to pause ([Introduction] there are two main types of silent pause: respiratory pauses (RPs) and punctuation-indicated pauses (PIPs); the former is inserted at word transitions without punctuation to utter long sentences fluently, and the latter is inserted at punctuation marks following text descriptions);
wherein the one or more predicted pause labels are generated by the at least one language model ([1.] [Fig. 2] the architecture is “based on BERT and BiLSTM,” and that the “CPI model … predicts both RPs and PIPs (including their position and category).”; Fig. 2 shows “text” input to “BERT”, then outputs “Probability” and “Category” for “RP” and “PIP” ),
wherein the at least one language model is trained to perform pause insertion ([Abstract] [5.1] “Pause insertion … is an essential part of TTS systems,” and “we propose more powerful pause insertion frameworks based on a pre-trained language model.” “Four labels, namely P-RPs, C-RPs, P-PIPs and C-PIPs, were the targets of the four outputs of the CPI model.”);
wherein the at least one processor-based digital human transforms the at least the portion of the given response into the spoken format, using the at least one text-to-speech model, based at least in part on the predicted pause label ([Introduction] enable phoneme-based TTS models to predict more accurate silent pauses; we further propose the CPI model for phoneme-based TTS models, in which both RPs and PIPs are categorized by duration, and their position and category are predicted);
The difference between the Yang the claimed invention is that Yang does not explicitly teach storing the at least one response, generated by the at least one language model, in at least one buffer; processing at least one token, comprising at least a portion of one or more words, from the at least one buffer, of a given one of the at least one response until a predicted pause label is detected in a given one of the at least one token; and providing at least a portion of the given response, from the at least one buffer, to at least one text-to-speech model of the at least one processor-based digital human for presentation to at least one user, in response to detecting at least one predicted pause label in the given token; wherein the at least one predicted pause label provides an indication to the at least one text-to-speech model to insert a pause into a designated location of the spoken format of the at least the portion of the given response.
Hassid teaches storing the at least one response, generated by the at least one language model, in at least one buffer ([Summary] [0098] at the TTS system, accumulating a first sub-string comprising a first portion of the text string received; the term “accumulate” may be considered to be an incremental receipt of characters and/or words at the TTS system);
processing at least one token, comprising at least a portion of one or more words, from the at least one buffer, of a given one of the at least one response until one or more of a predicted pause label and a final token indicator is detected in a given one of the at least one token ([00138] incrementally accumulating one successive word at a time from the received real-time streaming text into a second interim sub-string, and after each successive accumulation of a successive word into the second interim sub-string, applying punctuation model to the second interim sub-string to generate a pre-processed second interim sub-string; then setting a second trigger point is set to: (i) an occurrence in the pre-processed second interim sub-string of a second particular punctuation that delimits the second interim sub-string for TTS synthesis processing, or (ii) a signal indicating the endpoint of the received real-time streaming text); and
providing at least a portion of the given response, from the at least one buffer, to at least one text-to-speech model of the at least one processor-based digital human for presentation to at least one user, in response to detecting the one or more of the predicted pause label and the final token indicator in the given token ([0098] [00107] TTS synthesis processing can be applied to at least the pre-processed first sub-string; a trigger point can also be the end of a complete input string and/or detection a “send” command),
wherein the method is performed by at least one processing device comprising a processor coupled to a memory ([0006] processor; memory).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the teachings of Yang with teachings of Hassid by modifying the duration-aware pause insertion using pre-trained language model for multi-speaker text to speech as taught by Yang to include storing the at least one response in at least one buffer; processing at least one token, comprising at least a portion of one or more words, from the at least one buffer, of a given one of the at least one response until one or more of a predicted pause label and a final token indicator is detected in a given one of the at least one token; and providing at least a portion of the given response, from the at least one buffer, to at least one text-to-speech model of the at least one processor-based digital human for presentation to at least one user, in response to detecting the one or more of the predicted pause label and the final token indicator in the given token as taught by Hassid for the benefit of converting written language into artificially produced speech or spoken language (Hassid [0023]).
The difference between the prior art and the claimed invention is that Yang nor Hassid explicitly teach wherein the at least one predicted pause label provides an indication to the at least one text-to-speech model to insert a pause into a designated location of the spoken format of the at least the portion of the given response.
Fleizach teaches wherein the at least one predicted pause label provides an indication to the at least one text-to-speech model to insert a pause into a designated location of the spoken format of the at least the portion of the given response ([Fig. 9(b)] text marked up document 950 can be regarded as an example of a text-to-speech processing script created by the text-to-speech analyzer 151 of FIG. 1(b); marked up text document 950, pauses can indicate punctuation, for example a voice synthesizer can insert long pause 951 to indicate two new lines, a short pause 953 to indicate a dash, a short pause 963 to indicate a font style change, and a long pause 967 to indicate extended punctuation).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the teachings of Yang and Hassid with teachings of Fleizach by modifying the duration-aware pause insertion using pre-trained language model for multi-speaker text to speech as taught by Yang to include wherein the at least one predicted pause label provides an indication to the at least one text-to-speech model to insert a pause into a designated location of the spoken format of the at least the portion of the given response as taught by Fleizach for the benefit of presenting to a user so that the user can hear a summary of the document without having to process the document to produce its spoken text via text-to-speech conversion (Fleizach [Abstract]).
Claims 3, 10 and 17,
Yang further teaches the method of claim 2, wherein the at least one language model is instructed using a system prompt to insert a pause in a generated response where a speaker is expected to pause ([Abstract] [Introduction] [Section 2] pauses correspond to where human speakers usually insert silent pauses and defines pause types (RPs and PIPs) that are inserted for fluent speech; a BERT based pause insertion framework; the model predicts the position of pauses after tokens).
Claims 4 and 11,
Yang further teaches the method of claim 1, wherein the at least one language model is provided by at least one edge device ([Introduction] TTS system using BERT).
Claims 5, 12 and 18,
Hassid further teaches wherein the at least one response is provided to the at least one buffer using at least one message passing interface ([Summary] [0098] [00138] buffers/accumulates streaming text and accumulating a first sub-string comprising a first portion of the text string; it is unclear the meaning of “message passing interference” means as it is not described in the Applicant’s specifications; Examiner is interpreting is has an audio/text streaming being buffered).
Claims 6, 13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (Duration-Aware Pause Insertion… For Multi-Speaker Text-to-Speech”; IEEE 2023) in view of Hassid et al. (WO 2022093192) in view of Fleizach et al. (US 8,996,376) and further in view of Lee (EP 1540495).
Claims 6, 13 and 19,
Yang, Hassid and Fleizach teach all the limitations in claim 1. The difference between the prior art and the claimed invention is that Yang, Hassid nor Fleizach explicitly teach wherein the at least one buffer comprises one more of: (i) the at least one response associated with a given user and (ii) the at least one response associated with a plurality of users and wherein the at least one response comprises a session identifier
Lee teaches wherein the at least one buffer comprises one more of: (i) the at least one response associated with a given user and (ii) the at least one response associated with a plurality of users ([Fig. 4] the outbound chat message 400 comprises a message type 401 (e.g., text, speech, and so on), a number of intended recipients 402, a plurality of recipient identifiers 403,) and wherein the at least one response comprises a session identifier ([Fig. 4] the mobile terminal 100 generates the thread identifier 404 by aggregating a client identifier and a session identifier with a thread sequence number).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the teachings of Yang, Hassid and Fleizach with teachings of Lee by modifying the duration-aware pause insertion using pre-trained language model for multi-speaker text to speech as taught by Yang to include wherein the at least one buffer comprises one more of: (i) the at least one response associated with a given user and (ii) the at least one response associated with a plurality of users and wherein the at least one response comprises a session identifier as taught by Lee for the benefit of distinguishing old speech message when they occurred in the thread relative to other message in that thread ([Background] Lee).
Allowable Subject Matter
Claims 21-26 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims AND overcome the 101 Abstract Idea set forth.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHREYANS A PATEL whose telephone number is (571)270-0689. The examiner can normally be reached Monday-Friday 8am-5pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SHREYANS A. PATEL
Primary Examiner
Art Unit 2653
/SHREYANS A PATEL/ Examiner, Art Unit 2659 1865
Prosecution Insights