DETAILED ACTION
Introduction
1. This office action is in response to Applicant’s submission filed on 10/28/2024. Claims 1-20 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on 10/31/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
4. Claims 1-10 and 15-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Independent Claims 1 and 15 recite various limitations that, but for the recitation of generic computer components (i.e. memory devices, processors), can be performed in the human mind or with simple tools. The claims under their broadest reasonable interpretation cover the concept of receiving audio/text data and a format, generating input for a decoder, and generating text or speech (see MPEP 2106.04(a)(2) III).
This judicial exception is not integrated into a practical application because the claims only recite elements in the form of “a computing device." These elements are used to perform the claimed methods and steps and are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because they do not include subject matter that could not be performed by a human, as discussed above with respect to integration of the abstract idea into a practical application, the additional element of using the generic computing elements to perform the claimed elements amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are not patent eligible.
Regarding claims 2 and 16, these claims further limit the elements of claims 1 and 15 by reciting the unified vocabulary is generated by merging a speech token and a text token. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claims 2 and 16 are not patent eligible.
Regarding claims 3 and 17, these claims further limit the elements of claims 1 and 15 by reciting the special token is designated to guide the unified decoder-only model in performing the multiple tasks related to text and speech. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claims 3 and 17 are not patent eligible.
Regarding claims 4 and 18, these claims further limit the elements of claims 1 and 15 by reciting the data format is specified such that the special token and a token belonging to the unified vocabulary are alternately connected. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claims 4 and 12 are not patent eligible.
Regarding claims 5 and 19, these claims further limit the elements of claims 1 and 15 by reciting the special token comprises: a first special token indicating the beginning of the text; a second special token indicating the beginning of the speech; a third special token instructing the unified decoder-only model to generate the text; and a fourth special token instructing the unified decoder-only model to generate the speech. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claims 5 and 19 are not patent eligible.
Regarding claim 6, this claims further limit the elements of claim 1 by reciting the multiple tasks related to text and speech comprise: a first task related to STT (speech-to-text) for speech recognition; a second task related to TTS (text-to-speech) for speech synthesis; a third task related to TTT (text-to-text) for text generation; and a fourth task related to STS (speech-to-speech) for speech generation. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claim 6 is not patent eligible.
Regarding claim 7, this claims further limit the elements of claim 1 by reciting generating, by the processor, the input to include the second special token, a speech token from the unified vocabulary, and the third special token to guide the unified decoder-only model in performing the first task. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claim 7 is not patent eligible.
Regarding claim 8, this claims further limit the elements of claim 1 by reciting generating, by the processor, the input to include the first special token, a text token from the unified vocabulary, and the fourth special token to guide the unified decoder-only model in performing the second task. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claim 8 is not patent eligible.
Regarding claim 9, this claims further limit the elements of claim 1 by reciting generating, by the processor, the input to include the third special token and a token from the unified vocabulary to guide the unified decoder-only model in performing the third task. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claim 9 is not patent eligible.
Regarding claim 10, this claims further limit the elements of claim 1 by reciting generating, by the processor, the input to include the fourth special token and a speech token from the unified vocabulary to guide the unified decoder-only model in performing the fourth task. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claim 10 is not patent eligible.
Regarding claim 20, this claims further limit the elements of claim 15 by reciting the multiple tasks related to text and speech comprises: a first task related to STT (speech-to-text) for speech recognition; a second task related to TTS (text-to-speech) for speech synthesis; a third task related to TTT (text-to-text) for text generation; and a fourth task related to STS (speech-to-speech) for speech generation, and wherein the generating input for the unified decoder-only model comprises generating the input to include at least one of the first to fourth special tokens and a text token or a speech token from the unified vocabulary to guide the unified decoder-only model in performing any one of the first to fourth tasks. However, this does not preclude the limitations from being performed in the human mind or by simple tools. Similar to claim 1 above, no additional elements beyond the use of generic computing elements are claimed, therefore the judicial exception is not integrated into a practical application nor are the claim elements sufficient to amount to significantly more than the judicial exception. Therefore, claim 20 is not patent eligible.
Claim Rejections - 35 USC § 103
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-20 are rejected under 35 U.S.C. 103 as unpatentable over U.S. Pat. Ap. Pub. No. 20240428056 (Rubenstein et al., hereinafter “Rub”) in view of U.S. Pat. Ap. Pub. No. 20230028693 (Marzinzik et al., hereinafter “Mar”).
With regard to Claim 1, Run describes:
“A method for performing text and speech tasks, performed by a computing device comprising a processor and a storage medium, using a unified decoder-only model to execute multiple tasks related to text and speech, the method comprising:
reading, by the processor, a unified vocabulary from the storage medium or an external storage medium connected via a network; (Paragraph 48 describes that the device reads tokens from a stored vocabulary.)
generating, by the processor, input for the unified decoder-only model according to the data format, using a token from the unified vocabulary and a predetermined special token; and (Paragraph 53 describes that in input text tag (“special token”) is used to determine the task to be performed and to generate data for the decoder.)
providing, by the processor, the input to the unified decoder-only model to obtain inference results for the multiple tasks related to text and speech. (Paragraph 29 describe the use of a decoder only model for performing text and speech tasks.)
Rub does not explicitly describe “reading, by the processor, information regarding a predetermined data format from the storage medium or the external storage medium.” However, paragraph 27 of Mar describes that a speech/text device can translate output to a particular format.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the particular format as described by Mar into the system of Rub to allow for specifically selected formats, as described in paragraph 27 of Mar.
With regard to Claim 2, Rub describes “the unified vocabulary is generated by merging a speech token and a text token.” Paragraph 30 describes that the task token and input data tokens are used together to generate the output.
With regard to Claim 3, Rub describes “the special token is designated to guide the unified decoder-only model in performing the multiple tasks related to text and speech.” Paragraph 30 describes that the task taken guides the device to the proper text/speech task.
With regard to Claim 4, Rub describes “the data format is specified such that the special token and a token belonging to the unified vocabulary are alternately connected.” Paragraph 30 describes that the task token and input data tokens are used together to generate the output, and thus are “alternately connected.”
With regard to Claim 5, Rub describes “the special token comprises: a first special token indicating the beginning of the text; a second special token indicating the beginning of the speech; a third special token instructing the unified decoder-only model to generate the text; and a fourth special token instructing the unified decoder-only model to generate the speech.” Paragraph 31 describes that the text and/or speech is generated based on the input tokens as well as the task tokens which specify the text/speech task to be performed. The input data would include at least one token that would correspond to each of the beginning of the speech and/or text input.
With regard to Claim 6, Rub describes “the multiple tasks related to text and speech comprise: a first task related to STT (speech-to-text) for speech recognition; a second task related to TTS (text-to-speech) for speech synthesis; a third task related to TTT (text-to-text) for text generation; and a fourth task related to STS (speech-to-speech) for speech generation.” Paragraph 53 describes all of these possible tasks.
With regard to Claim 7, Rub describes “generating, by the processor, the input to include the second special token, a speech token from the unified vocabulary, and the third special token to guide the unified decoder-only model in performing the first task.” Paragraph 31 describes that the text and/or speech is generated based on the input tokens as well as the task tokens which specify the text/speech task to be performed. Paragraph 53 describes all of the claimed speech/text tasks.
With regard to Claim 8, Rub describes “the generating input for the unified decoder-only model comprises: generating, by the processor, the input to include the first special token, a text token from the unified vocabulary, and the fourth special token to guide the unified decoder-only model in performing the second task.
Paragraph 31 describes that the text and/or speech is generated based on the input tokens as well as the task tokens which specify the text/speech task to be performed. Paragraph 53 describes all of the claimed speech/text tasks.
With regard to Claim 9, Rub describes “generating, by the processor, the input to include the third special token and a token from the unified vocabulary to guide the unified decoder-only model in performing the third task.
Paragraph 31 describes that the text and/or speech is generated based on the input tokens as well as the task tokens which specify the text/speech task to be performed. Paragraph 53 describes all of the claimed speech/text tasks.
With regard to Claim 10, Rub describes “generating, by the processor, the input to include the fourth special token and a speech token from the unified vocabulary to guide the unified decoder-only model in performing the fourth task.”
Paragraph 31 describes that the text and/or speech is generated based on the input tokens as well as the task tokens which specify the text/speech task to be performed. Paragraph 53 describes all of the claimed speech/text tasks.
With regard to Claim 11, Rub describes “training, by the processor, the unified decoder-only model to perform the first task, wherein the training is performed based on training data that includes the second special token, a speech token from the unified vocabulary, the third special token, and a text token from the unified vocabulary.”
Paragraph 29 describes that the decoder-only device can be trained to perform multiple tasks or combinations of tasks that involve arbitrarily interleaved speech and text. For example, the system can obtain a sequence of input tokens that include text tokens, audio tokens, or both text and audio tokens, selected from a vocabulary of text tokens and audio tokens. Paragraph 53 describes all of the claimed speech/text tasks.
With regard to Claim 12, Rub describes “training, by the processor, the unified decoder-only model to perform the second task, wherein the training is performed based on training data that includes the first special token, a text token from the unified vocabulary, the fourth special token, and a speech token from the unified vocabulary.”
Paragraph 29 describes that the decoder-only device can be trained to perform multiple tasks or combinations of tasks that involve arbitrarily interleaved speech and text. For example, the system can obtain a sequence of input tokens that include text tokens, audio tokens, or both text and audio tokens, selected from a vocabulary of text tokens and audio tokens. Paragraph 53 describes all of the claimed speech/text tasks.
With regard to Claim 13, Rub describes “training, by the processor, the unified decoder-only model to perform the third task, wherein the training is performed based on training data that includes the third special token and a text token from the unified vocabulary.”
Paragraph 29 describes that the decoder-only device can be trained to perform multiple tasks or combinations of tasks that involve arbitrarily interleaved speech and text. For example, the system can obtain a sequence of input tokens that include text tokens, audio tokens, or both text and audio tokens, selected from a vocabulary of text tokens and audio tokens. Paragraph 53 describes all of the claimed speech/text tasks.
With regard to Claim 14, Rub describes “training, by the processor, the unified decoder-only model to perform the fourth task, wherein the training is performed based on training data that includes the fourth special token and a speech token from the unified vocabulary.”
Paragraph 29 describes that the decoder-only device can be trained to perform multiple tasks or combinations of tasks that involve arbitrarily interleaved speech and text. For example, the system can obtain a sequence of input tokens that include text tokens, audio tokens, or both text and audio tokens, selected from a vocabulary of text tokens and audio tokens. Paragraph 53 describes all of the claimed speech/text tasks.
With respect to Claims 15-20, device Claim 15 and method Claim 1 are related as a device programmed to perform the same method, with each claimed device function corresponding to each claimed method step. Accordingly, Claims 15-20 are similarly rejected under the same rationale as applied above with respect to Claims 1-7.
Conclusion
7. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Pat. Ap. Pub. No. 20220100676 (Yan et al.) also describes a decoder only architecture.
8. Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD TRACY whose telephone number is (571)272-8332. The examiner can normally be reached Monday-Friday 9 AM- 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EDWARD TRACY JR./Examiner, Art Unit 2656
Prosecution Insights