Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAIL ACTION
Priority
This application claims priority to U.S provisional Patent Application No. 63664423, filed on 6/26/2024 and is hereby incorporated by references.
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 4/17/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claimed invention is directed to non-statutory subject matter because the claim(s) as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea. As summarized in the 2019 Revised Patent Subject Matter Eligibility Guidance, examiners must perform a Two-Part Analysis for Judicial Exceptions.
Step 1
In Step 1, it must be determined whether the claimed invention is directed to a process, machine, manufacture or composition of matter. The instant invention encompasses three sets of claims: a computer-implemented method in claims 1-10 (i.e., a process), one or more non-transitory computer-readable media in claims 11-19 (i.e., a manufacture) and a system in claim 20 (i.e., a manufacture). All claims are directed to one of the four statutory categories and meet the requirements of step 1.
Step 2A
Prong One
The claimed invention is directed to an abstract idea without significant more. The instant invention is broadly directed to “training a speech language model to generate a text caption for the input audio associated with meta information”. Claim 1 recites the following (with emphasis added):
Claim 1: A computer-implemented method for training a speech language model, the method comprising:
generating a set of text captions based on meta information associated with a first set of audio that includes speech, wherein the meta information specifies at least one of a speaking style or speaker information associated with the speech included in the first set of audio; and
performing, using the first set of audio and the set of text captions, one or more first operations to train a speech language model to generate a text caption for first input audio that includes speech.
The bold portions of claim 1 encompass the abstract idea, which is also encompassed by the dependent claims 2-10, and substantially also encompassed by claims 11-19 and 20.
Claims 1, 11 and 20 recite the steps to train a speech language model to generate a text caption for the input audio associated with meta information. These limitations, when given their broadest reasonable interpretation, are directed to certain performing of organizing human activity and mental processes, which is abstract idea.
Prong Two
This judicial exception is not integrated into a practical application because mere instruction to implement on computers (i.e. storage medium or processors in claim 11 and 20) or a computer model (speech language model here in claim 1), or merely using computers as a tool to perform the abstract idea, adding insignificant extra solution activity, and/or generally linking the use of the abstract idea to a technological environment for field of use is not considered integration into a practical application. Claim 1 recites using audio data and text captions generated based on meta information with the audio data to train a speech model language model. Using a text data of an audio file to a train a speech language model is a generic feature of language model process, which does not represent a technological improvement. The using of the computer and language model process does not add improvement to the functioning of a computer or to any other technology field, which failed to enable the abstract idea to integrate into a practical application. The claims are drafted in a result-oriented fashion, without the requisite specificity needed to provide a nonabstract technological solution. The computing system and language model process are directed to the components of a system amount to merely field of use type limitations and/or extra solution activity to implement the abstract idea as presented.
Step 2B
Step 2B in the analysis requires us to determine whether the claims do significantly more than simply describe that abstract method. Mayo, 132 S. Ct. at 1297. We must examine the limitations of the claims to determine whether the claims contain an "inventive concept" to "transform" the claimed abstract idea into patent-eligible subject matter. Alice, 134 S. Ct. at 2357 (quoting Mayo, 132 S. Ct. at 1294, 1298). The transformation of an abstract idea into patent-eligible subject matter "requires 'more than simply stat[ing] the [abstract idea] while adding the words 'apply it."' Id. (quoting Mayo, 132 S. Ct. at 1294) (alterations in original). "A claim that recites an abstract idea must include 'additional features' to ensure 'that the [claim] is more than a drafting effort designed to monopolize the [abstract idea].'" Id. (quoting Mayo, 132 S. Ct. at 1297) (alterations in original). Those "additional features" must be more than "well-understood, routine, conventional activity." Mayo, 132 S. Ct. at 1298.
The present claims include the additional elements other than the abstract idea which include a processor, storage medium, language model (in claim 1, 11 and 20). These additional elements are merely conventional computer and computer model. Any potentially technical aspects of the claims are well-known generic computer components performing conventional functions (e.g., a processor performing a mental process). The present claims have been analyzed both individually and in combination and, the instant claims do not provide any improvement of the functioning of the computer or improvement to computer technology or any other technical field. There do not appear to be any meaningful limitations other than those that are well-understood, routine and conventional in the field. Thus, the present claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claims 1-10 are not patent eligible.
Claims 11-19 and 20 similar limitations of claims 1-10, thus are abstract idea and not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7, 11-17 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al (US 20240347064 A1) in view of MALHOTRA et al (US 20200020319 A1).
Regarding claim 1, Li discloses a computer-implemented method for training a speech language model [e.g. FIG. 2, 27 and 37-38; training speech audio transcription machine model], the method comprising:
generating a set of text captions [e.g. FIG. 38; speech to text process] based on meta information [e.g. FIG. 19 and 38; speaker ID] associated with a first set of audio that includes speech [e.g. speech audio signal 3802], wherein the meta information specifies at least one of a speaking style or speaker information associated with the speech included in the first set of audio [e.g. speaker identification]; and
performing, using the set of text captions [transcripts] associated with the speech included in the first set of audio, one or more first operations [training the large language model] to train a speech language model [e.g. FIG 37-38 and 42; 3808] to generate a text caption for first input audio that includes speech [e.g. enhanced transcript].
It is noted that Li differs to the present invention in that Li fails to explicitly disclose using the audio data to train the large language model.
However, MALHOTRA teaches the well-known concept of a computer-implemented method for retraining a speech language model by using the first set of audio and the set of text captions [e.g. FIG. 1-2; [0027]; training a general language model using corrected text with using audio during the training].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify a speech-to-text processing system disclosed by Li to exploit the well-known training language model technique taught by MALHOTRA as above, in order to provide improved ability to accurately recognize text [See MALHOTRA; [0008]].
Regarding claim 2, Li and MALHOTRA further disclose performing, using a second set of audio and expected text output corresponding to the second set of audio [e.g. Li: FIG. 37-38; audio/text segmentation with identified speaker ID; MALHOTRA: FIG. 2-3], one or more second operations to train the speech language model to generate a text response to second input audio that includes speech [e.g. Li: FIG. 27-28 and 37-39; question-answering text to text transformation tasks; MALHOTRA: FIG. 2-3].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify a speech-to-text processing system disclosed by Li to exploit the well-known training language model technique taught by MALHOTRA as above, in order to provide improved ability to accurately recognize text [See MALHOTRA; [0008]].
Regarding claim 3, Li and MALHOTRA further disclose the second set of audio includes one or more questions, and the expected text output includes one or more answers to the one or more questions [e.g. Li: FIG. 27-28; [0641]; question-answering text to text transformation tasks].
Regarding claim 4, Li and MALHOTRA further disclose the meta information specifies at least one of a pitch, a volume, a speaking speed, a gender, or a spoken content associated with the speech included in the first set of audio [e.g. Li: FIG. 22 and 27-28; [0467]; the vocal characteristics of a particular person despite variations in the speech of that particular person that may arise under differing conditions, such as speech volume, speech speed and/or pitch associated with differing emotional states].
Regarding claim 5, Li and MALHOTRA further disclose generating the set of text captions comprises applying one or more templates to the meta information [Li: FIG. 22 and 37-38; [0011]; perform a speaker diarization technique to identify a set of likely speaker changes in the speech audio; sentence pauses, and the set of likely speaker changes; use at least an acoustic model with each data segment of the multiple data segments to identify likely speech sounds in the speech audio].
Regarding claim 6, Li and MALHOTRA further disclose generating the set of text captions comprises processing one or more sentences that include the meta information and one or more text prompts [e.g. speaker diarization prompt] using a trained language model that outputs the set of text captions [Li: FIG. 22 and 37-38; [0011-0019]; sentence pauses maybe have speaker changes].
Regarding claim 7, Li and MALHOTRA further disclose the one or more text prompts instruct the trained language model to generate the set of text captions to (i) reflect spoken content of the speech included in the first set of audio [e.g. Li: FIG. 27-28 and 37-38: transcription], and (ii) describe one or more attributes of the first set of audio that are specified by the meta information [Li: e.g. speaker identification].
Regarding claim 11-14, this is a non-transitory computer-readable storage medium that includes same limitation as in claim 1-4 above, the rejection of which are incorporated herein.
Regarding claim 15, this is a non-transitory computer-readable storage medium that includes same limitation as in claim 5 and 6 together above, the rejection of which are incorporated herein.
Regarding claim 16, this is a non-transitory computer-readable storage medium that includes same limitation as in claim 5 and 7 together above, the rejection of which are incorporated herein.
Regarding claim 17, Li and MALHOTRA further disclose the trained language model comprises a trained large language model (LLM) [Li: FIG. 27-28 and 37-38; MALHOTRA: FIG. 1-3; a previously-trained general language model].
Regarding claim 20, this is an apparatus that includes same limitation as in claim 1 above, the rejection of which are incorporated herein.
Claim(s) 8-10 and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al (US 20240347064 A1) in view of MALHOTRA et al (US 20200020319 A1) and TUNG et al (US 20250356843 A1).
Regarding claim 8, Li and MALHOTRA further disclose an encoder and decoder, that encodes the first input audio to generate a representation of the first input audio [e.g. Li: FIG. 15-16, 23-25, 27-28 and 37-38; [645]; a transformer encoder to generate a sequence embedding]; a decoder that decodes the representation of the first input audio to generate a text transcription [e.g. Li: a transformer decoder to convert the sequence embedding to a second sequence of tokens]; and a language model that processes the text transcription, the one or more features, and a text prompt to generate an output text [e.g. Li: FIG. 23-25 and 37-38], Li and MALHOTRA fail to explicitly disclose an adapter.
However, TUNG teaches the well-known concept of an encoder that encodes the first input audio to generate a representation of the first input audio[e.g. FIG. 2-3; encoder]; a decoder that decodes the representation of the first input audio to generate a text transcription [e.g. decoder; generate text]; and an adapter that decodes the representation of the first input audio to generate one or more features [e.g. FIG. 2-3; an adapter 104; representations based on the acoustic features];
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify a speech-to-text processing system disclosed by Li to exploit the well-known training language model technique taught by MALHOTRA and the well-known concept of employing parameter-efficient low-rank adaptation for fine-tuning the machine learning model by TUNG as above, in order to provide improved ability to accurately recognize text [See MALHOTRA; [0008]] and automatic speech recognition with performance improvement and parameter efficiency [See TUNG; abstract].
Regarding claim 9, Li, MALHOTRA and TUNG further disclose the one or more first operations update one or more parameters of the adapter [e.g. FIG. Li: 37-38; parameters of large language model].
Regarding claim 10, Li, MALHOTRA and TUNG further disclose the language model comprises one or more LoRA (Low-Rank Adaptation of Large Language Models) adapters [e.g. TUNG: FIG. 1 and 4; fine-tune the machine learning model by employing parameter-efficient low-rank adaptation for speech recognition].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify a speech-to-text processing system disclosed by Li to exploit the well-known training language model technique taught by MALHOTRA and the well-known concept of employing parameter-efficient low-rank adaptation for fine-tuning the machine learning model by TUNG as above, in order to provide improved ability to accurately recognize text [See MALHOTRA; [0008]] and automatic speech recognition with performance improvement and parameter efficiency [See TUNG; abstract].
Regarding claim 18-19, this is a non-transitory computer-readable storage medium that includes same limitation as in claim 8-9 above, the rejection of which are incorporated herein.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
RUTOWSKI et al (US 20240087752 A1).
SARANA et al (US 20240355328 A1).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHUBING REN whose telephone number is (571)272-2788. The examiner can normally be reached Monday-Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHUBING REN/ Primary Examiner, Art Unit 2658
