DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination
2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant’s submission filed 21 May 2026 [hereinafter Response] has been entered, where:
Claims 1, 2, 4, 5, 7, 8, 9, 11, 12, 14, 15, 16, 18, and 19 have been amended.
Claims 3, 6, 10, 13, 17, and 20 have been cancelled.
Claims 1, 2, 4, 5, 7-9, 11, 12, 14-16, 18, and 19 are pending.
Claims 1, 2, 4, 5, 7-9, 11, 12, 14-16, 18, and 19 are rejected.
Claim Rejections – 35 U.S.C. § 112
3. The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
4. The rejection to claims 1, 4-9, 11-16 and 18-20 under 35 U.S.C. § 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention is WITHDRAWN in view of Applicant’s amendments to the claims.
Claim Rejections - 35 U.S.C. § 101
5. 35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
6. Claims 1, 2, 4, 5, 7-9, 11, 12, 14-16, 18, and 19 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a computer-implemented method, which is a process, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites the limitations of “generating, using a first machine learning model, candidate output sequences, for an input sequence;” “identifying, within the candidate output sequences, spans of tokens corresponding to specialized entities, each span being defined by a beginning token and an ending token,” “applying a first scoring methodology corresponding to an internal language model against the candidate output sequences to obtain a first plurality of prediction scores,” “applying a second scoring methodology corresponding to an external language model trained for the specialized entities against the specialized entities to obtain a second plurality of prediction scores;” “modifying the first plurality of prediction scores for the specialized entities with the second plurality of prediction scores” and “generating output sequences based upon the modified first plurality of prediction scores for the candidate output sequences.” The activities of “generating,” “identifying,” “applying,” “modifying,” and “generating output sequences” are limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)).
The claim recites more details or specifics to the abstract idea of “applying a first scoring methodology,” “wherein prediction scores correspond to a probability of a sequence of tokens representing at least a portion of the candidate output sequences,” and accordingly, is merely more specific to the abstract idea.
The claim also recites more details or specifics to the abstract idea of “modifying . . . scores,” “by removing a contribution corresponding to the first scoring methodology from the first plurality of prediction scores for the specialized entities,” and “adding a contribution corresponding to the second scoring methodology from the second plurality of prediction scores, such that the modified first plurality of prediction scores corresponds to a density ratio of the second scoring methodology relative to the first scoring methodology,” and accordingly, are merely more specific to the abstract idea. Thus, claim 1 recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include a “computer-implemented method,” which is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. The claim also recites a “first machine learning model,” “an internal language model,” and “an external language model,” which are recited at a high level of generality, and accordingly are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. Therefore, claim 1 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements include a “computer-implemented method,” which is a generic computer component used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites a “first machine learning model,” “an internal language model,” and “an external language model,” which are recited at a high level of generality, and accordingly are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not amount to significantly more than the abstract idea. Therefore, claim 1 is subject-matter ineligible.
Claim 8 recites a computer program product, which is a product, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites the limitations of “generating, using a first machine learning model, candidate output sequences, for an input sequence;” “identifying, within the candidate output sequences, spans of tokens corresponding to specialized entities, each span being defined by a beginning token and an ending token,” “applying a first scoring methodology corresponding to an internal language model against the candidate output sequences to obtain a first plurality of prediction scores,” “applying a second scoring methodology corresponding to an external language model trained for the specialized entities against the specialized entities to obtain a second plurality of prediction scores;” “modifying the first plurality of prediction scores for the specialized entities with the second plurality of prediction scores” and “generating output sequences based upon the modified first plurality of prediction scores for the candidate output sequences.” The activities of “generating,” “identifying,” “applying,” “modifying,” and “generating output sequences” are limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)).
The claim recites more details or specifics to the abstract idea of “applying a first scoring methodology,” “wherein prediction scores correspond to a probability of a sequence of tokens representing at least a portion of the candidate output sequences,” and accordingly, is merely more specific to the abstract idea.
The claim also recites more details or specifics to the abstract idea of “modifying . . . scores,” “by removing a contribution corresponding to the first scoring methodology from the first plurality of prediction scores for the specialized entities,” and “adding a contribution corresponding to the second scoring methodology from the second plurality of prediction scores, such that the modified first plurality of prediction scores corresponds to a density ratio of the second scoring methodology relative to the first scoring methodology,” and accordingly, are merely more specific to the abstract idea. Thus, claim 8 recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include a exception include a “non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to,” and a “computing device,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. The claim also recites a “first machine learning model,” “an internal language model,” and “an external language model,” which are recited at a high level of generality, and accordingly are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. Therefore, claim 8 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements include exception include a “non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to,” and a “computing device,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites a “first machine learning model,” “an internal language model,” and “an external language model,” which are recited at a high level of generality, and accordingly are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not amount to significantly more than the abstract idea. Therefore, claim 8 is subject-matter ineligible.
Claim 15 recites a computer system, which is a product, and thus one of the statutory categories of patentable subject matter. (35 U.S.C. § 101).
However, under Step 2A Prong One, the claim recites the limitations of “generate, using a first machine learning model, candidate output sequences, for an input sequence;” “identify, within the candidate output sequences, spans of tokens corresponding to specialized entities, each span being defined by a beginning token and an ending token,” “apply a first scoring methodology corresponding to an internal language model against the candidate output sequences to obtain a first plurality of prediction scores correspond to a probability of a sequence of tokens representing at least a portion of the candidate output sequences,” “apply a second scoring methodology corresponding to an external language model trained for the specialized entities against the specialized entities to obtain a second plurality of prediction scores;” “modify the first plurality of prediction scores for the specialized entities with the second plurality of prediction scores” and “generate output sequences based upon the modified first plurality of prediction scores for the candidate output sequences.” The activities of “generate,” “identify,” “apply,” “modify,” and “generate output sequences” are limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)).
The claim recites more details or specifics to the abstract idea of “apply a first scoring methodology,” “wherein prediction scores correspond to a probability of a sequence of tokens representing at least a portion of the candidate output sequences,” and accordingly, is merely more specific to the abstract idea.
The claim also recites more details or specifics to the abstract idea of “modify . . . scores,” “by removing a contribution corresponding to the first scoring methodology from the first plurality of prediction scores for the specialized entities,” and “adding a contribution corresponding to the second scoring methodology from the second plurality of prediction scores, such that the modified first plurality of prediction scores corresponds to a density ratio of the second scoring methodology relative to the first scoring methodology,” and accordingly, are merely more specific to the abstract idea. Thus, claim 15 recites an abstract idea.
Under Step 2A Prong Two, the claim as a whole is not integrated into a practical application, because the additional elements recited in the claim beyond the identified judicial exception include a “memory” and “a processor,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application.
The claim also recites a “first machine learning model,” “an internal language model,” and “an external language model,” which are recited at a high level of generality, and accordingly are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. Therefore, claim 15 is directed to the abstract idea.
Finally, under Step 2B, the additional elements, taken alone or in combination, do not represent significantly more than the abstract idea itself. The additional elements include a “memory” and “a processor,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites a “first machine learning model,” “an internal language model,” and “an external language model,” which are recited at a high level of generality, and accordingly are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that do not amount to significantly more than the abstract idea. Therefore, claim 15 is subject-matter ineligible.
Claim 2 depends from claim 1. Claim 9 depends from claim 8. Claim 16 depends from claim 15. These claims provide more details or specifics to the additional element of the “first machine learning model” that include “a sequence-to-sequence model configured to process the input sequence to generate an output sequence,” and accordingly, is merely more specific to the additional element. Therefore, claims 2, 9, and 16 are subject-matter ineligible.
Claim 4 depends from claim 1. Claim 11 depends from claim 8. Claim 18 depends from claim 15. The claims recite more details or specifics to the abstract idea of “applying, using the one or more machine learning models, a first scoring methodology,” where “the first scoring methodology is based upon, at least in part, a first probability distribution associated with an internal language model,” and accordingly, are merely more specific to the abstract idea. Therefore, claims 4, 11, and 18 are subject-matter ineligible.
Claim 5 depends directly or indirectly from claim 1. Claim 12 depends directly or indirectly from claim 8. Claim 19 depends directly or indirectly from claim 15. The claims recite more details or specifics to the abstract idea of “applying a second scoring methodology,” where “wherein the second scoring methodology is based upon, at least in part, a second probability distribution associated with the external language model,” and accordingly, are merely more specific to the abstract idea. Therefore, claims 5, 12, and 19 are subject-matter ineligible.
Claim 7 depends from claim 1. Claim 14 depends from claim 8. The claim recites more details or specifics of the abstract idea of “identifying the spans of tokens corresponding to specialized entities” by “tagging a plurality of specialized entities, thus defining one or more tagged portions,” and accordingly, are merely more specific to the abstract idea. Therefore claims 7 and 14 are subject-matter ineligible.
Response to Arguments
7. Examiner has fully considered Applicant’s arguments and responds below accordingly.
Claim Rejections – 35 U.S.C. § 101
8. Under Step 2A Prong One, Applicant submits that “Claim 1, as amended, does not recite a judicial exception. More specifically, amended claim 1 does not recite mental processes because these steps cannot practically be performed in the human mind. The mental processes are "concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions." MPEP § 2106.04(a)(2) III. Claim 1 is directed to a sequence-to-sequence processing system that improves recognition of specialized entities by identifying token spans corresponding to specialized entities in candidate output sequences and, for those spans, replacing internal language-model scoring with entity-specific external language-model scoring using a contextual density-ratio approach, including removing internal language-model contributions and adding external language-model contributions to produce modified prediction scores that are used to generate output sequences. This selective, token-span-based density-ratio replacement during decoding changes how the system computes prediction scores for specialized entities relative to internal-only scoring and addresses the difficulty of exploiting independently trained language models in end-to-end systems, resulting in improved output accuracy for specialized entities in a manner that cannot practically be performed in the human mind. Specification [0053]-[0059], [0061].” (Response at p. 9).
Examiner’s Response:
Examiner respectfully disagrees, because the rejection identifies the abstract idea (that is, the judicial exception) by referring to what is recited (i.e., set forth or described) in the claim and explain why it is considered an exception. (MPEP § 2106.07(a)).
The rejection sets out, inter alia, the activities of “generating,” “identifying,” “applying,” and “modifying” are limitations that can practically be performed in the human mind, including, for example, observations, evaluations, judgments, and opinions, and accordingly, are a mental process, (MPEP § 2106.04(a)(2) sub III), which is one of the groupings of abstract ideas. (MPEP § 2106.04(a)(2)).
Accordingly, the claims recite an abstract idea, as set out above in detail.
9. Under Step 2A Prong Two, Applicant submits that “[a]s amended, claim 1 improves the functioning of a computer by modifying how a sequence-to-sequence processing system generates outputs during decoding, rather than merely using a computer as a tool to manipulate or combine information. Traditional sequence-to-sequence systems rely primarily on internal language-model scoring, which can cause misrecognition or underweighting of specialized entities whose probability distributions differ from those learned by the internal model. The claimed invention improves this operation by identifying token spans corresponding to specialized entities and selectively removing internal language-model scoring contributions for those spans while adding contributions from an external language model trained for the specialized entities. This changes the internal operation of the sequence-to-sequence system by altering how prediction scores are computed and accumulated during decoding in a manner that enables accurate processing of specialized entities that the system could not previously handle using internal language-model scoring alone.
Furthermore, software-based improvements can be non-abstract when they improve the functioning of a computer. See Ex parte Desjardins, Appeal No. 2024-000567, pp. 6-7 (PTAB ARP, Sept. 26, 2025). Additionally, once a claim is found to integrate any judicial exception into a practical application, the§ 101 inquiry ends. Id. at pp. 7-8. Thus, claim 1 is patent eligible. The present invention enables the system to generate outputs that accurately reflect specialized entities without requiring retraining, user annotation, or post-hoc correction, which constitutes a concrete improvement to the computer's sequence-to-sequence processing functionality itself rather than an abstract idea. See Specification [0053]-[0059], [0061].” (Response at pp. 10-11).
Examiner’s Response:
Examiner respectfully disagrees because the rejection identifies any additional elements recited in the claim beyond the identified abstract idea, and evaluates the integration of the judicial exception into a practical application by explaining that the claim as a whole, looking at the additional elements individually and in combination, does not integrate the judicial exception into a practical application using the considerations set forth in MPEP §§ 2106.04(d), 2106.05(a)- (c) and (e)- (h). (MPEP § 2106.07(a)).
“Integration” may be based on the improvements in the functioning of a computer or an improvement to any other technology or technical field. (MPEP § 2106.04(d)(1)). The evaluation requires, [i]n sum, that (1) the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. Next, (2) if the specification sets forth such an improvement, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement.
By way of example to Desjardins, the MPEP provides under Step 2A Prong Two that “the [Desjardins] specification identified improvements as to how the machine learning model itself operates, including training a machine learning model to learn new tasks while protecting knowledge about previous tasks to overcome the problem of ‘catastrophic forgetting’ encountered in continual learning systems. Importantly, the [appeals review panel (ARP)] evaluated the claims as a whole in discerning at least the limitation ‘adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task’ reflected the improvement disclosed in the specification. Accordingly, the claims as a whole integrated what would otherwise be a judicial exception instead into a practical application at Step 2A Prong Two, and therefore the claims were deemed to be outside any specific, enumerated judicial exception (Step 2A: NO).” (MPEP § 2106.04(d) sub III; see “Advance Notice of Change to the MPEP in light of Ex Parte Desjardins” (05 December 2025) at p. 2)).
The claims recite additional elements including, inter alia, a “computer-implemented method,” which is generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not serve to integrate the abstract idea into a practical application. The claim also recites a “machine learning model,” an “internal language model,” and an “external language model”, which are recited at a high level of generality, and accordingly are also a generic computer components that used to implement the abstract idea, (MPEP § 2106.05(f)). The specification does not require specialized computer components. (Specification ¶¶ 0017-19 (“any suitable computer usable or computer readable medium (or media) may be utilized”)). Thus, the generic computer components are used in the conventional and intended manner.
Also, Applicant submits the instant claims are directed to the improvement of the functioning of a computer by modifying how a sequence-to-sequence processing system generates outputs during decoding, rather than merely using a computer as a tool to manipulate or combine information. Rather, the improvement appears to be to the abstract idea, which remains an abstract idea.
The claims, taken as a whole, does not integrate the abstract idea into a practical application of the abstract idea because it does not apply, rely on, or use the abstract idea in a manner that imposes a meaningful limit on the abstract idea.
Accordingly, the claims are directed to an abstract idea, as set out above in detail.
10. Under Step 2B, “the elements of claim 1, individually and in combination, provide improvements to the functioning of a computer. As summarized in MPEP § 2106.05(a), the Enfish line of cases identifies claims as patent-eligible when they are directed to a specific improvement in how a computer stores, retrieves, or processes data, rather than to an abstract idea implemented on a generic computer. In Enfish, eligibility turned on the use of a non-conventional data structure that altered the way a computer operated, resulting in improved efficiency and performance. Independent claim 1 similarly recites a non-conventional, computer-specific mechanism that improves how a sequence-to-sequence processing system operates by modifying decoding-stage prediction scoring using an external, entity-specific language model. Rather than relying solely on an internal language model to score candidate output sequences, the claimed method identifies spans of tokens corresponding to specialized entities and replaces internal language-model contributions for those token spans with contributions from an external language model trained for the specialized entities. This selective removal and addition alters the internal decoding behavior of the system by changing how token probabilities are computed and accumulated, enabling the computer to correctly generate outputs that reflect specialized entities that would otherwise be mischaracterized or underweighted by internal language-model scoring alone. As a result, the claim is directed to a concrete improvement in the functioning of sequence-to-sequence processing systems themselves, rather than to the abstract idea of scoring or modifying predictions implemented on a generic computer.
Consistent with Enfish as explained in the MPEP, claim 1 is not directed to an abstract idea because it does not merely organize or present information using a computer, but rather improves the functioning of a sequence-to-sequence processing system itself The claimed system modifies how the computer processes input sequences by identifying token spans corresponding to specialized entities and replacing internal language-model scoring with entity-specific external language-model scoring during decoding, thereby enabling the computer to generate outputs that accurately reflect specialized entities that could not be produced using internal language-model scoring alone.
This technical architecture changes the internal operation of the sequence-to-sequence processing system by enabling the computer to condition decoding behavior on identification of specialized entity token spans and to selectively replace internal language-model scoring with entity-specific external language-model scoring for those spans, which conventional internal-only scoring systems cannot perform. As a result, the claim integrates any alleged abstract concept into a practical application by effecting a concrete improvement in the way the computer performs sequence-to-sequence decoding, mirroring the Enfish example's emphasis on structural and operational changes to computer behavior, rather than merely using the computer as a general-purpose tool.” (Response at pp. 11-12).
Examiner’s Response:
Examiner respectfully disagrees because the rejection explains why the additional elements, taken individually and in combination, do not result in the claim, as a whole, amounting to significantly more than the identified judicial exception. (MPEP § 2106.07(a)).
The Office guidance sets out that Step 2B includes a consideration of whether the additional element (or combination of elements) is a well-understood, routine, conventional activity. A claim may be found to lack significantly more (and thus be ineligible) based on one or more of these judicial considerations.
However, the rejections set out above set out the use of generic computer components to implement the abstract idea. Specifically, he additional elements include a “computer-implemented method,” and a “computing device,” which are generic computer components used to implement the abstract idea, (MPEP § 2106.05(f)), that does not amount to significantly more than the abstract idea. The claim also recites “one or more machine learning models,” a “first machine learning model of the plurality of machine learning models,” and a “second machine learning model of the plurality of machine learning models, which are recited at a high level of generality, and accordingly is also a generic computer component that is used to implement the abstract idea, (MPEP § 2106.05(f)), that does not that does not amount to significantly more than the abstract idea. The specification does not require specialized computer components. (Specification ¶¶ 0017-19 (“any suitable computer usable or computer readable medium (or media) may be utilized”)). Thus, the generic computer components are used in the conventional and intended manner.
Accordingly, the claims are subject-matter ineligible, as set out above in detail.
Claim Rejections – 35 U.S.C. § 102
11. Applicant submits that “Claims 1-8, 11-19, and 21-29 stand rejected under 35 U.S.C. § 102 as allegedly being unpatentable over Zhao et al., US Pub. No. 2020/0357388 Al (hereinafter "Zhao"). Office Action at 9. Claim 1, as amended, recites:
1. A computer-implemented method:
generating, using a first machine learning model, candidate output sequences, for an input sequence;
identifying, within the candidate output sequences, spans of tokens corresponding to specialized entities , each span being defined by a beginning token and an ending token;
applying a first scoring methodology corresponding to an internal language model against the candidate output sequences to obtain a first plurality of prediction scores, wherein prediction scores correspond to a probability of a sequence of tokens representing at least a portion of the candidate output sequences;
applying a second scoring methodology corresponding to an external language model trained for the specialized entities against the specialized entities to obtain a second plurality of prediction scores;
modifying the first plurality of prediction scores for the specialized entities with the second plurality of prediction scores by:
removing a contribution corresponding to the first scoring methodology from the first plurality of prediction scores for the specialized entities; and
adding a contribution corresponding to the second scoring methodology from the second plurality of prediction scores, such that the modified first plurality of prediction scores corresponds to a density ratio of the second scoring methodology relative to the first scoring methodology; and
generating output sequences based upon the modified first plurality of prediction scores for the candidate output sequences.
[(claim 1 (emphasis added by Examiner)]
Zhao generally describes processing input data using a speech recognition model and
combining internal recognition scores with contextual scores during decoding. The Office relies
on paragraph [0051] of Zhao, which discloses that a score combiner combines speech recognition
scores with context scores to produce combined scores, as allegedly anticipating the limitation of
modifying prediction scores based on multiple scoring methodologies. However, nothing in the cited portions of Zhao discloses that such modifying comprises removing or canceling contributions of an internal language model for specialized-entity tokens and replacing those contributions with entity-specific external language-model scoring using a density-ratio approach, nor that tokens are scored independently of whether they are within a named entity context. During the Interview, Applicant understood Examiner Smith to agree with this distinction. Accordingly, amended claim 1 is patentable over Zhao. Support for this amendment is found in the originally filed specification at least at paragraph [0059].” (Response at p. 13).
Examiner Response:
Examiner finds Applicant’s amendments and arguments persuasive, and accordingly WITHDRAWS the rejection under Section 102.
Conclusion
12. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
(McDermott et al., "A Density Ratio Approach to Language Model Fusion in End-to-End Automatic Speech Recognition," arXiv (2020)) teaches a density ratio approach to integrating external Language Models (LMs) into end-to-end models for Automatic Speech Recognition (ASR). Applied to a Recurrent Neural Network Transducer (RNN-T) ASR model trained on a given domain, a matched in-domain RNN-LM, and a target domain RNN-LM, the proposed method uses Bayes’ Rule to define RNN-T posteriors for the target domain, in a manner directly analogous to the classic hybrid model for ASR based on Deep Neural Networks (DNNs) or LSTMs in the Hidden Markov Model (HMM) framework.
(Zeineldeen et al., "Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Models," arXiv (2021)) teaches an attention-based encoder-decoder (AED) models learn an implicit internal language model (ILM) from the training transcriptions. The integration with an external LM trained on much more unpaired text usually leads to better performance. A Bayesian interpretation as in the hybrid autoregressive transducer (HAT) suggests dividing by the prior of the discriminative acoustic model, which corresponds to this implicit LM, similarly as in the hybrid hidden Markov model approach. We also investigate other methods to suppress the ILM mainly by decreasing the capacity of the AED model, limiting the label context, and also by training the AED model together with a pre-existing LM.
(US Patent 10388274 to Hoffmeister et al.) teaches As part of the language modeling (or in other phases of the ASR processing) the speech recognition engine 258 may, to save computational resources, prune and discard low recognition score states or paths that have little likelihood of corresponding to the spoken utterance, either due to low recognition score pursuant to the language model, or for other reasons. Such pruned paths are considered inactive.
(US Published Application 20220310062 to Sainath et al.) teaches Automated speech recognition (ASR) systems have evolved from multiple models where each model had a dedicated purpose to integrated models where a single neural network is used to directly map an audio waveform (i.e., input sequence) to an output sentence (i.e., output sequence). This integration has resulted in a sequence-to-sequence approach, which generates a sequence of words (or graphemes) when given a sequence of audio features. With an integrated structure, all components of a model may be trained jointly as a single end-to-end (E2E) neural network.
13. Any inquiry concerning this communication or earlier communications from the Examiner should be directed to KEVIN L. SMITH whose telephone number is (571) 272-5964. Normally, the Examiner is available on Monday-Thursday 0730-1730.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, KAKALI CHAKI can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.L.S./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights