DETAILED ACTION
This action is in response to the amendment filed on 03/30/2026.
Although the originally filed specification is directed to a plurality of language models which are each trained to process a different language, the claim language does not require this limited interpretation of the recited language models. To further prosecution, the claim language could be amended so that each of the language models process a different language.
Response to Amendment
Applicant’s amendment filed on 03/30/2026 has been entered. Claims 1 and 15 have been amended. No claims have been canceled. No claims have been added. Claims 1- 20 are still pending in this application, with claims 1 and 15 being independent.
Allowable Subject Matter
Claim 4 (with dependent claim 5) is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The prior art fails to teach or suggest in reasonable combination the limitations recited in claim 4:
A multilingual speech recognition system, comprising: a ring buffer configured to store at least one audio stream spoken by a user; a memory configured for storage of encoded instructions executable by at least one processor; and the at least one processor configurable by the encoded instructions to execute: a plurality of language models connected in parallel to the ring buffer, wherein each language model is associated with a language and trained according to a plurality of words and phonemes associated with the language, and wherein each language model is configured to :receive the audio stream from the ring buffer ;detect a candidate intent of the spoken audio stream by associating the audio stream with a sequence of phonemes corresponding to the trained language, wherein each candidate intent is associated with a selection frequency indicative of a number of times the candidate intent has been selected as a final intent; and record the candidate intent in an intent buffer; and an arbitrator configured to: attempt a selection of a final intent from the one or more recorded candidate intents, wherein the arbitrator is configured to set as a default language the language associated with the single candidate intent.
Claim 16 (with dependent claims 17 and 18) is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The prior art fails to teach or suggest in reasonable combination the limitations recited in claim 4:
A computer-assisted method for multilingual speech recognition, the method comprising: receiving, via an input device, an audio stream spoken by a user; storing, via a ring buffer, the audio stream; detecting, via each of a plurality of language models configured for parallel operation, each language model associated with a language and trained according to a plurality of words and phonemes associated with the language, one or more candidate intents of the user by associating the audio stream with a sequence of phonemes associated with the language, each candidate intent associated with a selection frequency indicative of a number of times the candidate intent has been selected as a final intent; recording, via an intent buffer corresponding to each language model, each detected candidate intent; and attempting to select or infer, via an arbitrator, a final intent from the one or more recorded candidate intents stored to the intent buffers, wherein the one or more recorded candidate intents consist of a single candidate intent; and wherein attempting to select or infer, via an arbitrator, a final intent from the one or more recorded candidate intents stored to the intent buffers includes selecting as the final intent, via the arbitrator, the single candidate intent and setting as a default language the language associated with the single candidate intent.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 – 3, 6 – 9, 11, 12, 15, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Last et al. (US 2026/0011328) (“Last”) in view of Garcia et al. (US 2021/0027780) (“Garcia”) , and further in view of Kim (US 2023/0335120) and further in view of Kawamura et al. (US 2014/0088967) (“Kawamura”).
For claim 1, Last discloses a multilingual speech recognition system (service system and nodes, Fig.5, 104, 404 and 506; Abstract), comprising components to execute: a plurality of language models (nodes comprising an acoustic model, word recognition component, language model and intent recognition component, Fig.6, 104, 602, 604, 614, 616 and 618; [0048] [0060][0061] ) connected in parallel (Fig.5, 104a-104c; [0047]), wherein each language model is associated with a language (e.g. English, [0049 – 0052] [0062 – 0065] [0096]) and trained according to a plurality of words and phonemes associated with the language ([0049] [0050] [0059 – 0066]), and wherein each language model is configured to: receive audio spoken by a user (Fig.2, S222, S224 and Fig.6, 102; [0060] [0073] [0074]); detect a candidate intent of the audio by associating the audio with a sequence of phonemes corresponding to the trained language (Fig.2, S224 and Fig.6, 602, 612, 614, 618, 616 and 604; [0060] [0061] [0066] [0073] [0074]); and an arbitrator (controller, Fig.5, 110) configured to: attempt a selection of a final intent (user intent, Fig.1, 108) from the one or more candidate intents (contextual intent, Fig.6, 606) received from the nodes, Fig.1, (Fig.2, S204, S206, S208 and S210; [0060] [0075 – 00801).
Yet, Last fails to teach the following: the components comprise a memory configured for storage of encoded instructions executable by at least one processor and at least one processor configurable by the encoded instructions to perform the method; the audio spoken by the user is an audio stream which has been recorded in a ring buffer; the language models are connected to the ring buffer, wherein the audio is received by the language models from the ring buffer; the candidate intent is associated with a selection frequency indicative of the number of times the candidate intent has been selected as the final intent; and the candidate intent is recorded in an intent buffer.
However, Garcia discloses a system and method for operating a virtual assistant to provide natural assistant interaction (Abstract) comprising the following: an electronic device further comprises a processor and memory storing instructions (programs) which are configured to be executed by the processor to execute a method ([0008]); an audio stream spoken by a user is received by the virtual assistant ([0250] [0251]); the audio stream is stored in a ring buffer (Fig.8, 812) ([0251] [0252]); and speech recognition is performed on the audio stream to determine candidate intents ([0260] [0263] [0270- 0272]).
Additionally, Kim discloses a method for processing dialogue (Abstract), comprising the following: a dialogue processing system further comprises a database storing the frequency of a intent selected for a user ([0011] [0105 – 0115] [0148 - 0152]); and the system further adjusts an initial intent classification score based on the selection frequency ([0010 – 0015] [0160] [0161]).
Moreover, Kawamura discloses an apparatus and method for performing speech recognition (Abstract), comprising the following: candidate intents/recognition results are stored in a ring buffer ([0036] [0069]).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve Last’s invention in the same way that Garcia’s invention has been improved to achieve the following, predictable results for the purpose of providing a temporary storage to manage the processing of the audio stream according to design requirements (acceptable delay, processor usage, etc.): the audio spoken by the user is an audio stream, wherein the audio stream is stored in a buffer, e.g. ring buffer; and the audio stream (audio spoken by the user) is further received by each language model from the ring buffer; and the components of the system further comprise a memory configured for storage of encoded instructions executable by at least one processor and at least one processor configurable by the encoded instructions to perform the method.
Additionally, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Last and Garcia in the same way that Kim’s invention has been improved to achieve the following, predictable results for the purpose of reducing the effects of biased training data used to train the language models (which perform intent classification) (Kim, [0008]): the system further comprises a database storing the frequency of a intent selected for a user; and the system further adjusts intent classification scores (Last, [0072 – 0078]) based on the selection frequency.
Moreover, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Last, Garcia and Kim in the same way that Kawamura’s invention has been improved to achieve the following, predictable results for the purpose of increasing user satisfaction with interacting with the speech recognition system by improving the response speed as well as recognition accuracy of a speech recognition system (Kawamura, [0019]): the candidate intent (recognition result) is further associated with a selection frequency; and the candidate intent (recognition result) is further recorded in a buffer, e.g. intent buffer.
For claim 2, Last and Kawamara further disclose, wherein the one or more recorded candidate intents consist of a single candidate intent (Last, Single candidate intent/recognition result has the highest recognition/confidence score, [0072 – 0078]), and wherein the arbitrator is configured to select as the final intent the single candidate intent (Last, [0078] [0079] [0080]) (Kawamara, Candidate intents/recognition results are stored in the buffer, [0069]).
For claim 3, Last and Kim further disclose, wherein: the arbitrator is configured to increment the selection frequency associated with the single candidate intent selected as the final intent (Last, [0080] [0081]) (Kim, A database comprising a frequency of intent used/selected by a user implies incrementing the frequency according to user/selection, [0105 – 0115] [0139] [0149] [0151] [0152])
For claim 6, Last and Kawamara further disclose, wherein each language model is configured to: determine a confidence level associated with each detected candidate intent (Last, [0072 – 0078]); and record the confidence level in the intent buffer (Last, [0072 – 0078]) (Kawamara, Candidate intents/recognition results are stored in the buffer, [0069]).
For claim 7, Last and Kawamara further disclose, wherein the arbitrator is configured to infer as the final intent the recorded candidate intent having a highest confidence level of the one or more recorded candidate intents (Last, [0072 – 0080]) (Kawamara, Candidate intents/recognition results are stored in the buffer., [0037] [0069]).
For claim 8, Last, Kim, and Kawamara further disclose, wherein the arbitrator is configured to store to the memory one or more of: the inferred final intent (Last, [0072- 0080]) (Kim, Fig.12, [0149])(Kawamara, Candidate intents/recognition results are stored in the buffer., [0037] [0069]); the sequence of phonemes associated with the inferred final intent; or the confidence level associated with the inferred final intent.
For claim 9, Last, Kim and Kawamara further disclose, wherein: the one or more recorded candidate intents includes two or more first recorded candidate intents sharing a highest confidence level (Last, [0153 – 0156]) (Kawamara, Candidate intents/recognition results are stored in the buffer., [0037] [0069]); and wherein the arbitrator is configured to infer as the final intent the first recorded candidate intent having a highest selection frequency of the one or more recorded candidate intents (Last, [0142] [0143] [0148] [0160]) (Kim, [0008 – 0013]) (Kawamara, Candidate intents/recognition results are stored in the buffer., [0037] [0069]).
For claim 11, Last, Kim and Kawamara further disclose, wherein: each recorded candidate intent corresponds to at least one voice command (Last, user intended action, [0012] [0080] [0113 – 0116]) (Kim, [0053]) (Kawamara, Candidate intents/recognition results are stored in the buffer., [0037] [0069]) executable by a controlled system (Last, third party system, [0116]) (Kim, vehicle, [0075] [0098] [0099]) operatively coupled to the speech recognition system (Last, [0113 – 0116]) (Kim, Fig.3, 1 and 2, [0098]); and wherein the arbitrator is configured to forward to the controlled system at least one voice command corresponding to the selected final intent (Last, third party system, [0116]) (Kim, vehicle, [0075] [0098] [0099]).
For claim 12, Last and Garcia further disclose: an input device (Last, Fig.5, 502) (Garcia, The audio is received at the ring buffer from a user’s device comprising a microphone, wherein the ring buffer is located at a server, [0006] [0040] [0042] [0048] [0056] [0249] [0250]), the input device configured for receiving the spoken audio stream from the user (Last, [0113] [0114]) (Garcia, [0040] [0042] [0048] [0056] [0251]).
For claim 15, Last discloses a computer assisted method for multilingual speech recognition (Abstract), comprising: receiving, via an input device (Fig.5, 502), input speech spoken by a user ([0114]); detecting, via each of a plurality of language models (nodes comprising an acoustic model, word recognition component, language model and intent recognition component, Fig.6, 104, 602, 604, 614, 616 and 618; [0048] [0060] [0061]) configured for parallel operation (Fig.5, 104a-104c; [0047]), each language model associated with a language (e.g. English, [0049 – 0052] [0062 – 0065] [0096]) and trained according to a plurality of words and phonemes associated with the language ([0049] [0050] [0059 – 0066]), one or more candidate intents of the audio by associating the audio with a sequence of phonemes corresponding to the trained language (Fig.2, S224 and Fig.6, 602, 612, 614, 618, 616 and 604; [0060] [0061] [0066] [0073] [0074]); and attempting to select or infer, via an arbitrator (context selector of a controller, Fig.5, 108 and 110) a final intent (user intent, Fig.1, 108) from the one or more received candidate intents (contextual intent, Fig.6, 606) (Fig.2, S204, S206, S208 and S210; [0060] [0075 – 00801).
Yet, Last fails to teach the following: the audio spoken by the user is an audio stream which has been recorded in a ring buffer; the language models are connected to the ring buffer, wherein the audio is received by the language models from the ring buffer; the candidate intent is associated with a selection frequency indicative of the number of times the candidate intent has been selected as the final intent; and the candidate intent is recorded in an intent buffer.
However, Garcia discloses a system and method for operating a virtual assistant to provide natural assistant interaction (Abstract) comprising the following: an audio stream spoken by a user is received by the virtual assistant ([0250] [0251]); the audio stream is stored in a ring buffer (Fig.8, 812) ([0251] [0252]); and speech recognition is performed on the audio stream to determine candidate intents ([0260] [0263] [0270- 0272]).
Additionally, Kim discloses a method for processing dialogue (Abstract), comprising the following: a dialogue processing system further comprises a database storing the frequency of a intent selected for a user ([0011] [0105 – 0115] [0148 - 0152]); and the system further adjusts an initial intent classification score based on the selection frequency ([0010 – 0015] [0160] [0161]).
Moreover, Kawamura discloses an apparatus and method for performing speech recognition (Abstract), comprising the following: candidate intents/recognition results are stored in a ring buffer ([0036] [0069]).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve Last’s invention in the same way that Garcia’s invention has been improved to achieve the following, predictable results for the purpose of providing a temporary storage to manage the processing of the audio stream according to design requirements (acceptable delay, processor usage, etc.): the audio spoken by the user is an audio stream, wherein the audio stream is stored in a buffer, e.g. ring buffer; and the audio stream (audio spoken by the user) is further received by each language model from the ring buffer.
Additionally, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Last and Garcia in the same way that Kim’s invention has been improved to achieve the following, predictable results for the purpose of reducing the effects of biased training data used to train the language models (which perform intent classification) (Kim, [0008]): the system further comprises a database storing the frequency of a intent selected for a user; and the system further adjusts intent classification scores (Last, [0072 – 0078]) based on the selection frequency.
Moreover, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Last, Garcia and Kim in the same way that Kawamura’s invention has been improved to achieve the following, predictable results for the purpose of increasing user satisfaction with interacting with the speech recognition system by improving the response speed as well as recognition accuracy of a speech recognition system (Kawamura, [0019]): the candidate intent (recognition result) is further associated with a selection frequency; and the candidate intent (recognition result) is further recorded in a buffer, e.g. intent buffer.
For claim 19, Last and Kawamura further disclose, wherein: detecting, via each of a plurality of language models configured for parallel operation, at least one candidate intent of the user includes: determining, via each language model, a confidence level of the candidate intent (Last, [0072 – 0078]) and recording, via each language model, the confidence level to the intent buffer with its associated candidate intent (Sung, [0050]) (Kawamara, Candidate intents/recognition results are stored in the buffer, [0069]); and wherein attempting to select or infer, via an arbitrator, a final intent from the one or more recorded candidate intents stored to the intent buffers includes: inferring as the final intent, via the arbitrator, the recorded candidate intent having a highest confidence level of the one or more recorded candidate intents (Last, [0078] [0079] [0080]) (Kawamara, Candidate intents/recognition results are stored in the buffer, [0069]).
For claim 19, Last, Kim and Kawamara further disclose, wherein: the one or more recorded candidate intents includes two or more first recorded candidate intents sharing a highest confidence level (Last, [0153 – 0156]) (Kawamara, Candidate intents/recognition results are stored in the buffer., [0037] [0069]); and wherein attempting to select or infer, via an arbitrator, a final intent from the one or more recorded candidate intents stored to the intent buffers includes: inferring as the final intent, via the arbitrator, the first recorded candidate intent having a highest selection frequency of the one or more recorded candidate intents (Last, [0142] [0143] [0148] [0160]) (Kim, [0008 – 0013]) (Kawamara, Candidate intents/recognition results are stored in the buffer., [0037] [0069]).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Last et al. (US 2026/0011328) (“Last”) in view of Garcia et al. (US 2021/0027780) (“Garcia”) , and further in view of Kim (US 2023/0335120), and further in view of Kawamura et al. (US 2014/0088967) (“Kawamura”) and further in view of Cross (US 2008/0235027).
For claim 10, the combination of Last, Garcia, Kim and Kawamura fails to teach, wherein the arbitrator is configured to store to the memory a similarity metric corresponding to the two or more first recorded candidate intents, the similarity metric based on one or more of: a similarity of at least one voice command associated with each first recorded candidate intent; a similarity of the sequence of phonemes associated with each first recorded candidate intent; or a similarity of the language associated with each first recorded candidate intent.
However, Cross discloses a system and method for performing speech processing (Abstract), comprising the following: storing a language associated with each recognition result generated by speech engine ([0108] [0112] [0123]).
Furthermore, the combination of Last, Garcia, Kim and Kawamura discloses that the similarity metric between the two recorded candidate intents is language (Sung, All of the candidate intents/recognition results which have the highest score are generated by the same language recognizer, [0050] [0059]) (Kawamara, Recognition results are stored in the buffer., [0037] [0069])
Therefore, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Last, Garcia, Kim and Kawamura in the same way that Cross’ invention has been improved to achieve the following, predictable results for the purpose of increasing user satisfaction with interacting with the speech recognition system by improving accuracy by comparing current recognition results with past recognition results: the arbitrator is further configured to store to the memory a similarity metric corresponding to the two or more first recorded candidate intents, the similarity metric based on one or more of similarity of the language associated with each first recorded candidate intent.
Claim(s) 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Last et al. (US 2026/0011328) (“Last”) in view of Garcia et al. (US 2021/0027780) (“Garcia”), and further in view of Kim (US 2023/0335120), and further in view of Kawamura et al. (US 2014/0088967) (“Kawamura”) and further in view of Kizuki et al. (US 2019/0147851) (“Kizuki”).
For claim 13, the combination of Last, Garcia, Kim and Kawamura fails to teach, wherein: the arbitrator fails to select or infer a final intent from the one or more recorded candidate intents.
However, Kizuki discloses a speech recognition apparatus and method (Abstract), comprising the following: an arbitrator (First server, Fig.2, 210 and Fig.3, 210; [0064] [0081]) fails to select or infer a final intent (The response selecting section fails to select a response because the language specific ASRs fail to provide recognition results which correspond to an intent, [0050 – 0055] [0064 – 0066] [0081 – 0083] [0091] [0093] [0094]); and the arbitrator provides a notification to a user based on the failure ([0083] [0084] [0086] [0096]).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of applicant’s filing to improve the invention disclosed by the combination of Last, Garcia, Kim and Kawamura in the same way that Kizuki’s invention has been improved to achieve the following, predictable results for the purpose of providing a technique that is capable of, even if the speech recognition fails, outputting a message in the language used by the operator (Kizuki, [0006] [0007]): the arbitrator fails to select or infer a final intent from the one or more recorded candidate intents.
For claim 14, Last, Garcia and Kizuki further discloses: an alert system (Kizuki, Asking response selection section and Communicating section, Fig.3, 115 and 323; [0029] [0030] [0081] [0083]) operatively coupled to the at least one processor (Last, Fig.5,506) (Garcia, ([0008]) (Kizuki, [0029] [0033] [0080] [0081]) the alert system configured for, when the arbitrator fails to select or infer the final intent, at least one of: alerting the user to the failure to select; and prompting the user to repeat the audio stream (Kizuki, [0083] [0084]).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 – 3, 6 – 15, 19 and 20 have been considered but are moot in view of the new ground(s) of rejection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Shukla (US 2019/0371318) (inventive concept of multilingual speech processing using parallel speech recognition)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SONIA L GAY whose telephone number is (571)270-1951. The examiner can normally be reached Monday-Friday 9-5 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at 571-272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SONIA L GAY/Primary Examiner, Art Unit 2657
Prosecution Insights